Re: The future of an SCC that has been given up on
Thorsten,
> >> Packages like emile that build for all architectures, so you can build
> >> e.g. bootable Mac CDs on i386. But maybe LP can do that for us.
> >
> >I still don't get it - such packages should be submitted to Debian as wishlist
> >packages for inclusion, from there it's all taken care of, no?
>
> No, since Debian no longer accepts packages that are only of use
> only to m68k users (even if those packages are run on !m68k systems),
> if they have other issues (like needing a special toolchain to build,
> or build their arch:all part only on m68k, i.e. aranym would be kept).
OK, so we'll have to maintain them entirely
> >> Uhm, I don’t know what you mean here. Debian/m68k currently uses TLS
> >> via SYS_333 syscalls.
> >
> >And CF uses something different?
>
> There is no Debian/m68k on CF ;-)
But CF was touted as the way out in relation to the 'you can still buy the
hardware' rule. Efforts were underway to unify the CF user space and Debian. I
might have misunderstood though.
> >As I recall it, TLS (in particular VDSOs) were touted as a way of creating a
> >uniform userland across m68k and CF. Ignore me if I'm wrong.
>
> VDSO ≠ TLS
>
> But yes, a VDSO could solve a whole heck of issues (things like, using
> the right cmpxchg sequence, which CPU flavour do we run on?, fast lookup
> of the TLS map base, oh and, classical, fast time).
OK, so no unified user space in view anytime soon.
> >Creating a sub-list of important stuff that is holding back major portions would
> >be preferred (IIRC Stephen did use to provide that).
>
> Right now, that’s one thing: X.org (and subsequently, the whole shit
> fd.org/gnome stack). I’ve done everything else keeping us back manually.
As Geert has mentioned, X lacks support for the Atari abd Amiga frame buffer
data formats. While I think X could be built (sorting out the correct build
order of all pieces up front) I doubt it will be useful before the frame buffer
support has been readded. Even then - I have maxed out my Falcon with 512 MB RAM
and that won't nearly be enough for a modern desktop system. Might just run with
loads of swapping but not really useful.
> >Packages are only flagged dep-wait by the appropriate log reply to buildd (last
> >I dealt with it).
>
> Could you please check if that is still so? At least on stock Debian,
> I believe otherwise. We might need to invest some into d-p.org infra-
> structure, but the main archive looks like it has what’s cool.
I'd have to check what the current w-b code looks like. Downloading sbuild 0.59
source it does seem to use the same merge-quinn module like it always did, i.e.
packages are placed in state needs-build when the installed binary version is
older than the source version, regardless of whether the necessary dependencies
are fulfilled.
buildd-tools and wb team - is my assessment correct? Do more recent versions of
sbuild take missing or uninstallable build dependencies into account, and set
the package state accordingly?
> >Manually editing package state was extremely awkward. We cannot afford
> >trying every package just to find out that 60% are not buildable
> >because of broken or uninstallable dependencies.
>
> Right. Can such edit process be automated? Possibly hack something
> with edos-debcheck. (Of course, B-D trial on the W-B side would be
> infinitely preferrable.)
I'm confident it can be automated - once the new package versions have been
registered with w-b a separate process can inspect each package to build and
change the state if needs be, with detailed depencency information according to
the results of the check.
> Ah. While at it, I’d like to get a way to tell W-B to take its hands
> off certain packages, e.g. while I’m building them, or while there’s
w-b --take (IIRC). Then just sit on it (while building or not). If you really
want to push it, run a cron job to take new versions as they appear (overriding
other takers as needed).
I presume you do have ssh access to the w-b running on d-p.org, of course.
> a newer version in the archive that would build cleanly but lack the
> platform specific patches because I’ve not sent them in yet (gcc, at
> the moment) or they weren’t included yet (gdb).
Same - though you could set those ones not-for-us as well (and clear that state
once the patches are in).
Since we are going to use our own w-b for the forseeable future, a --hold could
be added that will ignore future versions as well.
> >Most notably because about 10% of these attempts will leave the chroot
> >in a broken state in need of manual cleanup.
>
> Ouch! Is sbuild so fragile? (If so, is it *still* so fragile?)
I don't know - haven't tried in a long time.
> In which case you should probably invest on doing throwaway-chroot
> builds. I know for sure that several official buildds are running
> that, while others still do the removal of B-D after the build.
Good point.
> >I'm sure I'm slighlty exaggerating here but I hope you catch my drift.
>
> Sure – I’m just saying this is not necessarily your grandfather’s
> Debian any more. In fact, with all the recent work and the very
edos-builddebcheck does make things easier but it would need to be run in
concert with w-b. That could help a lot. Otherwise, it sure looks like what I
knew and hated five years ago.
> active kernel development (not to forget gcc development, but that
> doesn’t contribute to this), m68k has been BETTER than some release
> architectures in some parts!
>
> >The 'throw packages at them manually' would imply a w-b database of some sorts,
>
> No. You can tell sbuild to build a package. I’m not too clear
Yep, and that was always awkward. You could even send mail to a buildd asking it
to build a particular package (slightly less awkward, haven't really seen it
used except for security builds IIRC). You still need to know _what_ packages to
throw at it that have a more than even chance of succeeding. I hate to belabor
the point but that is what I still see as the main holdup.
> about the specifics because I think sbuild is still from when
> m68k was the only “other” architecture, and cowbuilder is in-
> finitely more clear to use (but still shows its age).
sbuild was used by me on m68k and powerpc. At a time when the number or
architectures had climbed into the scary zone.
Most may have switched to cowbuilder since, which seems to paper over the
problem with trashing the chroot somewhat. For all I care, use cowbuilder as
back end.
> >Uploading logs should be as easy as setting up a mail server to archive them, at
> >least that's what we used to do.
>
> Ouch. My three fastest VMs run on my workstation at work, and
> due to our shit new firewall policies I can’t mail out from
> them. Not even to submit@bugs.d.o which sucks. Also, sending
> mails is a total waste of bandwidth… IMHO, there should be
> some rsync-over-ssh (or at least ssh, with compression) pickup
> site.
KISS. Mail no longer being a viable option sucks whales through nanotubes but
don't switch to something that sucks whales through monoatomic layers. We want
something that makes it easy to add new build hosts without extensive key
exchange. The requirement for package signing keys is enough of a hassle really.
IMHO mail incoming is a must unless you want to have your buildd poll the
database all the time.
Mail outgoing or log uploads might be done some other way, see below.
> >Uploading the packages to d-p.org should still work I hope.
>
> That’s what I’m doing all the time.
What protocol does that use? We could possibly upload logs the same way - they
can be moved into a suitable directory that can be browsed by the old build
logs interface if the log name contains sufficient information identifying
architecture, package and build host. Compressed if you prefer. IF it cannot be
implemented on d-p.org it can be handled via a m68k upload queue host.
> >> Does dpo support autosigning in buildds nowadays, btw? That would
> >> remove the need to inspect maybe-succeeded builds, in most cases.
> >
> >I don't know - autosigning (at least semiautomatic) was never the major holdup.
> >I still have script extracting the changes file from the mail stream and mailing
>
> Mh. Such a scenario won’t work for my specific (private) mail setup,
> either, so I’m not volunteering as buildd manager. (It’s way too
> mutt specific.)
My solution was pine specific. I could imagine something based on
fetchmail/procmail (actually the extraction of the changes file always ran on
procmail in my setup). What I'm trying to say - with a minimum of glue scripts
you can automate buildd operation a great deal. Filing meaningful bug reports or
crafting meaningful fail messages does require a well trained AI though. We do
have tools to report on build dependency problems now, that should take some of
the grunt work off the task.
> >I've just pushed out my 3.4 patches to linux-m68k. The EtherNAT patches should
> >apply to 3.2 with a minimum of work. The obsolete old EtherNEC one likewise -
>
> Are you willing to maintain them inside Debian’s 3.2 as well, which
> includes tests on the hardware? If so, contact the kernel team. They
> do accept patches, but usually only if they’re already in Linus’ tree
> in some form.
I can only test on one particular variant of Atari. Hardly any of the
interesting bits are in Linus' tree yet. Unless the kernel team is willing to
pull patches from the m68k tree, I don't think it will be easy.
> >I'll look into what's missing (does anyone at the Debian kernel team maintain a
> >git repository with all the Debian patches as separate commits on top of
> >whatever the base release branch in 3.2 would be? That would save a lot of
> >manual patch/commit work ...).
>
> There’s an SVN (waldi’s working on converting to git) with the packaging.
> As patches are applied in order, conditionally, and even UNAPPLIED in
> between versions, and Debian kernels always ship all patches that were
> ever applied to a specific upstream version, a single git branch to work
> against is not possible.
One release branch per architecture, fetching from the respective master
repository, rebasing each branch after fetch and merging cherry-picked commits
from the Debian development branch into each release branch as needed might work.
> What you _can_ do is run something like 'debian/rules ARCH=m68k patch'
> or something (would need to look up the exact command) to get a tree
> with all patches applied, then move the .git directory from a vanilla
> checkout into it, 'git add .' and 'git commit -m "add debian patches"',
> then you can hack on this branch, and then do a format-patch only back
> to the commit making a debian tree out of a vanilla tree. This is how
> I occasionally worked. DO NOT MERGE v3.2-m68k on top of it and make a
> format-patch from that. Instead, cherry-pick or rebase or 'git am' and
> make a format-patch from that – otherwise, the patches WILL NOT APPLY
> because their baseline would not be correct.
Something like that, yes. I'd have applied the Debian patches on top of the
matching baseline vanilla git checkout, adding one commit per patch, and then
cherry picked anything not in mainline from Geert's tree. I'll have to think
about it a bit. Anyway, if the kernel team does not want to acceot patches
outside Linus' tree the whole thing is pointless and I'll set up a git server
for the Atari branch somewhere.
> schmitz dixit:
>
> >> Some time ago (last year or so?) I converted vivaldi and arrakis to use the
> >> newer kernels. Maybe I should do some upgrade again, but basically these
> >> machines are running for some time now without doing actual work as a
>
> Which kernels are they running?
Ingo's machines? Some recent 3.2 Debian release kernel IIRC.
> > Rebuild the chroot systems from scratch based on Thorsten's current repository,
> > I'd guess. I've lost my notes on how to use debootstrap to install a chroot
> > long ago. Maybe there's something useful in the buildd/sbuild docs these days.
>
> Just use the base.cow as chroot. It’s the result of a clean debootstrap.
> (Purge the wtf-debian-keyring package inside first, to make it contain
> only original Debian and Debian-Ports/unreleased matherial. Then run
> apt-get update and apt-get --purge dist-upgrade --no-install-recommends
> inside, then install any additional packages needed by sbuild, if any;
> the chroot contains --variant=buildd, ie. minbase+build-essential.)
I'll give that a shot - in fact I'll install that one on my Falcon now :-)
Again, I will not have much time for any of this in the near future. I'll check
back in September or so; by then I should have a good overview how hectic the
next year will be.
Cheers,
Michael
Reply to: