[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Crush 2.0 abandoned

On Thu, 6 Aug 2009 18:13:03 +0200
Simon Richter <sjr@debian.org> wrote:

One important point - it is Crush 2.0 that has been abandoned, not
Crush itself. Once multiarch is implemented and working, I'm happy to
work on Crush. Whether that is Crush 3.0 or 4.0 I have no idea, I doubt
multiarch will be ready and working for 2.0. Even if multiarch has
basic support in Debian 6.0 (Squeeze), I don't see that there is time
to mould the Crush build system to multiarch *and* fix all the bugs and
other issues before Squeeze itself is released.

Crush development is stalled until multiarch is fully implemented. I
should have made that clear - principally because I expect my health to
improve sufficiently by then that I'll be able to pick up development
of Crush once more. I'd be very pleased to find that others have joined
the effort in the meantime, of course, as that will make extending
Crush to more than one or two architectures achievable.

It's not nice to have a gap in the release history of any distribution,
but in the case of Crush, I feel that it is both inevitable and
reasonable - given the scale of the problems at this moment. Crush
(like Grip) does need to be tied to the Debian release cycle and I
really think it is best to aim for Squeeze+1.

Don't let the undoubted enthusiasm mask the size of the task. Hundreds
and hundreds of developer hours are needed and Squeeze (and therefore
Crush) could be in release freeze within 7 months. The release freeze
really needs to be about implementation fixes, not core structural
development. I wouldn't like Crush 2.0 to be worse than 1.0 - it was
hard enough dealing with 1.0, we shouldn't expect to release 2.0 in an
even worse state than 1.0.

> Hi,
> > > 2. Continue discussion on how xcontrol support can actually be
> > > implemented. Check how it is used in the CodeAudit and tie that into
> > > the developments at DebConf9.
> xcontrol is dead, long live xcontrol.
> In Cáceres, I had a (long) talk with Guillem Jover from the dpkg team. In
> general, we both want to get rid of the xcontrol file in the long run, and
> have the same information provided in the regular control file. There was a
> small multi-arch cabal meeting, where Wookey, Héctor and myself presented
> the emdebian use cases (cross compiling and toolchain bootstrap), which led
> to a small syntax change (as to not close doors) and a request to write a
> proper paper on what needs to be added to multiarch to support
> cross-compiling and toolchain bootstrap.
> Multiarch in general allows us to install packages for any architecture on
> a single host, provided they do not have file conflicts (with some
> exceptions) and the version numbers of a single package is the same across
> all architectures.

Until multiarch arrives, we're still going to need a new version of
xcontrol to implement the necessary package renaming support that is
necessary for Crush 2.0.

> So, this would allow us to get rid of dpkg-cross (as soon as all libraries
> are converted to multi-arch).

IMHO dpkg-cross (and therefore apt-cross) will still be core to the
generation of Crush 2.0. Correct me if I'm wrong but multiarch requires
at least one release cycle before it can actually help us - i.e. Squeeze
+1 which will be the basis of any Crush 3.0, should it be possible.

The state of multiarch in Debian was one reason for me choosing to drop
Crush 2.0 as the best way to fit my reduced available time into a
continued desire to do the best I can for Emdebian. The dpkg-cross,
apt-cross, xcontrol mechanism used for Crush 1.0 was horribly fragile
and extending it to cope with package renaming for Crush 2.0 is not
something I really want to think about. Debian is at a cross-roads as
far as cross-compiling is concerned, my concern is that the previous
system used for Crush 1.0 will not bear the weight of Crush 2.0 and
that the correct answer, multiarch, will not be ready in time for Crush

IF multiarch really can be implemented before Squeeze is frozen, then
it is still a lot of work to redesign the entire Crush build system to
work with it and then fix the bugs that are likely to result. If those
bugs turn out to be in the multiarch system itself, it means that there
is even less time to do anything about fixing them as multiarch support
is not likely to be an RC bug in Squeeze, certainly not if the bug in
question primarily affects cross-building support.

> > > 6. Create a local mirror and use the scripts in emdebian-qa and
> > > emdebian-tools to upload cross-built packages into the repo and start
> > > building a small set of packages for armel. Don't proceed to other
> > > architectures or make the mirror publicly accessible until the rest of
> > > the issues in the Code Audit have been fixed.
> Basically, we want an autobuilder that cross-compiles packages and
> something lintianish to check whether the resulting packages are what we
> want (specifically, if the generated binaries match the target
> architecture).

Those scripts (with lintian and edos-debcheck support) already exist in
the emdebian-tools and emdebian-qa packages, it remains to fix whatever
bugs arise when extending to more than one architecture and more than
one buildd etc. The autobuilder used for Crush 1.0 was one of my home
machines and that will not be available for Crush 2.0, it will be busy
with Grip.

> > > 8. Start developing ways to test and fix an existing problem in that
> > > the ./configure scripts in some packages insist on looking for stuff
> > > in /usr/lib/ and then linking against the shared libraries
> > > in /usr/arm-linux-gnueabi/lib/ etc.
> In principle, the configure script should never look at any path directly,
> but rather trust the compiler.

That is the principle but it does not actually happen. It was to be the
next component of the CodeAudit but no actual data yet exists on just
how many packages break this principle - I just remember that there are
more than a handful and less than a majority.

The difficult part is that when fixing cross-builds, it's also useful
to be able to check the build using the native compiler. So, the tests
need to be done in clean chroots. Not a big hurdle but it all adds to
the time required and time is short before the Squeeze release freeze.
(It is already down to half the time it took to get Crush 1.0 into a
releaseable state.) During the Lenny release freeze, I was mainly
working on implementation bugs and also on Grip. 

In total, I was putting in about 20-30 hours a week just into Emdebian
Crush and Grip for about a year.

The biggest problems will come with the most important packages, like
gcc and gtk. Thankfully, the new libc6 source is much easier to handle
(until you start thinking about solving the gconv problems) as is
tzdata. GTK is still borked due to problems in apt-cross. See bug #
502433 and last time I tried, gcc-4.4 does not build at all for Crush.

> > > 10. test and document the proper support for CC_BUILD in autotools.
> IIRC the GNU project uses "CC_FOR_BUILD".

AFAICT it does but in a completely undocumented manner and not all
packages that do implement it do so in a consistent manner.

> > > 26. Finalise the cache value support in dpkg-cross and add to it if
> > > necessary. Consider deploying the cache values into the Debian package
> > > such that the files can be created at build time on a
> > > per-package-per-architecture basis.
> I'd rather fix the configure scripts for good than hack around them. For a
> lot of packages, simply rebuilding the script with current autoconf is
> sufficient (for example, current autoconf knows to determine endianness by
> analyzing the binary with objdump rather than running it).

Fixing the configure scripts means fixing the autoconf macros which
will take time. That won't happen for Crush 2.0.

I respect the enthusiasm to try and fill the gap but even before my
illness I had begun to doubt that Crush 2.0 would be possible. IMHO it
is certainly unlikely that Crush 2.0 would be able to support more than
just armel - possibly i386 if another two or three people start work on
it now, but I really don't think that multiarch is going to help Crush
2.0 at this stage, not with the amount of available time left before
the Squeeze release freeze and the requirement for multiarch to be
supported by dpkg in Squeeze before being tested in actual packages in


Neil Williams

Attachment: pgpkI491fDGBo.pgp
Description: PGP signature

Reply to: