[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Crush 2.0 abandoned -- Review and Feedback from Neil's Post of August 7th

Hello Everyone,

I have been keeping an eye on this list looking to see what additional feedback and comments there might be about the list Neil posted a week ago.

Clearly, everyone is busy and at the moment this project is stalled as Neil must now attend to other important personal matters. However, before setting this project aside, he has taken the time to post his own suggestions of how things might move ahead.

Now, some of you have expressed an interest in a few areas mentioned in Neil's post, but we haven't heard from many people who I presume, based on your membership on this list, have a stake in the project successful outcome.

I'm going to ask the members of this list who haven't as yet spoken up about the items on Neil's post to please do so by Friday.  Once we have those idea, suggestions, and recommendations we can put those ideas into a draft summary.

Thanks again to everyone.



On Wed, Aug 5, 2009 at 3:53 PM, Neil Williams <codehelp@debian.org> wrote:
On Wed, 5 Aug 2009 14:52:22 +0000
Prince Riley <wmarketing3@gmail.com> wrote:

> Neil,
> We are all very glad and appreciative of the great work you've done
> so far. We all understand and respect your decision and wish you the
> best as you try to recover your health.
> My question, and one I'm sure has also occurred to others interested
> in this project, is how do we move ahead during your absence. Perhaps
> you can offer some advice or recommendations on how members of this
> list can help you transition your efforts so we can maintain the
> project's momentum and achieve its goals.

It isn't about helping me transition anything, someone else needs to
take over maintenance and development. This is the "bus" scenario -
whereby a project has had only a single active developer and once that
developer is lost, it's about going back to first principles and
picking up the threads. I can help in small areas but my time is too
limited to go through the details, just the overview.

Seeing as you asked, maybe this little list will give you some idea of
why I had to abandon this particular task - I'm not convinced that the
problems can be fixed before Squeeze, even if someone else takes on the
role. Several stages rely on other tasks within Debian that may or may
not make it into Squeeze. The list is by no means complete either! Even
before my illness, I had *serious* doubts about whether Crush 2.0 was

0. You'll find plenty of hacks, errors and mistakes that have remained
in the current scritps from the earlier development phases. It is worth
revisiting things like that and seeing what breaks without them. Ensure
you test with more than a few packages - there are various gotchas that
only apply to a handful of packages and some of those packages have
since had new upstream releases that may well have fixed the issues
that led to the hacks in the emdebian-tools scripts. Other improvements
elsewhere could also lead to some procedures within the current scripts
becoming redundant or new ones needing to be devised.

1. Read the EmdebianCodeAudit wiki page and supporting pages, go
through and cross-build each package to work out what each audit point
means. Results are for armel, using emsource and emdebuild. Replicate
results using chroots builds (with empdebuild) wherever possible.
Various other parts of the wiki also need to be cleaned up, as does the
website itself which still has a fair bit of outdated information.

2. Continue discussion on how xcontrol support can actually be
implemented. Check how it is used in the CodeAudit and tie that into
the developments at DebConf9.

3. Learn the emdebian-tools scripts and manpages, read the source code
and get a handle on how things currently work - taking over
maintenance of a native package means taking over the source code too,
in effect the "upstream".

4. Struggle with the inherent weaknesses in apt-cross which lie in the
horrible apt perl bindings - the most recent version of apt has (once
again) broken these bindings and apt-cross needs a patch to continue
operating. (Something to do with /etc/apt/preferences.d/). Hope that
someone gets apt to understand multiarch so that this whole issue can
be removed - you need a C++ expert for that role.

5. Get the toolchains back into installable state - I can't upgrade my
own armel boxes today but I don't have time to investigate why. Build
scripts that ensure that this persistent problem is fixed for the long
term and across all supported architectures.

6. Create a local mirror and use the scripts in emdebian-qa and
emdebian-tools to upload cross-built packages into the repo and start
building a small set of packages for armel. Don't proceed to other
architectures or make the mirror publicly accessible until the rest of
the issues in the Code Audit have been fixed.

7. Pick up development of emdebian-tools from existing SVN - there are
some changes in SVN that still need testing.

8. Start developing ways to test and fix an existing problem in that
the ./configure scripts in some packages insist on looking for stuff
in /usr/lib/ and then linking against the shared libraries
in /usr/arm-linux-gnueabi/lib/ etc. This is a disaster waiting to
happen and must be fixed before Crush can progress to more than one
architecture. In the same manner, check builds for /usr/include/
pollution and drop the apt-get build-deps stage in a clean chroot. If
no xcontrol file, allow (nay require) native deps.

9. Make notes in CodeAudit of packages that fail to rebuild twice in a
row and detail why.

10. test and document the proper support for CC_BUILD in autotools.

11. Get Build-Depends split into debian/control and normal Debian?
Possibly as well as Depends-Install (installed on build) and
Depends-Runtime ? Building a *static* chroot for another machine, only
needs runtime stuff. Allows native cross-versions of packages, smaller
installed images. No way to put a prefix in front of maintainer scripts
for config files. Need a mangling frontend in dpkg before calling the
script. Install time scripts vs runtime scripts. (A topic briefly
discussed with Wookey, hopefully my notes aren't too brief for the idea
to be reconstructed. I knew what I meant at the time.)

12. Create a new wiki page to detail things like this and whatever else
comes up as you go. Link it from the EmdebianTracker page?

13. Please don't come to me continually for input, if Crush is to
continue, someone needs to take over maintenance and that means filing
and fixing the bug reports without looking at me to do it. (You'd alter
the Maintainer: field in debian/control to remove my name and enter
your own so that the bug messages for the emdebian-tools source
package go to you or to this list.)

14. Work out how to fix "X-Build-Cross-Libtool: yes" which is probably
tied into other libtool issues identified in the CodeAudit to do with
refreshing the autoconf and libtool metadata prior to attempting the

15. Get CMake build systems cross-building properly.

16. Get a mini-perl and some kind of python support.

17. Solve what ever other issues turn up during these processes.

18. Fix all the packages that used to cross-build for Crush 1.0 but not
longer cross-build in unstable. (The CodeAudit identifies these
packages, one is gcc-4.4). Work out and gain consensus on whether such
packages should *not* be cross-built from source and just "borrowed"
directly from Grip. This is a Policy decision about whether Crush
should be 100% buildable from source or whether compromises are
acceptable. Mixing Crush and Grip will *only* work if the xcontrol
support is fixed such that changed functionality is always encoded
within the binary package name like libgconf-noldap-2-4 etc.

19. Identify *new* packages that have become dependencies of necessary
packages in Crush since the release of 1.0 and then work out how to get
each of these to cross-build.

20. Test these packages with real systems and see if the rootfs scripts
can still operate and what new bugs arise at installation/deployment

21. Manage the repository to cope with britney (the script that
migrates packages from unstable into testing in Debian).

22. Manage the repository to cope with testing-proposed-updates once
the Squeeze release freeze starts to happen.

23. Test and report bugs with the locale support via Emdebian TDebs and

24. Fix issues like the bulky gconv files in libc6 and test with the
tzdata support.

25. uClibc support

26. Finalise the cache value support in dpkg-cross and add to it if
necessary. Consider deploying the cache values into the Debian package
such that the files can be created at build time on a
per-package-per-architecture basis.

27. Keep an eye on multiarch developments and feed into the process to
ensure Debian gets a usable cross-build environment based on multiarch.
Get dpkg-cross merged into dpkg as soon as multiarch is supportable.

28. Whatever else I've forgotten.

Is that enough for now?


Neil Williams

Reply to: