On Sat, 25 Apr 2009 17:34:11 +0200 Martin Fuzzey <mfuzzey@gmail.com> wrote: > My understanding of the emdebian vision is that someone wanting to > build a linux device would: > > 1) Port the linux kernel to their hardware (including writing any > specific device drivers) > 2) Select the _prebuilt_ emdebian packages needed to support their > application > 3) Package their application as debian package(s) > 4) Build a root filesystem using emdebian tools from 1,2,3 Agreed. (Feel free to put that on the Wiki somewhere - indeed, a lot of your points could go on a Wiki page to help others.) > For me the essential difference between emdebian and other contenders > (buildroot, open embedded) is that emdebian is based on binary > packages whereas the others are source based. Definitely. My original problems with OpenEmbedded always tracked back to difficulties actually building stuff when all I wanted was to test some prebuilt stuff. Debian is binary-package based and Emdebian follows that model. (We don't have the manpower to alter it.) > So emdebian is like debian rather than gentoo... Yes. > However the two things I currently believe get in the way of this > vision are : > a) The restricted set of architectures (crush is only available for > ARM) b) The limited set of available packages. > > a) is being actively worked on so I won't say any more about it. > > However I think there is a problem with b) because I don't see how the > currently small emdebian team can possibly be expected to provide all > the packages that may be required. Actually, a) and b) are interlinked. The solution for a) will also help solve b) because allowing more than one architecture to be supported means providing sufficiently simple build mechanisms that multiple architectures can be built for the same packages. It turns out that such mechanisms also make it easier to build new packages. (The fix for problem A is the Emdebian Code Audit.) > Unless I am mistaken crush 1.0 is currently a (small) _fixed_ subset > of lenny cross compiled with modifications where required to reduce > footprint (remove perl, ...). The set isn't fixed, other than that those are the only packages we've managed to get cross-building in the current build system. The fix for problem a) - fixing the build system will make it easier to fix b). The set was determined by my own needs and the dependencies of those packages. One problem that will recur is that when you have a set of packages and want to get to a state where you can install another set of packages, it is very difficult to work out which packages you are going to build, which ones you will modify to drop dependencies and which ones will turn out to be buildable but not actually used. i.e. there is a lot of wasted effort involved. 'empath' was one attempt to solve this problem but that codebase is moribund due to recursion problems. > There doesn't seem to be any process to > allow contributions of "emdebianisations" of _lenny_ packages, only > _sid_ ones. But I'm not at all sure people developping a linux > embedded device want to be running sid any more than a system > administrator would want to run it on their production servers. A few issues here. 1. Yes, Emdebian is Debian, not Gentoo or OE. One of the core features of Debian is that the *default* build environment for all those binary packages is *Sid* and that migrations into testing and finally into stable primarily happen involving pre-built packages that were originally built against Sid. There are carefully managed mechanisms in Debian to allow packages to be uploaded directly to testing and stable but these are intensely labour intensive, relatively complicated compared to standard Sid builds and are only used when essential - i.e. to fix release critical bugs. Out of 20,000 packages that went into Lenny for Debian, I'd guess that it was about a hundred or so which went via testing-proposed-updates. As a proportion, the number of packages that would have done the same in Emdebian Crush is close to one out of the 700 which migrated automatically from sid builds - and one package did go into Crush 1.0 directly, busybox. It was a very complicated and difficult thing to do and was only done because it was a critical bug fix in Crush (which has still not been fixed in Debian). 2. Emdebian has none of the infrastructure or manpower of the Debian Release Team or Stable Release Managers. (Frankly, we don't have the experience of either of those teams either.) These migrations are fraught with difficulty and potential bugs. 3. Cross-building for testing or stable is immensely more difficult than cross-building for sid because all the dependencies change. Until Lenny was released with (an old version of) emdebian-tools available, it was all but impossible. Lenny has made that achievable. 4. Since Lenny was released, there hasn't been time to fix the build system and the build mechanisms used for Emdebian Crush 1.0 are only usable for the builds you want - cross-building packages for Lenny on Lenny. There are known bugs in emdebian-tools 1.4.3 - backporting 1.6.0 might be manageable but not later versions. However, the patches themselves have changed (as a result of the Code Audit) to work with emdebian-tools 1.9.0 and later. The existing packages might be upgradable using the existing sources in the repository but patches for new packages are going to be out-of-tree and that has different problems. A branch in SVN is probably the simplest method but care will still be needed. 5. All the builds for testing-proposed-updates in Debian happened when Debian unstable and Debian testing were all but the same - during the Lenny release freeze. *Since* that time, unstable has changed beyond recognition. Updates for Emdebian Crush 1.0 *must* be built exclusively on Debian Lenny. > Sure nothing prevents me or anyone else emdebianising lenny packages > in a private repository but if everyone does this it will lead to > lots of duplicated effort and an overall lower adoption of emdebian > since the entry barrier is higher. > > So I am envisionning emdebian-crush-stable, emdebian-grip-stable > which both consist a subset of lenny (same package versions) but > where that subset expands over time as people contribute to "scratch > their itch". Grip already has stable-proposed-updates and support for that in the scripts. Grip is so easy to manage that anyone only has to ask for a package to be added to stable-proposed-updates. (Although I probably haven't stated as much before.) Debian doesn't add lots and lots of packages with updates to stable releases but I think Emdebian can get away with doing that in the 1.0 release as it's such early days. I don't think we should expect that to be possible in future stable releases. > So in this scheme: > * debian - defines set of packages that constitutes a stable > release and hence the maximum theoretical set of emdebian Yes. > * emdebian developpers - work on tools, infrastructure and core > packages and maintain repositories Shortage of time for this currently. > * contributers - submit emdebianisations of stable packages. For Crush, possibly. > I guess the problem with this scheme is that it will mean more > packages to migrate when the tools change... I don't think we should be considering changing anything in Emdebian Crush 1.0 other than via migrations from stable-proposed-updates. That does not include changing the entire build system beneath a stable release. Migrations are difficult to do well, yes. There are tools that can help but the risk of completely breaking a stable release would be high. Hence, updates need to go into stable-proposed-updates and be available for testing without replacing the package released in Crush 1.0 until Crush 1.0.2 or later is ready. > Just my 2c > > Martin Thanks - some very clear summaries. Hopefully, the problems outlined won't put you off actually getting some builds done using Debian 5.0 "lenny". Please be careful and consider the points above to minimise risks of breaking a stable release. -- Neil Williams ============= http://www.data-freedom.org/ http://www.linux.codehelp.co.uk/ http://e-mail.is-not-s.ms/
Attachment:
pgpFRTfnW0q3W.pgp
Description: PGP signature