On Sun, 7 Nov 2010 14:04:21 +0000 Luke Kenneth Casson Leighton <lkcl@lkcl.net> wrote: > On Fri, Nov 5, 2010 at 4:35 PM, Hector Oron <zumbi@debian.org> wrote: > > * Handling 'flavours' (ISA and optimisation options) (e.g. ARM > > v5/v6/v7/VFP/Neon, x86 i586/i686/MMX) within Debian (dpkg support, > > package metadata, partial archives, multilib gcc/install paths, > > multilib- or flavoured- cross-tools) Partial archives are the simplest solution here - any one flavour will need to have entire collections of packages all built with the same flags and options. Therefore, these should go into different components (as Emdebian does with main, dev, debug, java - we could add components for flavour names). Alternatively, the entire mirror gets a different location - much like debian-ports. > i haven't looked around, but has anyone suggested adding a > base64-encoded extension to the .deb name, containing a bit-field of > "capabilities"? This needs a *lot* of support in all the packaging and mirroring tools. It's not going to happen. > the problem with splatting the capabilities into the > .deb itself is that you then end up with dozens of combinations of > capabilities, all with the exact same .deb filename. That's not a problem as long as the .deb files can live in different directories or locations. It's easier to identify pool/neon vs pool/main than to decode some bitmask. Packages that don't benefit from the tweaks can be pulled from the main component by the mirroring tools. `apt-cache policy` will show the rest. > by encoding the > capabilities in the actual filename, you can still pick a particular > package by its version number etc. etc. _but_ you can identify whether > it will work on a particular CPU by decoding its capabilities bits. Better to have the same filename in different locations and then use the apt sources to pick the best match. deb http://somewhere/debian unstable myflavourname > and for x86, because, duh, you _so_ aren't going to get x86 > instructions on ARM CPUs :) All the more reason to put these into separate port mirrors. > > * Cross compilation environment (cross toolchains and target > > packages) - bare metal support > > the level of expertise / machine-specific information is so vast that > i strongly advise against "beginning from scratch". whilst wookey (or > neil?) pointed out a few months back that it took several months - > almost a year - to do the armel port, most of which was done by hand, > and someone else pointed out recently that they managed in a couple of > weeks to recompile automatically several hundred packages for Cortex > A8, gaining a 40% performance jump by doing so, this is still an > insanely ridiculous situation... when you compare it to openembedded. ... which hardly ever works to completion ... We've done some more experiments with Yocto and the perennial problem persists: whether you get a complete build appears to depend on whether you're wearing a lucky armband or whether /dev/random on your box precisely matches /dev/random on the developer boxes at OE. > i believe that a sensible and sane approach to the dog's dinner mess > of doing cross-compiling for dozens of targets and hundreds of > machines would be to part-replace the capabilities of buildd with the > capabilities of bitbake. in other words, write a bitbake recipe for > openembedded that understood about debian/rules, dpkg-buildpackage and > the debian cross buildpackage tools (note: openembedded already > understands .deb packaging). NO!! That way lie dragons! Only cross-build if functional changes are essential, otherwise use the native builds - binary compatible so that you know where the bugs lie. Debian does not have support for tiny embedded distributions - this is why Crush was so much work - and the more variants you want to support, the less work each variant *must* require to maintain it. Therefore, it has to be as binary-compatible with Debian as possible or the maintenance becomes prohibitive. Removing coreutils and/or perl from Debian is a pipe-dream. We tried it with Crush, it failed and I don't see any chance of large scale cross-building support in Debian until after Wheezy is released. Select the packages with extreme prejudice, cross-build only those ones and it can work. More than that you get to rearrange these words into a sentence: "for own rod your back created you a have" > now, that ain't gonna cut it for the _entire_ suite of debian > packages: some of them simply aren't going to cross-compile as-is. > two things help solve that: 1) openembedded has the ability to use > qemu to configure packages! yes, you actually end up running _really_ > native (only not really) via command-line qemu. 2) there is precedent > for "OS-specific" debian/rules #includes (from the painful ubuntu) if > you're absolutely desperate but i imagine that qemu pretty much solves > it except in extreme rare cases (for example, that package mentioned > last week which requires 1gb RAM to compile: current stable versions > of qemu only do up to 256mb RAM, damnit) Emulators are never the same as the real device - that way just adds a new source of bugs. > as far as cross toolchains are concerned: openembedded has it all, > done already. why the f*** would you want to "start from scratch" > when openembedded has everything that's needed?? umm, because OE doesn't work on Debian? (for more than about 0.001% of attempts) (Out of a few dozen test builds over the years on various Debian boxes, I have never had a 100% successful OE build. Not once.) > question: why not recommend portage / gentoo instead of openembedded? > answer: because openembedded has over one hundred ARM platforms > supported already, with ARM CPU variants dating back over 10 years, > and gentoo um... well... i don't have to look it up to know that it's > not going to have anything _like_ the level of support for ARM CPUs. Native is increasingly the only answer - except in the few specific cases where a functional change is essential or the package cannot go into Debian itself. Look beyond simply building the packages. What is the point of adding a new source of bugs just because you cross-built the package in some weird emulator instance only to find that the package behaves differently on-device? How do you debug quickly when you can't use the same changes on your development machine? Use a core of packages for which the bugs are known and you have a platform upon which you can put your bespoke stuff. The further you deviate from the common ground, the more work you have to do to keep up. It's OK if all you want is a kernel and busybox but if you're going to add an Xorg server, 90% of your packages should come unchanged from some standard set with lots and lots of users and maintainers, not from a tiny subset of cross-built hacks. Yes, we created Emdebian Crush with huge amounts of effort. What people might forget is that it simply didn't work properly at runtime. Cross-building 700 packages added several dozens of new bugs which ALL disappeared when the device was switched to Emdebian Grip. There is an inescapable logic in that result. -- Neil Williams ============= http://www.linux.codehelp.co.uk/
Attachment:
pgpzi62_cFYro.pgp
Description: PGP signature