Re: [sprint] openembedded / dpkg-cross-buildpackage hybrid
Dnia poniedziałek, 22 listopada 2010 o 21:04:32 Luke Kenneth Casson
Leighton napisał(a):
> ok, i've been working with openembedded continuously now for a good
> couple of weeks, so have a better handle on what it is, can, and
> cannot solve.
I will extend few points from my perspective as OpenEmbeddeded developer
(since 2004) and Debian (rather advanced) user (since 1999). I know how
to build Debian packages, alter them, know how buildd works from user
perspective (I have few packages in Ubuntu).
> a comment was made the last time i mentioned the idea of combining
> openembedded with dpkg-buildpackage, which was, to roughly quote,
> "openembedded is source-based whereas debian is binary-based". i
> made a rebuttal of this at the time but i had no idea that
> openembedded actually creates -dev packages "by default" as well as
> creating cross-compile corresponding "-dev" packages at the same
> time!
Basically OE has more granulated packages then Debian ones. You get -
dev, -dbg, -doc, -locale-XY (each lang in other package) and often we
have additional rules to split packages.
> * debian has a _package_ called debootstrap. nobody in their right
> mind actually bootstraps openembedded to build openembedded inside
> openembedded in order to _get_ that far! (ok, nobody's ever tried it:
> all the distros so far created are, duh, embedded ones)
Nearly 4 years ago we reached moment when we had self hosted
distributions. OK, maybe none of our users would use it but it was
possible and should still be.
1. http://marcin.juszkiewicz.com.pl/2006/12/18/self-hosted-angstrom-
build/
> * (someone please correct me if i'm wrong on this) - debian does NOT
> have an automated "compile-from-scratch" system to create bootstrap
> images (but it does have buildd). openembedded does.
Ask Konstantinos how we bootstrapped armhf architecture. But yes, I
think it was manual process at start to have something working.
> * debian does NOT have an automated "cross-compile-from-scratch"
> system to create bootstrap images. openembedded does.
As long you have packages you can use debootstrap to create packages.
> the latter item _is_ a serious problem, for multi-arch and weird
> hardware capability combinations not yet imagined. as mentioned
> previously the sheer number of combinations means that whatever you
> specify as the defaults (-no-neon, -thumb, -no-thumb, -mmx, -no-mmx)
> it's GOING to be wrong.
And thats oe of things which will be discussed at sprint - how to handle
subarch repositories.
> so, basically, openembedded "has it all" - the only bit that's
> missing is a capability to understand debian/control and
> debian/rules files
Feel free to write that "missing bit". I think that once you will get
something working there will be few OE developers who will be interested
and can help.
> * openembedded's dependency system is a bit... quirky. as it reads
> all the .bb files under the sun (i'm looking at 8,000 right now) it's
7232 at the moment in development branch ;D
> a leeetle time-consuming to do even the most basic extensions to
> parsing, so they have avoided allowing in regex matching on the
> dependencies like the plague.
Bitbake is developed by 2-4 people only and mostly in their free time.
Something has to suffer. Parsing time is not so bad - initial parsing
took 3 minutes on my desktop, next reparse took only 2 (as it did not
had to check for scm versions cause this is cached).
> however, i believe that something like
> the debian dependency format - package-dev >= N.N.N - should not be
> spectacularly onerous.
Patches which would add that would be accepted for sure.
> right now the way to specify that a build is to be dependent on one
> particular version of a library is to "blacklist" all the other
> versions of the same library! that's... truly dreadful :)
PREFERRED_VERSION_libpng = "1.2.44" is whitelisting not blacklisting.
Selecting versions is job of distributions (there are few which use OE
to build packages) and it is done by selecting versions not blacklisting
every version (which would be hard to do as new version can appear at
any time).
> * openembedded's build system doesn't have anything like the sandbox
> build (using debootsrap / chroot) that debian typically uses. ok,
> correction: nobody _wants_ to use it when doing cross-compiles that
> can take potentially days as it is. however, this is merely
> "procedure" - it isn't something that's hard-coded into bitbake or
> openembedded "thou shalt not do chroots for builds".
We have packaged staging for few years now. It allows to do builds from
scratch in very short time as all dependencies are populated from
previously built packages - just like pbuilder does. It works, it's
stable and it is default for quite long time.
http://marcin.juszkiewicz.com.pl/2008/07/01/packaged-staging-and-what-
it-gives/
> rough principle:
>
> * once bitbake has been modified to understand versioned
> dependencies, a build would, instead of reading .bitbake files,
> check,
> update/download an apt-cache repository and INTERNALLY create bitbake
> recipe objects based not on .bb files but on the debian package
> information.
> and also to run
> intermediate automated tasks such as "sanity checks" to be done on
> the configure, compile and install stages.
For non familiar with OE: there are set of checks done after do_configure
task (like "configure looked into system headers" error) and several ones
are run before packages are created. In Debian world some of those tests
are part of Lintian.
> advantages:
>
> * openembedded is full of tricks such as replacement libtools and
> replacement pkgconfig scripts which understand cross-compiling:
libtool and pkgconfig itself know what cross compilation is. OE uses
upstream versions. Some changes were made by our people and sent
upstream.
> the
> replacement pkgconfig script can even create simultaneous equivalents
> of dpkg-cross {libraryname}.pc files as well as a {libraryname}.pc
> file which will end up in the corresponding "-dev" package
The only mangling which is done is removal of build directories from .pc
files.
> * whilst it's not a panacea, many of the equivalent openembedded
> packages to debian packages are pretty much up-to-date and have had
> the cross-compilation problems *solved* already. including tracking
> down very specific combinations of options that will only work on a
> particular CPU, for that specific package.
And at same time lots of packages are missing. You will not get KDE or
OpenOffice.org for example.
> disadvantages:
>
> * bitbake is _quirky_ :) recipes are a hybrid mix of shell script,
> python, makefile-like variable declaration rules, and can inherit
> "classes" and declare "tasks" which end up being inserted into the
> chain and executed on the package!
debian/rules is quirky. it is makefile which can call shell/python/perl
scripts, can include cdbs/debhelper/other things which adds tasks etc,
etc, etc...
> ok - enough! thoughts anyone?
Like Neil wrote - Debian is Debian, OpenEmbedded is OpenEmbedded. Please
do not use OE to build Debian - unless your day has 48h and you do not
need sleep.
Reply to: