crossbuild dependency satisfaction
Hi,
I started composing this email to Helmut only as he's interested in this work
for rebootstrap but then I thought, this could also be interesting for a wider
audience.
Firstly, http://bootstrap.debian.net is back. The immediate effect of this is,
that http://bootstrap.debian.net/importance_metric.html gets regenerated again.
During the past months it was not (because the high amount of memory it needs).
Operation of bootstrap.debian.net as a whole was restricted to only processing
amd64 (because of restricted CPU cycles) and now all arches are back again
(because now I have tons of memory and CPU once again). While this has probably
no real immediate benefit (even less so due to the freeze), once Jessie is
released, this information can be incorporated into the PTS/tracker/UDD because
then packages with build profiles can be uploaded so that the issues
highlighted by bootstrap.debian.net can actually be fixed in the archive.
Secondly, I was working on creating a dose3 wrapper which would show more than
just the first reason why a package is uninstallable/uncompilable. After
writing a prototype (which ran dose3, collected its results and then modified
the input Packages files to remove the problem and make it possible to look for
the next error) I moved the whole caboodle to an OCaml application to avoid the
time spent on parsing all Packages and Sources files over and over again. The
resulting program is still slow (because it turns out that dose3 does not like
modifying its universe in-place so it has to be regenerated ever time) but
works at a rate of about 6.9 source packages per minute. This is already two
orders of magnitude faster than the original mock-up. The program will become
public with the next botch release, which will happen after the next dose3
release (botch currently FTBFS because of dose3). The script can be made much
faster in the future after dose3 gets support for modifying an existing
universe in place.
Thirdly, http://bootstrap.debian.net/cross.html is currently only run on a
selection of around 550 source packages (the actual selection changes daily
depending on package interdependencies in sid). The page also only shows the
first encountered problem for non-satisfiability and does not yet use the
improved version as outlined above. But what I did now do was to run it with an
extended set of packages. Here is the same kind of output generated for all
source packages in Debian sid: https://mister-muffin.de/p/Lc4-.html One can see
that especially in the first table ("missing") there are many low hanging
fruits of packages that, if they were made m-a:foreign, would help a large
amount of packages.
Fourthly, I fed the selection of 550 source packages that cross.html displays
into the new dose3 wrapper. The result is available here:
https://mister-muffin.de/p/xOJO.html Most results in the "conflicts" table seem
to come from build-essential:host being drawn in by build dependencies on
gobject-introspection:host. I might replace the current cross.html page with
daily results from this program in the future. But first, dose3 needs to be
fixed.
Fifthly, I added a new history overview at
http://bootstrap.debian.net/history_cross.html which shows how many source
packages succeeded to satisfy their cross build dependencies from 2005 until
today. As more and more dependencies get fixed with the help of cross.html and
rebootstrap, this graph should go up in the future. The greatest impact so far
was making debhelper M-A:foreign in the middle of 2012. I guess there will be
similar effects once other helper scripts are fixed for perl, python, java and
others as there are a large number of source packages with only very few
dependencies.
cheers, josch
Reply to: