Re: possible MBF: automatically detecting unused build dependencies
Hi,
Quoting Steve Langasek (2014-07-09 00:32:18)
> But it absolutely does have this effect if we add bootstrap-specific metadata
> unnecessarily, because:
>
> - it introduces a syntax incompatibility with older versions of the package
> tools
we are currently trying to get a minimal change to dpkg, apt and python-apt
into wheezy backports to solve this problem:
http://lists.debian.org/20140706211617.14505.38487@hoothoot
> - it makes the packaging more complex to understand
While this is true in one sense, one could also argue that annotating build
dependencies with what they are used for (with <!profile.nodoc> or
<!profile.nocheck>) does also add some clarity. Though I guess you were mainly
talking about complexity in debian/rules. Sure, more conditionals add
complexity.
> The latter point is as likely to cause problems for the bootstrappers
> themselves as it is for the maintainers, since the more maintainers who have
> to get this metadata right and keep it up to date, the more it's going to
> bitrot.
If you are worried about bitrot, then we have to throw more continuous testing
at it. With botch we can create a build order and then verify it once enough
source packages have build profile information attached. Should Debian not have
sufficient resources for that I am willing to do those rebuilds on my own
hardware.
The bitrot will happen even if bootstrap data is added out of necessity. Due to
changing dependencies, some stage1 information might not be needed anymore at
some point because it becomes better to break cycles at another point of the
graph. So continuous testing is needed in any case.
> I'm happy that tools like botch exist and think botch has been a useful
> accelerator for bootstrappability of Debian. However, my own goal is to see
> that future port bootstraps can be done using the standard buildd
> infrastructure (for each of Debian and Ubuntu), which doesn't currently make
> use of such techniques - rather, they work by building everything which is
> buildable. If you plan to wire up botch to wanna-build for archive-friendly
> bootstrappability, that would be great to see!
I would need to know far more about wanna-build until I can do that. I'm also
not too sure whether wanna-build is the right machinery to do bootstraps from
scratch? Maybe it is in the sense that botch could provide the info of which
source package to best build with reduced build dependencies once nothing is
buildable anymore.
But it would probably be prudent to show that such an automated bootstrap is
possible without wanna-build in the first place. And we are still quite far
away from being able to do automated bootstraps, sadly :(
> But until that's happened, I stand by my claim that stage1 data not needed
> for breaking build-dep loops is counterproductive for bootstrapping ports.
I agree with you that it is unwise to add such extra information to more
packages than needed (currently about 60 source packages) before there is
enough infrastructure to run and test everything.
But again, except for self-cycles there are no hard requirements on a source
package needing stage1 annotations. Botch can show which source packages, if
modified accordingly could solve the problem with a (close to) minimal number
of source packages changed. But if one source package cannot be changed because
of technical reasons or because the maintainer refuses to implement these
changes, then there are ways to work around that by modifying other source
packages. But I guess that's what you meant by "need".
I guess either once jessie is released or once the wheezy backport happens,
build profiles will slowly be introduced with the most obvious packages first.
Then will come the other, harder packages until we can for example do a native
amd64 bootstrap, starting from a minimal build system. Once we are that far
(and that will probably take another release at least) we can talk again about
adding "bootstrap-specific metadata unnecessarily". :)
So let me retract my claim from my earlier email and instead say that I agree
that adding <!profile.stage1> annotation should be done very selectively and
only if necessary.
Nevertheless the generated information should be stored somewhere so that a
bootstrapper can use it as a source of information which build dependencies can
be dropped without much effort from a source package if so necessary.
cheers, josch
Reply to: