Re: Bootstrappable Debian - proposal of needed changes
On Thu, Jan 17, 2013 at 09:34:16AM +0100, Johannes Schauer wrote:
> Hi,
>
> On Thu, Jan 17, 2013 at 08:33:57AM +0100, Wouter Verhelst wrote:
> > If wanna-build is updated to support these two fields, then I imagine
> > it can run the bootstrapping dependency algorithm. While you wouldn't
> > want to upload a package to the debian.org archive when the
> > architecture is as yet incomplete, the same isn't true for the
> > debian-ports.org archive.
> >
> > It would require some patches for wanna-build to understand that package
> > "foo" would need to be rebuilt once a particular profile is fully built
> > and available, but I don't think that's impossible.
>
> I'm not sure if wanna-build is the right tool to do this
Why not?
It already needs to do build-dependency tracking, marking packages as
"can't be built yet because build-depends aren't there yet" all the
time. That's exactly the sort of thing you need to be doing, no?
Of couse, it's entirely possible that I'm missing something obvious here
-- you're the one who spent half a year figuring out what needs to be
done, not me ;-)
> but we also did not yet implement anything which would require a
> specific tool for the build process.
>
> The output of the current version of the algorithm produces is an ASCII,
> comma separated list of source packages. All packages which can be built
> in parallel are given on the same line. Source packages that need to be
> built with reduced build dependencies are annotated as such. The
> rebuilding of profile built packages is part of this output and
> triggered after all source packages of a strongly connected component
> finished building.
I see.
> After all profile built packages successfully rebuilt
> fully, *all* source packages of the strongly connected component are
> rebuilt. This is to make sure that every source package was built once
> with all build dependencies available.
If you have enough information on what a package needs to build
properly, you wouldn't need to do this.
But I agree that's a pretty big "if".
> This list gave me so far an idea of the build order and a way to check
> whether it made sense. By coming up with a more machine readable version
> of this output, a tool can be triggered/controlled which then gives the
> actual build jobs and does things like you mentioned above.
Right.
> > However, to do that, there's one thing I'm missing in your mail: there
> > will be cases where packages, when built in a particular profile do
> > not support some functionality. That is, the package is available and
> > does most of what the full package does, but because some
> > build-dependency is missing, it doesn't support some feature or other
> > -- for example, let's say that a document translation package (which
> > we'll call "foo") which supports many input formats does not support
> > XML as an input format when built in the stage1 profile. The output of
> > its stage1 build would not change the number of binary packages built
> > with the source package, it would just build a binary package with
> > reduced functionality.
> >
> > Now it might be that a package build-depends on our package foo because
> > it needs to translate documents in that XML format into something else,
> > With your proposal, there's no way for the build-depending package to
> > declare that it needs a version of foo that was built at a richer
> > profile than stage1.
> >
> > Is this something you've considered?
>
> Glad you bring this up because we indeed forgot to mention this in our
> initial email. Yes, what you mention can indeed happen and this is also
> why extensive rebuilding is part of the build order: to make sure that
> all features are always compiled into the binaries.
>
> We have thought about this issue but did not manage to come up with a
> global solution. A solution would require a way to depend on "features"
> of a package and therefor create an overcomplicated dependency syntax
> which would not justify its purpose.
I'm not sure I agree with that.
Your build profiles are essentially creating additional "variants" of a
particular binary package. I imagine we should be able to come up with a
syntax for the build-depends header which would allow one to declare you
depend on a particular variant of a binary package.
Let's say something like the following (extending the syntax as
suggested by Ian Jackson earlier):
Build-Depends: foo (profile:stage1 >= 1.3) [profile:stage1], foo (>= 1.3) [!profile:stage1]
Essentially, with this syntax I'm saying that to build a the package
which contains this build-dependency header, under normal circumstances
you'd need the package "foo" at version 1.3 or above, as built with no
profiles enabled; but if you enable the "stage1" profile for this
package, you can use the "stage1" version of the "foo" package instead.
I don't think this is too confusing or complicated, and I think it makes
sense to add the profile to the version information in this manner; a ()
qualifier for a package could then be interpreted as a statement of
*what* a package needs to comply with (version or profile of the
build-dependency), whereas a [] qualifier would state *when* we need a
particular build-dependency (architecture or profile of our *current*
build).
> There are two things that a developer could do to avoid such FTBFS
> errors during the bootstrap process when he modifies a source package
> with reduced "stage1" build dependencies:
>
> - only remove build dependencies which do not modify the binary
> packages which are produced (instead, let some binary packages not be
> produced, like foo-doc, foo-java or foo-gtk)
>
> - look at the reverse dependencies in the generated dependency graph to
> make sure that features which are changed in built binary packages
> are not depended upon (naturally, those reverse dependencies will
> change over time...)
>
> Both solutions are very fragile. This is also why the developed
> algorithms try to build as few source packages as possible with reduced
> build dependencies.
That makes sense, I suppose.
> Do we have some experience of how common such errors are during a manual
> bootstrap?
Not me, I'm afraid.
--
Copyshops should do vouchers. So that next time some bureaucracy requires you
to mail a form in triplicate, you can mail it just once, add a voucher, and
save on postage.
Reply to: