[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Extending the build profile namespace



Hi,

Quoting Helmut Grohne (2016-04-25 20:06:07)
> >  2. Require the implementation of a new bootstrap planner which would be
> >     able to mine the profile-specific build-dependency information to
> >     construct a bootstrap plan.
> 
> There is another reason to require a bootstrap planner: prevail sanity.
> The days of manually ordering architecture bootstrap have long passed.
> Everyone uses at least partially automatic ordering in one way or
> another.
> 
> For instance, rebootstrap orders builds and decides which packages to
> build by running dose-builddebcheck. That tooling allows selecting 30%
> and ordering 70% of the packages (increasing as more maintainers apply
> patches) even though it doesn't do any profile selection yet. The major
> problem with adding profile selection currently is the lack of metadata
> and well-defined semantics.
> 
> So in a way, I agree: It requires a bootstrap planner. Johannes Schauer
> spent a gsoc and much more to lay the groundwork for it. Though it can
> only start working once we clean the mess created by inconsistent stage
> profiles with no meaning.
> 
> The hard part is not the writing of the planner. The hard part is
> getting sufficiently correct and complete metadata for it to operate.

I agree with Helmut. We already have the planner: It is called botch. The
missing bit is the part which does not read droppable build dependencies from a
ASCII text file but instead from the package metadata itself. Back when I
started implementing botch during my GSoC in 2012 build profiles didn't exist
yet and now that we have them, we are still changing bits here and there. Once
Stretch is out, I plan to add build profile parsing support to botch. The
existing algorithms will be able to deal even with large amounts of profiles.
There is no reason to limit oneself to just a "stage1" profile. One of the main
reasons I wrote botch in the first place was to automate the process of
choosing the correct profile to build source packages with and the algorithms
to do that in a computationally efficient manner exist. See my master thesis
[1], these publications [2] [3] and some of the talks I gave about the topic
[4] [5] for more details.

[1] http://mister-muffin.de/bootstrap/thesis.pdf
[2] http://mister-muffin.de/bootstrap/cbse2013.pdf
[3] http://mister-muffin.de/bootstrap/informatiktage2013.pdf#page=96
[4] https://fosdem.org/2013/schedule/event/debian_bootstrap/
[5] http://penta.debconf.org/dc13_schedule/events/984.en.html

The main remaining problem, as Helmut already pointed out is that currently,
the metadata botch is working on is missing or wrong [6]. Until that is fixed,
there is little a bootstrap planner can do. But continuous integration efforts
like rebootstrap will help to find these problems and get them fixed.

[6] https://wiki.debian.org/Sprints/2014/BootstrapSprint/Results

Thanks!

cheers, josch

Attachment: signature.asc
Description: signature


Reply to: