Re: possible MBF: automatically detecting unused build dependencies
Hi,
Quoting Holger Levsen (2014-07-28 13:47:13)
> > You offered to run it on jenkins.d.n and its specs are surely enough to
> > regularly crunch through a couple of hundred (or more) source packages. But
> > after having consulted Helmut Grohne on how jenkins works I'm not so sure
> > anymore whether it's the right place to implement something like this. If
> > one chooses one job per source package, then one ends up with hundreds if
> > not thousands of jobs and that doesnt seem to work well?
>
> well, 1-3 hundred more jobs would be feasable, I think, so how many source
> packages do you need to rebuild exactly?
From the bootstrap perspective it needs to be at minimum 488 right now but
could be more in the future. See http://bootstrap.debian.net/history.html for a
development of the problem size over time. At maximum it needs to be 730 source
packages. That there is not one single number is because dependency
disjunctions and Provides relationships allow different solutions to the
problem.
On the other hand you can see that even the minimum number is more than the 1-3
hundred more jobs you deem to be feasible.
> > If one choose one job for the whole procedure then the problem is that it
> > must be possible for the job to be running uninterrupted for weeks. Even
> > with one job per source package, one job might be running for several days.
> > For example each compilation of src:llvm-toolchain needs 3.5 h here and
> > since 14 compilations have to be done, that would amount to more than two
> > days.
>
> 2 days is still ok'ish I say but why do you need to rebuild it 14 times????
There are two passes for each source package. The first pass is a normal build
with fatrace in the back which tracks which files are used during the build.
That information is then used to determine which build dependencies contain
files that are never used.
The second pass then goes over these results and replaces each one of these
build dependencies of which no files were used with an empty equivs package
without dependencies. This pass makes sure that also the dependencies of the
binary package are not used.
Fourteen rebuilds is the most extreme case which I used because I thought it
was useful to know the maximum amount of needed resources. On average, a source
package needs 0.5 rebuilds because for most source packages no droppable build
dependencies were found in the first pass.
> > Or do you see a way to make this kind of QA measure work with jenkins
> > nevertheless?
>
> I think I first need to understand how many jobs exactly and (if they are
> indeed way too many) whether they could be regrouped somehow. And why 14
> rebuilds...
The number of jobs depends on what you think the best way would be to tackle
this problem with jenkins if you still find it feasible.
cheers, josch
Reply to: