Re: possible MBF: automatically detecting unused build dependencies
Hi josch,
thank you very much for this effort, just two remarks:
1. +1 to what hmh said
2. You have missed at least one virtual package: libdb-dev is used to
depend on latest libdbX.Y-dev, so if you can remove that before MBF...
Ondrej
On Mon, Jul 7, 2014, at 13:51, Johannes Schauer wrote:
> Hi,
>
> we would like to propose an MBF with severity wishlist to drop unused
> build
> dependencies in a number of source packages indicated by the attached two
> text
> files and the dd-list output.
>
> TLDR; We searched a selection of source packages relevant to
> bootstrapping for
> build dependencies which are never used during the package build. Fewer
> build
> dependencies help making bootstrapping easier so we want to submit bugs
> for the
> results we generated. The results were generated automatically by a
> script
> using fatrace(1) and fake packages. It follows the MBF template email and
> a
> detailed explanation of our procedures. Attached is the list of the
> unused
> build dependencies we found and the output of dd-list. We would like you
> to
> review the list, drop unused build dependencies or report errors.
>
> MBF template email:
>
> --%<---------------------------------------------------------------------------
> Subject: Please consider removing the build dependencies on $foo, $bar
> and $baz
> Severity: wishlist
> Usertag: unusedbd
> User: bootstrap@lists.debian.org
>
> Dear Maintainer,
>
> the build dependencies $foo, $bar and $baz of this source package do not
> seem
> to be needed. Neither are any of their files accessed during the build
> nor are
> their dependencies on other binary packages required. Please consider
> dropping
> those build dependencies to make bootstrapping Debian easier.
>
> You can find more detail about the procedures that were used to find this
> problem in the MBF announcement on debian-devel: $email
> -->%---------------------------------------------------------------------------
>
> Long version:
>
> Our "bootstrappable Debian" GSoC student Peter Pentchev found a number of
> source packages that did not make use of some of their build dependencies
> [1].
> We thought we could automate this process so we set up a simple sbuild
> based
> package builder. The machinery works in two passes. In the first pass,
> all
> source packages are built normally but with fatrace(1) active. From the
> fatrace
> output we determine all build dependencies of which no files were used
> during
> the build. In the second pass we go through all source packages for
> which
> candidate droppable build dependencies where found with fatrace and test
> whether those build dependencies are really droppable by replacing them
> with an
> empty equivs package without dependencies one by one.
>
> The first pass makes sure that the files of a specific build dependency
> are not
> needed. This is to make sure not to produce false positives of build
> dependencies which, if not installed on the build system, do not result
> in a
> build failure but instead just in a reduction of functionality of the
> output
> binary. The second pass makes sure that also the dependencies of a
> specific
> build dependency are not needed. This makes sure to not produce false
> positives
> of meta or virtual packages.
>
> We executed the build on the amd64 snapshot 20140601T000000Z of Debian
> Sid. A
> selection of 549 source was made from the source packages that are
> involved in
> the central strongly connected component of the bootstrapping dependency
> graph.
> We needed to patch fatrace to be able to handle files larger than 2 GB
> (bug
> #722901).
>
> We ran the whole process with `sbuild --arch-all` and then again with
> `sbuild
> --no-arch-all`. This way we can find build dependencies which can be
> dropped
> from full package builds as well as those which can be moved to
> Build-Depends-Indep. We did not use the results to also check for build
> dependencies that can be removed from Build-Depends-Indep.
>
> The source code to run the whole machinery can be found at [2].
>
> Executing everything took two weeks and over 2000 package builds. We
> found 277
> build dependencies could be dropped from the 549 tested source packages,
> affecting 158 source packages.
>
> We checked the impact on the bootstrap dependency graph. If all the found
> build
> dependencies were really droppable, then the minimum (strong) dependency
> graph
> would shrink from containing 483 source packages to 447 source packages.
> More
> importantly, the solution to the feedback arc set problem for making it
> acyclic
> would shrink from 74 to 56 source packages. Thus, the generated results
> are
> very relevant for making solving the bootstrap problem easier.
>
> You can find the results per source package in the attachment together
> with the
> dd-list output. The file drop-from-bd.txt lists the build dependencies
> that can
> be dropped from Build-Depends while move-to-bdi lists the build
> dependencies
> that can be moved from Build-Depends to Build-Depends-Indep.
>
> Can you spot obvious mistakes in the results or in the procedure used to
> generate them?
>
> cheers, josch
>
> [1] #749616 #749972 #751702 #751897 #752938
> [2] https://github.com/josch/findunusedbd
> Email had 3 attachments:
> + dd-list.txt
> 22k (text/plain)
> + drop-from-bd.txt
> 22k (text/plain)
> + move-to-bdi.txt
> 1k (text/plain)
--
Ondřej Surý <ondrej@sury.org>
Knot DNS (https://www.knot-dns.cz/) – a high-performance DNS server
Reply to: