[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: possible MBF: automatically detecting unused build dependencies

Hi josch,

thank you very much for this effort, just two remarks:

1. +1 to what hmh said

2. You have missed at least one virtual package: libdb-dev is used to
depend on latest libdbX.Y-dev, so if you can remove that before MBF...


On Mon, Jul 7, 2014, at 13:51, Johannes Schauer wrote:
> Hi,
> we would like to propose an MBF with severity wishlist to drop unused
> build
> dependencies in a number of source packages indicated by the attached two
> text
> files and the dd-list output.
> TLDR; We searched a selection of source packages relevant to
> bootstrapping for
> build dependencies which are never used during the package build. Fewer
> build
> dependencies help making bootstrapping easier so we want to submit bugs
> for the
> results we generated. The results were generated automatically by a
> script
> using fatrace(1) and fake packages. It follows the MBF template email and
> a
> detailed explanation of our procedures. Attached is the list of the
> unused
> build dependencies we found and the output of dd-list. We would like you
> to
> review the list, drop unused build dependencies or report errors.
> MBF template email:
> --%<---------------------------------------------------------------------------
> Subject: Please consider removing the build dependencies on $foo, $bar
> and $baz
> Severity: wishlist
> Usertag: unusedbd
> User: bootstrap@lists.debian.org
> Dear Maintainer,
> the build dependencies $foo, $bar and $baz of this source package do not
> seem
> to be needed. Neither are any of their files accessed during the build
> nor are
> their dependencies on other binary packages required. Please consider
> dropping
> those build dependencies to make bootstrapping Debian easier.
> You can find more detail about the procedures that were used to find this
> problem in the MBF announcement on debian-devel: $email
> -->%---------------------------------------------------------------------------
> Long version:
> Our "bootstrappable Debian" GSoC student Peter Pentchev found a number of
> source packages that did not make use of some of their build dependencies
> [1].
> We thought we could automate this process so we set up a simple sbuild
> based
> package builder. The machinery works in two passes. In the first pass,
> all
> source packages are built normally but with fatrace(1) active. From the
> fatrace
> output we determine all build dependencies of which no files were used
> during
> the build.  In the second pass we go through all source packages for
> which
> candidate droppable build dependencies where found with fatrace and test
> whether those build dependencies are really droppable by replacing them
> with an
> empty equivs package without dependencies one by one.
> The first pass makes sure that the files of a specific build dependency
> are not
> needed. This is to make sure not to produce false positives of build
> dependencies which, if not installed on the build system, do not result
> in a
> build failure but instead just in a reduction of functionality of the
> output
> binary. The second pass makes sure that also the dependencies of a
> specific
> build dependency are not needed. This makes sure to not produce false
> positives
> of meta or virtual packages.
> We executed the build on the amd64 snapshot 20140601T000000Z of Debian
> Sid. A
> selection of 549 source was made from the source packages that are
> involved in
> the central strongly connected component of the bootstrapping dependency
> graph.
> We needed to patch fatrace to be able to handle files larger than 2 GB
> (bug
> #722901).
> We ran the whole process with `sbuild --arch-all` and then again with
> `sbuild
> --no-arch-all`. This way we can find build dependencies which can be
> dropped
> from full package builds as well as those which can be moved to
> Build-Depends-Indep. We did not use the results to also check for build
> dependencies that can be removed from Build-Depends-Indep.
> The source code to run the whole machinery can be found at [2].
> Executing everything took two weeks and over 2000 package builds. We
> found 277
> build dependencies could be dropped from the 549 tested source packages,
> affecting 158 source packages.
> We checked the impact on the bootstrap dependency graph. If all the found
> build
> dependencies were really droppable, then the minimum (strong) dependency
> graph
> would shrink from containing 483 source packages to 447 source packages. 
> More
> importantly, the solution to the feedback arc set problem for making it
> acyclic
> would shrink from 74 to 56 source packages. Thus, the generated results
> are
> very relevant for making solving the bootstrap problem easier.
> You can find the results per source package in the attachment together
> with the
> dd-list output. The file drop-from-bd.txt lists the build dependencies
> that can
> be dropped from Build-Depends while move-to-bdi lists the build
> dependencies
> that can be moved from Build-Depends to Build-Depends-Indep.
> Can you spot obvious mistakes in the results or in the procedure used to
> generate them?
> cheers, josch
> [1] #749616 #749972 #751702 #751897 #752938
> [2] https://github.com/josch/findunusedbd
> Email had 3 attachments:
> + dd-list.txt
>   22k (text/plain)
> + drop-from-bd.txt
>   22k (text/plain)
> + move-to-bdi.txt
>   1k (text/plain)

Ondřej Surý <ondrej@sury.org>
Knot DNS (https://www.knot-dns.cz/) – a high-performance DNS server

Reply to: