RFC: Generate index of source packages from unpacked
Hi,
About a year ago[1], I suggested a prototype patch for including the
debian packaging in the index of source packages. Jakub will suggested
that it would be better to have two indices, one for the unpacked source
package and one for the orig tarballs[2].
I recently tried the "simple" approach of having index depend on
unpacked and running find (with a rather long format arg) to generate
the /source/ index. The binary index generation has not been altered.
This works rather well, though it has some caveats. Most of these
appear to be non issues for /source/ packages[3].
* hardlink information is lost (probably not an issue)
* owner/group info is lost/faked as root/root (prob. not an issue)
* permissions become subject to umask + set{u,g}id bits of unpacking.
- If the user part of the umask is restricted (umask & 0700 != 0),
then Lintian (most likely) breaks in "fun" ways anyway. So I guess
this is not an issue as well.
- The Lab on lintian.d.o is setgid and dirs unpacked inside it will
inherit this bit. Again, probably not an issue if properly
documented.
* Extending "collection" time as index + unpacked cannot be run in
parallel anymore.
If you spot other caveats do let me know. If nothing else, they deserve
to be documented.
In terms of unpacked time, I have observed that unpacking time of source
packages do not appear to be severely affected.
My example package eclipse 3.7.2-1 used to spend ~6 seconds in
coll/unpacked and ~5.4 seconds in coll/index (both run in parallel, so
"max" time is 6 seconds).
Using find to generate the index, coll/index drops to about half a
second, so the total run time is increased from ~6 to ~6.5 seconds.
NB: It was unpacked to a tmpfs. On xfs, coll/unpacked is ~12 seconds
and the find variant of coll/index is ~0.7 seconds. Assuming this
applies to all FSes, then the speed regression appears to be insignificant.
To avoid speed regressions with binary packages, we can make the
dependency "type specific" (i.e. "applies only to source packages").
Our current code does not support it, but prototype patch is available.
The "using find to generate source index" is available on my
"unpacked-index" branch[4]. The "type-specific dependencies" is
"coll-type-spec-needs"[5].
If there are no additional caveats or objections, I intend to merge
these two branches into master in a couple of days.
~Niels
[1] https://lists.debian.org/debian-lint-maint/2011/10/msg00150.html
[2] https://lists.debian.org/debian-lint-maint/2011/10/msg00153.html
[3] Though most of them are deal-breakers for binary packages , so I
cannot recommend using this approach for binary packages.
[4]
http://anonscm.debian.org/gitweb/?p=users/nthykier/lintian.git;a=shortlog;h=refs/heads/unpacked-index
[5]
http://anonscm.debian.org/gitweb/?p=users/nthykier/lintian.git;a=shortlog;h=refs/heads/coll-type-spec-needs
Reply to: