[Prototype Patch] Always include debian packaging in index
Hi
Jakub have asked to get debian packaging included in the index (for
source packages). Currently this only happens for native packages.
I have attached a prototype patch that I intend to use a basis. The
patch will make index include the .debian.tar.$ext in the list of
tarballs and forge index data for 1.0 packages based on the lsdiff output.
There are at least two major corner cases it does not handle, which
should be fixed before applying it to the master branch.
* 3.0 (quilt) where upstream has a debian dir in the orig tarball
* 1.0 diff.gz removes a file from the upstream part.
Alternatively, we can create a separate index for debfiles that contains
this information. In that case, we will have to split the "upstream
part" and the "debian part" of the index for native packages.
On a related note, how do we want to handle index info for files that
are removed during unpacking? In terms of checking for "non-free"
content we would want the file to "appear", but a lot of our checks
assumes a "1:1" mapping between the index and the contents of the
unpacked/ dir.
~Niels
>From 4c85d32d41c74dda825509ea69056ef0f5e0179c Mon Sep 17 00:00:00 2001
From: Niels Thykier <niels@thykier.net>
Date: Wed, 26 Oct 2011 15:54:15 +0200
Subject: [PATCH] Always include debian packaging in index
With this patch, the debian packaging for non-native packages is
included in the index. For 3.0 (quilt) source files, it means
added the tarball to the list of tarballs to be scanned. For 1.0
sources, this means hand-crafting it based on the lsdiff output.
There are some corner cases this patch does not handle:
* debian dir present in the orig tarball
* files removed with the diff.gz
In both cases those should probably be removed; else the index will
point to non-existing files.
Signed-off-by: Niels Thykier <niels@thykier.net>
---
collection/index | 57 ++++++++++++++++++++++++++++++++++++++++++++++++-----
1 files changed, 51 insertions(+), 6 deletions(-)
diff --git a/collection/index b/collection/index
index 38192be..7df98bd 100755
--- a/collection/index
+++ b/collection/index
@@ -57,6 +57,7 @@ sub gather_tarballs {
my @tarballs;
my $base;
my $baserev;
+ my $diffgz;
fail "Cannot resolve \"dsc\" link for $pkg or it does not point to a file.\n" unless $file and -e $file;
(undef, $dir, undef) = File::Spec->splitpath($file);
$data = get_dsc_info($file) or fail "Could not parse dsc file for $pkg.\n";
@@ -75,21 +76,26 @@ sub gather_tarballs {
next if ($t[2] =~ m,/,);
# Look for $pkg_$version.orig(-$comp)?.tar.$ext (non-native)
# or $pkg_$version.tar.$ext (native)
- # - This deliberately does not look for the debian packaging
- # even when this would be a tarball.
- if ($t[2] =~ /^(?:\Q$base\E\.orig(?:-(.*))?|\Q$baserev\E)\.tar\.(?:gz|bz2|lzma|xz)$/) {
+ # or $pkg_$verson.debian.tar.$ext (debian packaging)
+ if ($t[2] =~ m/^(?:\Q$base\E\.orig(?:-(.*))?|\Q$baserev\E(\.debian)?)\.tar\.(?:gz|bz2|lzma|xz)$/) {
+ if ($2) {
+ push @tarballs, [$t[2], 'debian'];
+ next;
+ }
push @tarballs, [$t[2], $1//''];
+ } elsif ($t[2] eq "$baserev.diff.gz") {
+ $diffgz = $t[2];
}
}
fail('could not find the source tarball') unless @tarballs;
- return @tarballs;
+ return (\@tarballs, $diffgz);
}
# Creates an index for the source package
sub index_src {
- my @tarballs = gather_tarballs();
+ my ($tarballs, $diffgz) = gather_tarballs();
my @result;
- foreach my $tardata (@tarballs) {
+ foreach my $tardata (@$tarballs) {
my ($tarball, $compname) = @$tardata;
my @index;
# Collect a list of the files in the source package. tar currently doesn't
@@ -183,6 +189,45 @@ sub index_src {
}
push @result, @index;
}
+ if ($diffgz) {
+ require POSIX;
+ import POSIX qw/strftime/;
+ # We have a diff.gz as well, time to use lsdiff
+ # - we have to check if it adds any new dirs (usually only debian/ and debian/source/)
+ my %dirs = ();
+ my %files = ();
+ my $date = strftime ('%Y-%m-%d %H:%M', localtime);
+ for my $i (@result) {
+ my (undef, undef, undef, undef, undef, $dir) = split m/\s++/o, $i, 6;
+ $dirs{$i} = 1 if $i =~ m/^d/;
+ $files{$i} = 1 if $i !~ m/^d/;
+ }
+ open my $fd, '-|', "lsdiff -z --strip=1 \Q$diffgz\E"
+ or fail "cannot run lsdiff on $diffgz";
+ while (my $f = <$fd>) {
+ # We have already seen that file (i.e. it is a modification)
+ # - it may be a removal, but we ignore that :)
+ next if $files{$f};
+ my $mode = '-rw-r--r--';
+ # dpkg-source adds the exec-flag on unpack, so pretend it was there all
+ # the time.
+ $mode = '-rwxr-xr-x' if $f eq 'debian/rules';
+ # Create an index entry for it (we lie about size and date).
+ # Technically we could find the size by reading the actual patch, but we
+ # rarely check it beyond whether it is 0 or not. The date is usually not
+ # available in the dpkg-source generated patch, so we just use "now".
+ # - we could extract the date from the changelog as a better guess, but
+ # it is probably not worth it.
+ # User should be 0/0 (because the package is build as (fake)root).
+ push @result, "$mode 0/0 1 $date $f";
+ # strip the file name (leave trailing /) and check if the dir exists
+ $f =~ s,[^/]+$,,o;
+ next if $dirs{$f};
+ # Also create the dir entry - use size as size, that is what tar does.
+ push @result, "drwxr-xr-x 0/0 0 $date $f";
+ }
+ close $fd;
+ }
# Now that we have the file names we want, write them out sorted to the index
# file.
spawn({ fail => 'error', out_append => 'index' },
--
1.7.7
Reply to: