On 18/09/14 18:00, Jakub Wilk wrote: > Hi Stuart! Hi guys, (CCing Niels T.) Since I'm the original author of this tag, I took some time to fix this bug. > > * Stuart Prescott <stuart@debian.org>, 2014-09-18, 23:35: > >The package-contains-timestamped-gzip tag complains about gzipped files > >that are in the upstream tarball. While it is true that these files were > >compressed and contain a timestamp, it is not true that this timestamp > >will be different each time the package is built, > [...] > >It would be best if lintian didn't complain about compressed files that > >are also present in the upstream package. > > I think the following heuristics, which doesn't require access to the source > package, should work well: > > If the gzip timestamp is older than the timestamp from the changelog > trailer, then the file wasn't generated at build time, and > package-contains-timestamped-gzip shouldn't be emitted. This is what I did precisely. It required some changes to handling of dates in Lintian (UTC stuff and second precision). I attach 2 patches that implement this and another one that updates tests. As far as I can tell the UTC/mtime handling should not break anything: 'time' was never exported or used anyway so a different format for it should not make a difference. Switching to UTC *could* break something, but I've run the testsuite and it looks fine. Cheers, Tomasz
From 4c50ac057ef9d783a5df195acad5ed2604d71691 Mon Sep 17 00:00:00 2001
From: Tomasz Buchert <tomasz@debian.org>
Date: Thu, 14 May 2015 11:32:18 +0200
Subject: [PATCH 1/3] Use UTC time & add 'timestamp' method to Path
---
collection/unpacked | 4 ++--
lib/Lintian/Path.pm | 17 ++++++++++++++++-
2 files changed, 18 insertions(+), 3 deletions(-)
diff --git a/collection/unpacked b/collection/unpacked
index fdb9736..31e992f 100755
--- a/collection/unpacked
+++ b/collection/unpacked
@@ -174,7 +174,7 @@ sub extract_and_index_deb {
push(
@jobs,
{
- '_pipeline' => [['tar', '-tvf', '-'], '|', @sort_gzip, '&'],
+ '_pipeline' => [['tar', '--utc', '--full-time', '-tvf', '-'], '|', @sort_gzip, '&'],
'fail' => 'error',
'pipe_in' => FileHandle->new,
'out' => "$dir/index.gz",
@@ -185,7 +185,7 @@ sub extract_and_index_deb {
@jobs,
{
'_pipeline' =>
- [['tar', '--numeric-owner', '-tvf', '-'], '|',@sort_gzip, '&'],
+ [['tar', '--utc', '--full-time', '--numeric-owner', '-tvf', '-'], '|',@sort_gzip, '&'],
'fail' => 'error',
'pipe_in' => FileHandle->new,
'out' => "$dir/index-owner-id.gz",
diff --git a/lib/Lintian/Path.pm b/lib/Lintian/Path.pm
index 48de275..4c7c25e 100644
--- a/lib/Lintian/Path.pm
+++ b/lib/Lintian/Path.pm
@@ -33,6 +33,7 @@ use overload (
'fallback' => 0,
);
+use Date::Parse qw(str2time);
use Carp qw(croak confess);
use Scalar::Util qw(weaken);
@@ -195,7 +196,7 @@ NB: Returns the empty string for the "root" dir.
Lintian::Path->mk_ro_accessors(
qw(name owner group link type uid gid
- size date operm parent_dir dirname basename
+ size date time operm parent_dir dirname basename
));
=item children
@@ -212,6 +213,20 @@ sub children {
return @{$self->{'_sorted_children'} };
}
+=item timestamp
+
+Returns a Unix timestamp for the given path. This is a number of
+seconds since the start of Unix epoch in UTC.
+
+=cut
+
+sub timestamp {
+ my ($self) = @_;
+ my $date = $self->{'date'};
+ my $time = $self->{'time'};
+ return str2time("$date $time", "GMT");
+}
+
=item child(BASENAME)
Returns the child named BASENAME if it is a child of this directory.
--
2.1.4
From 52ce68d43fa238a5d113156391a43f3d0cc88ef4 Mon Sep 17 00:00:00 2001
From: Tomasz Buchert <tomasz@debian.org>
Date: Thu, 14 May 2015 11:32:58 +0200
Subject: [PATCH 2/3] Don't report timestamped-gzip if the file is pregenerated
(#762105)
This is a heuristic proposed in https://bugs.debian.org/762105.
If the gzipped file has modification time *after* changelog,
then very likely it is unreproducible. On the other hand,
if it is *before*, then it is probably pregenerated in upstrem
sources.
---
checks/files.pm | 18 +++++++++++++++++-
1 file changed, 17 insertions(+), 1 deletion(-)
diff --git a/checks/files.pm b/checks/files.pm
index b08000f..e8831fc 100644
--- a/checks/files.pm
+++ b/checks/files.pm
@@ -277,6 +277,18 @@ sub run {
my $isma_same = $info->field('multi-arch', '') eq 'same';
my $ppkg = quotemeta($pkg);
+ # get the last changelog timestamp
+ # if for some weird reasons the timestamp does
+ # not exist, it will remain 0
+ my $changes = $info->changelog;
+ my $changelog_timestamp = 0;
+ if (defined $changes) {
+ my ($entry) = $changes->data;
+ if ($entry && $entry->Timestamp) {
+ $changelog_timestamp = $entry->Timestamp;
+ }
+ }
+
# find out which files are scripts
my %script = map {$_ => 1} (sort keys %{$info->scripts});
@@ -1433,7 +1445,11 @@ sub run {
if ($isma_same && $file !~ m/\Q$arch\E/o) {
tag 'gzip-file-is-not-multi-arch-same-safe', $file;
} else {
- tag 'package-contains-timestamped-gzip', $file;
+ # see https://bugs.debian.org/762105
+ my $diff = $file->timestamp - $changelog_timestamp;
+ if ($diff >= 0) {
+ tag 'package-contains-timestamped-gzip', $file;
+ }
}
}
}
--
2.1.4
From 38c3407d01dc7d33dd4fc0ed900c977d0e50fb59 Mon Sep 17 00:00:00 2001 From: Tomasz Buchert <tomasz@debian.org> Date: Thu, 14 May 2015 11:57:44 +0200 Subject: [PATCH 3/3] Update files-gzip tests --- t/tests/files-gzip/debian/debian/rules | 7 +++++++ t/tests/files-gzip/tags | 2 +- t/tests/files-gzip/upstream/good.gz | Bin 23 -> 0 bytes t/tests/files-gzip/upstream/timestamped-now.gz | Bin 0 -> 23 bytes t/tests/files-gzip/upstream/timestamped-past.gz | Bin 0 -> 23 bytes 5 files changed, 8 insertions(+), 1 deletion(-) delete mode 100644 t/tests/files-gzip/upstream/good.gz create mode 100644 t/tests/files-gzip/upstream/timestamped-now.gz create mode 100644 t/tests/files-gzip/upstream/timestamped-past.gz diff --git a/t/tests/files-gzip/debian/debian/rules b/t/tests/files-gzip/debian/debian/rules index efe8cf2..1d8af89 100755 --- a/t/tests/files-gzip/debian/debian/rules +++ b/t/tests/files-gzip/debian/debian/rules @@ -3,6 +3,13 @@ pkg=files-gzip %: dh $@ +override_dh_install: + dh_install + # we set the mtime to the past + touch -m -d "2003-06-21 12:12:12 UTC" debian/$(pkg)/etc/timestamped-past.gz + # we set the mtime to right now + touch -m debian/$(pkg)/etc/timestamped-now.gz + override_dh_installdocs: dh_installdocs echo "Hello world :)" > debian/$(pkg)/usr/share/doc/$(pkg)/changelog.gz diff --git a/t/tests/files-gzip/tags b/t/tests/files-gzip/tags index e003044..74c8c3e 100644 --- a/t/tests/files-gzip/tags +++ b/t/tests/files-gzip/tags @@ -1,3 +1,3 @@ -I: files-gzip: package-contains-timestamped-gzip etc/good.gz +I: files-gzip: package-contains-timestamped-gzip etc/timestamped-now.gz W: files-gzip: gz-file-not-gzip etc/bad.gz W: files-gzip: gz-file-not-gzip usr/share/doc/files-gzip/changelog.gz diff --git a/t/tests/files-gzip/upstream/good.gz b/t/tests/files-gzip/upstream/good.gz deleted file mode 100644 index 0f545350800b7dbeb9006e8faf9a56a53a9bebd1..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 23 ecmb2|=3uy)xW$u+`Sgh=Obk_3`EQvS7#IL!F9*8- diff --git a/t/tests/files-gzip/upstream/timestamped-now.gz b/t/tests/files-gzip/upstream/timestamped-now.gz new file mode 100644 index 0000000000000000000000000000000000000000..0f545350800b7dbeb9006e8faf9a56a53a9bebd1 GIT binary patch literal 23 ecmb2|=3uy)xW$u+`Sgh=Obk_3`EQvS7#IL!F9*8- literal 0 HcmV?d00001 diff --git a/t/tests/files-gzip/upstream/timestamped-past.gz b/t/tests/files-gzip/upstream/timestamped-past.gz new file mode 100644 index 0000000000000000000000000000000000000000..0f545350800b7dbeb9006e8faf9a56a53a9bebd1 GIT binary patch literal 23 ecmb2|=3uy)xW$u+`Sgh=Obk_3`EQvS7#IL!F9*8- literal 0 HcmV?d00001 -- 2.1.4
Attachment:
signature.asc
Description: Digital signature