Bug#738591: lintian: Add checker for timestamped gzip files
On 11/02/14 10:09, Jérémy Bobbio wrote:
> Tomasz Buchert:
> > > >> +Severity: normal
> > > >
> > > > It think it should be at most "wishlist", perhaps even "pedantic".
> > > >
> >
> > Let's make it "pedantic", but hopefully one day
> > it will be "normal".
>
> Could we go for “wishlist” instead?
Hi,
I reworked the patch so that it reuses the machinery
in files.pm. I also made it "wishlist" this time. I attach the patch.
Currently, it will emit "package-contains-timestamped-gzip"
on any file ending with ".gz", being a gzip file and containing
a timestamp. It means that currently the tag "gzip-file-is-not-multi-arch-same-safe"
will imply "package-contains-timestamped-gzip". Moreover, the new tag breaks multiple
tests (files-gzip, manpages-general, etc.) because they use timestamped gzips. I will
fix it, but first I'd like to know that implementation of the tag is ok.
Cheers,
Tomasz
>
> I know that switching to reproducible builds sounds like a major
> shift in Debian's current practices but we already have way more
> packages reproducible that one might expect.
>
> The following wiki page describe the last large scale experiment that
> was done: <https://wiki.debian.org/ReproducibleBuilds/Rebuild20140126>
> 67% out of the 6887 packages that were tested were reproducible. 103 of
> them failed due to one or more timestamp in gzip files.
>
> I think “wishlist” is more appropriate because we are trying to get the
> the archive reproducible and asking interested maintainers for help.
> I don't think this fall under a “particular Debian packaging style”
> as worded in the man page about `--pedantic`.
>
> In any cases, my dear Lintian maintainers, I trust you to sort things
> out appropriately. :)
>
> --
> Lunar .''`.
> lunar@debian.org : :Ⓐ : # apt-get install anarchism
> `. `'`
> `-
>From 4dcc45c75df792820c356beca0fa84b067cf0268 Mon Sep 17 00:00:00 2001
From: Tomasz Buchert <tomasz.buchert@inria.fr>
Date: Tue, 11 Feb 2014 10:11:20 +0100
Subject: [PATCH] new tag: package-contains-timestamped-gzip (+ test)
---
checks/files.desc | 8 ++++++++
checks/files.pm | 14 +++++++++-----
t/tests/reproducibility/debian/debian/control.in | 17 +++++++++++++++++
.../debian/debian/unreproducible-pkg.install | 1 +
t/tests/reproducibility/debian/file | 1 +
t/tests/reproducibility/debian/file-with-timestamp.gz | Bin 0 -> 39 bytes
.../reproducibility/debian/file-without-timestamp.gz | Bin 0 -> 34 bytes
t/tests/reproducibility/debian/prepare | 4 ++++
t/tests/reproducibility/desc | 6 ++++++
t/tests/reproducibility/tags | 1 +
10 files changed, 47 insertions(+), 5 deletions(-)
create mode 100644 t/tests/reproducibility/debian/debian/control.in
create mode 100644 t/tests/reproducibility/debian/debian/unreproducible-pkg.install
create mode 100644 t/tests/reproducibility/debian/file
create mode 100644 t/tests/reproducibility/debian/file-with-timestamp.gz
create mode 100644 t/tests/reproducibility/debian/file-without-timestamp.gz
create mode 100755 t/tests/reproducibility/debian/prepare
create mode 100644 t/tests/reproducibility/desc
create mode 100644 t/tests/reproducibility/tags
diff --git a/checks/files.desc b/checks/files.desc
index 760f86a..e8237f0 100644
--- a/checks/files.desc
+++ b/checks/files.desc
@@ -1448,3 +1448,11 @@ Info: The given file is in PATH but consists of non-ASCII characters.
.
Note that Lintian may be unable to display the filename accurately.
Unprintable characters may have been replaced.
+
+Tag: package-contains-timestamped-gzip
+Severity: wishlist
+Certainty: certain
+Info: The package contains a gzip'ed file that has timestamps.
+ Such files make the produces packages unreproducible.
+ .
+ Pass "-n" flag to gzip to avoid it.
diff --git a/checks/files.pm b/checks/files.pm
index 5c5a60d..21a0f0c 100644
--- a/checks/files.pm
+++ b/checks/files.pm
@@ -1400,23 +1400,27 @@ sub run {
my $finfo = $info->file_info($file) || '';
if ($finfo !~ m/gzip compressed/) {
tag 'gz-file-not-gzip', $file;
- } elsif ($isma_same && $file !~ m/\Q$arch\E/o) {
+ } else {
my $path = $info->unpacked($file);
my $buff;
+ my $mtime;
open(my $fd, '<', $path);
# We need to read at least 8 bytes
if (sysread($fd, $buff, 1024) >= 8) {
# Extract the flags and the mtime.
# NN NN NN NN, NN NN NN NN - bytes read
# __ __ __ __, $mtime - variables
- my (undef, $mtime) = unpack('NN', $buff);
- if ($mtime){
- tag 'gzip-file-is-not-multi-arch-same-safe',$file;
- }
+ (undef, $mtime) = unpack('NN', $buff);
} else {
fail "reading $file: $!";
}
close($fd);
+ if ($mtime != 0) {
+ if ($isma_same && $file !~ m/\Q$arch\E/o) {
+ tag 'gzip-file-is-not-multi-arch-same-safe', $file;
+ }
+ tag 'package-contains-timestamped-gzip', $file;
+ }
}
}
diff --git a/t/tests/reproducibility/debian/debian/control.in b/t/tests/reproducibility/debian/debian/control.in
new file mode 100644
index 0000000..a7e8050
--- /dev/null
+++ b/t/tests/reproducibility/debian/debian/control.in
@@ -0,0 +1,17 @@
+Source: {$source}
+Priority: extra
+Section: devel
+Maintainer: {$author}
+Standards-Version: {$standards_version}
+Build-Depends: debhelper (>= 9)
+
+Package: unreproducible-pkg
+Architecture: all
+Depends: $\{misc:Depends\}
+Description: {$description} - gzip files
+ This is a test package designed to exercise some feature or tag of
+ Lintian. It is part of the Lintian test suite and may do very odd
+ things. It should not be installed like a regular package. It may
+ be an empty package.
+ .
+ Contains a few GZIP files.
diff --git a/t/tests/reproducibility/debian/debian/unreproducible-pkg.install b/t/tests/reproducibility/debian/debian/unreproducible-pkg.install
new file mode 100644
index 0000000..15b72c1
--- /dev/null
+++ b/t/tests/reproducibility/debian/debian/unreproducible-pkg.install
@@ -0,0 +1 @@
+*.gz usr/share/pkg-with-gzips/
diff --git a/t/tests/reproducibility/debian/file b/t/tests/reproducibility/debian/file
new file mode 100644
index 0000000..96bc543
--- /dev/null
+++ b/t/tests/reproducibility/debian/file
@@ -0,0 +1 @@
+This is a text.
diff --git a/t/tests/reproducibility/debian/file-with-timestamp.gz b/t/tests/reproducibility/debian/file-with-timestamp.gz
new file mode 100644
index 0000000000000000000000000000000000000000..3d4e78818ffafb9ad11f313d0c7a7abbfde1d697
GIT binary patch
literal 39
vcmb2|=HTd#{u#u;oR*oB%D{c{giaX4BA3=lSGt63^%#WmPkj(zU|;|M2G$JF
literal 0
HcmV?d00001
diff --git a/t/tests/reproducibility/debian/file-without-timestamp.gz b/t/tests/reproducibility/debian/file-without-timestamp.gz
new file mode 100644
index 0000000000000000000000000000000000000000..0dbd7dd9a0aa228b8af804b0fc8c304ffa6fcfd1
GIT binary patch
literal 34
lcmb2|=3oE=X6}<Gbix=GxwKBY(j{c8#~_q{>Vp7K3IL1w2=@R0
literal 0
HcmV?d00001
diff --git a/t/tests/reproducibility/debian/prepare b/t/tests/reproducibility/debian/prepare
new file mode 100755
index 0000000..a0feb41
--- /dev/null
+++ b/t/tests/reproducibility/debian/prepare
@@ -0,0 +1,4 @@
+#!/bin/bash
+
+gzip file -c > file-with-timestamp.gz
+gzip file -nc > file-without-timestamp.gz
diff --git a/t/tests/reproducibility/desc b/t/tests/reproducibility/desc
new file mode 100644
index 0000000..f0e18bd
--- /dev/null
+++ b/t/tests/reproducibility/desc
@@ -0,0 +1,6 @@
+Testname: reproducibility
+Sequence: 6000
+Version: 1.0
+Description: Test if package is reproducible
+Test-For:
+ package-contains-timestamped-gzip
diff --git a/t/tests/reproducibility/tags b/t/tests/reproducibility/tags
new file mode 100644
index 0000000..8057dae
--- /dev/null
+++ b/t/tests/reproducibility/tags
@@ -0,0 +1 @@
+I: unreproducible-pkg: package-contains-timestamped-gzip usr/share/pkg-with-gzips/file-with-timestamp.gz
--
1.8.5.3
Reply to: