--- Begin Message ---
Package: release.debian.org
Severity: normal
User: release.debian.org@packages.debian.org
Usertags: unblock
Hello release team!
I'm here to talk about an eventual unblock request of my package
mmdebstrap. In this context I want to discuss with you, which changes
will still be permitted at this time of the freeze. It is very unlikely
that any of these changes will break anything because mmdebstrap is
intensively tested in 91 different scenarios, exercising all reasonably
testable code-paths according to the Devel::Cover Perl module.
If these patches cannot be part of the initial Buster release, they
should definitely go into the next point release.
I would not say that any of these are of RC severity but they they are
not less than important severity either. Each of these fixes is targeted
and minimal and does not break any interfaces.
mmdebstrap has no reverse-Depends (but some reverse-Recommends).
I find especially support for old apt versions and derivative
distributions important. If you tell me to go ahead, then I would need:
unblock mmdebstrap/0.4.1-3
All the patches I list in the following are attached to this mail as
part of a debdiff where I applied all of them.
1. 0001-cleanup-auxfiles-after-running-apt-get-update-or-oth.patch
This closes #927151. The workaround is a manual rm of the auxfiles
directory. This problem only occurs when producing a Debian chroot with
an apt version before the auxfiles directory existed.
2. 0002-chdir-before-apt-get-update-to-accomodate-for-apt-1..patch
Another fix is for apt (<< 1.5) which will attempt to chdir() after an
"apt update" which might fail if that directory is not readable by the
user running apt (apt bug #860738). The workaround is to start
mmdebstrap from a world readable directory.
3. 0003-add-Dir-State-Status-to-apt-config-for-apt-1.3.patch
For apt apt (<< 1.3) it is necessary to explicitly set the
Dir::State::Status or otherwise, the dpkg status database outside the
chroot will be used. There exists no equivalent workaround.
4. 0004-chmod-0755-on-qemu-user-static-binary.patch
On Ubuntu, the binfmt mechanism seems to require that interpreters are
marked executable. This requirement does not seem to exist on Debian
which is why the executable bit was originally not set for the
qemu-user-static binary. There is no workaround.
5. 0005-chdir-to-parent-of-root-before-remove_tree-to-preven.patch
This is similar to (2.) because the function remove_tree of the perl
module File::Path also attempts to chdir to the current directory before
removing a completely different directory. This chdir will fail if the
current directory is not readable by the user running the function. The
workaround is to start mmdebstrap from a world readable directory.
6. 0006-Only-error-out-on-W-and-Err-lines-on-apt-get-update.patch
This closes #928079. Instead of using exit codes, apt communicates
failure using "W:" and "Err:" lines on standard output (apt bugs
#778357, #776152, #696335 and #745735). But sometimes, maintainer
scripts also emit non-fatal "W:" lines. The workaround is to install the
affected packages after mmdebstrap finished manually.
diff -Nru mmdebstrap-0.4.1/debian/changelog mmdebstrap-0.4.1/debian/changelog
--- mmdebstrap-0.4.1/debian/changelog 2019-03-18 14:46:01.000000000 +0100
+++ mmdebstrap-0.4.1/debian/changelog 2019-05-04 23:48:19.000000000 +0200
@@ -1,3 +1,15 @@
+mmdebstrap (0.4.1-3) unstable; urgency=medium
+
+ * Backport patches from upstream:
+ - cleanup auxfiles *after* running apt-get update (closes: #927151)
+ - chdir() before 'apt-get update' for apt (<< 1.5)
+ - chdir() before remove_tree()
+ - add Dir::State::Status to apt config for apt (<< 1.3)
+ - chmod 0755 on qemu-user-static binary for Ubuntu
+ - Only error out on W: and Err: lines (closes: #928079)
+
+ -- Johannes 'josch' Schauer <josch@debian.org> Sat, 04 May 2019 23:48:19 +0200
+
mmdebstrap (0.4.1-2) unstable; urgency=medium
* Mark autopkgtest as flaky (closes: #924854)
diff -Nru mmdebstrap-0.4.1/debian/patches/0001-cleanup-auxfiles-after-running-apt-get-update-or-oth.patch mmdebstrap-0.4.1/debian/patches/0001-cleanup-auxfiles-after-running-apt-get-update-or-oth.patch
--- mmdebstrap-0.4.1/debian/patches/0001-cleanup-auxfiles-after-running-apt-get-update-or-oth.patch 1970-01-01 01:00:00.000000000 +0100
+++ mmdebstrap-0.4.1/debian/patches/0001-cleanup-auxfiles-after-running-apt-get-update-or-oth.patch 2019-05-04 23:44:01.000000000 +0200
@@ -0,0 +1,49 @@
+From b2d5a45932a303a65baa4ec5081c5fa1f4fa80a5 Mon Sep 17 00:00:00 2001
+From: Johannes 'josch' Schauer <josch@mister-muffin.de>
+Date: Tue, 23 Apr 2019 13:28:55 +0200
+Subject: [PATCH 1/6] cleanup auxfiles *after* running apt-get update or
+ otherwise it will be re-created
+
+---
+ coverage.sh | 2 --
+ mmdebstrap | 12 ++++++------
+ 2 files changed, 6 insertions(+), 8 deletions(-)
+
+--- a/coverage.sh
++++ b/coverage.sh
+@@ -155,8 +155,6 @@ grep -v '^Priority: ' /tmp/debian-$dist-
+ diff -u status1 status2
+ rm status1 status2
+ rm /tmp/debian-$dist-debootstrap/var/lib/dpkg/status /tmp/debian-$dist-mm/var/lib/dpkg/status
+-# this file is only created by apt 1.6 or newer
+-rmdir /tmp/debian-$dist-mm/var/lib/apt/lists/auxfiles
+ # debootstrap exposes the hosts's kernel version
+ rm /tmp/debian-$dist-debootstrap/etc/apt/apt.conf.d/01autoremove-kernels \
+ /tmp/debian-$dist-mm/etc/apt/apt.conf.d/01autoremove-kernels
+--- a/mmdebstrap
++++ b/mmdebstrap
+@@ -1652,18 +1652,18 @@ sub setup {
+
+ # clean up temporary configuration file
+ unlink "$options->{root}/etc/apt/apt.conf.d/00mmdebstrap" or error "failed to unlink /etc/apt/apt.conf.d/00mmdebstrap: $!";
+- # apt since 1.6 creates the auxfiles directory. If apt inside the chroot
+- # is older than that, then it will not know how to clean it.
+- if (-e "$options->{root}/var/lib/apt/lists/auxfiles") {
+- rmdir "$options->{root}/var/lib/apt/lists/auxfiles" or die "cannot rmdir /var/lib/apt/lists/auxfiles: $!";
+- }
+-
+ info "cleaning package lists and apt cache...";
+ run_apt_progress({
+ ARGV => ['apt-get', '--option', 'Dir::Etc::SourceList=/dev/null', 'update'],
+ });
+ run_apt_progress({ ARGV => ['apt-get', 'clean'] });
+
++ # apt since 1.6 creates the auxfiles directory. If apt inside the chroot
++ # is older than that, then it will not know how to clean it.
++ if (-e "$options->{root}/var/lib/apt/lists/auxfiles") {
++ rmdir "$options->{root}/var/lib/apt/lists/auxfiles" or die "cannot rmdir /var/lib/apt/lists/auxfiles: $!";
++ }
++
+ if (defined $options->{qemu} and $options->{mode} ne 'proot' and $options->{mode} ne 'fakechroot') {
+ unlink "$options->{root}/usr/bin/qemu-$options->{qemu}-static" or error "cannot unlink /usr/bin/qemu-$options->{qemu}-static: $!";
+ }
diff -Nru mmdebstrap-0.4.1/debian/patches/0002-chdir-before-apt-get-update-to-accomodate-for-apt-1..patch mmdebstrap-0.4.1/debian/patches/0002-chdir-before-apt-get-update-to-accomodate-for-apt-1..patch
--- mmdebstrap-0.4.1/debian/patches/0002-chdir-before-apt-get-update-to-accomodate-for-apt-1..patch 1970-01-01 01:00:00.000000000 +0100
+++ mmdebstrap-0.4.1/debian/patches/0002-chdir-before-apt-get-update-to-accomodate-for-apt-1..patch 2019-05-04 23:44:09.000000000 +0200
@@ -0,0 +1,64 @@
+From dd5214a8c893f127e9f161e4dc1182b794523f65 Mon Sep 17 00:00:00 2001
+From: Johannes 'josch' Schauer <josch@mister-muffin.de>
+Date: Thu, 25 Apr 2019 08:49:28 +0200
+Subject: [PATCH 2/6] chdir() before 'apt-get update' to accomodate for apt <<
+ 1.5 (Debian bug #860738)
+
+---
+ mmdebstrap | 16 ++++++++++++----
+ 1 file changed, 12 insertions(+), 4 deletions(-)
+
+--- a/mmdebstrap
++++ b/mmdebstrap
+@@ -535,7 +535,7 @@ sub print_progress {
+ }
+
+ sub run_progress {
+- my ($get_exec, $line_handler, $line_has_error) = @_;
++ my ($get_exec, $line_handler, $line_has_error, $chdir) = @_;
+ pipe my $rfh, my $wfh;
+ my $got_signal = 0;
+ my $ignore = sub {
+@@ -568,6 +568,13 @@ sub run_progress {
+ # redirect stderr to stdout so that we can capture it
+ open(STDERR, '>&', STDOUT);
+ my @execargs = $get_exec->($fd);
++ # before apt 1.5, "apt-get update" attempted to chdir() into the
++ # working directory. This will fail if the current working directory
++ # is not accessible by the user (for example in unshare mode). See
++ # Debian bug #860738
++ if (defined $chdir) {
++ chdir $chdir or error "failed chdir() to $chdir: $!";
++ }
+ exec { $execargs[0] } @execargs or error 'cannot exec() ' . (join ' ', @execargs);
+ }
+ close $wfh;
+@@ -687,7 +694,7 @@ sub run_apt_progress {
+ return $2;
+ }
+ };
+- run_progress $get_exec, $line_handler, $line_has_error;
++ run_progress $get_exec, $line_handler, $line_has_error, $options->{CHDIR};
+ }
+
+ sub run_chroot(&$) {
+@@ -1126,7 +1133,7 @@ sub setup {
+ run_hooks('setup', $options);
+
+ info "running apt-get update...";
+- run_apt_progress({ ARGV => ['apt-get', 'update'] });
++ run_apt_progress({ ARGV => ['apt-get', 'update'], CHDIR => $options->{root} });
+
+ # check if anything was downloaded at all
+ {
+@@ -1655,8 +1662,9 @@ sub setup {
+ info "cleaning package lists and apt cache...";
+ run_apt_progress({
+ ARGV => ['apt-get', '--option', 'Dir::Etc::SourceList=/dev/null', 'update'],
++ CHDIR => $options->{root},
+ });
+- run_apt_progress({ ARGV => ['apt-get', 'clean'] });
++ run_apt_progress({ ARGV => ['apt-get', 'clean'], CHDIR => $options->{root} });
+
+ # apt since 1.6 creates the auxfiles directory. If apt inside the chroot
+ # is older than that, then it will not know how to clean it.
diff -Nru mmdebstrap-0.4.1/debian/patches/0003-add-Dir-State-Status-to-apt-config-for-apt-1.3.patch mmdebstrap-0.4.1/debian/patches/0003-add-Dir-State-Status-to-apt-config-for-apt-1.3.patch
--- mmdebstrap-0.4.1/debian/patches/0003-add-Dir-State-Status-to-apt-config-for-apt-1.3.patch 1970-01-01 01:00:00.000000000 +0100
+++ mmdebstrap-0.4.1/debian/patches/0003-add-Dir-State-Status-to-apt-config-for-apt-1.3.patch 2019-05-04 23:44:11.000000000 +0200
@@ -0,0 +1,20 @@
+From 14d3a4e30f3652261fdeac1a749d82796f2d7c40 Mon Sep 17 00:00:00 2001
+From: Johannes 'josch' Schauer <josch@mister-muffin.de>
+Date: Thu, 25 Apr 2019 08:51:42 +0200
+Subject: [PATCH 3/6] add Dir::State::Status to apt config for apt << 1.3
+
+---
+ mmdebstrap | 2 ++
+ 1 file changed, 2 insertions(+)
+
+--- a/mmdebstrap
++++ b/mmdebstrap
+@@ -949,6 +949,8 @@ sub setup {
+ print $conf "Apt::Architectures \"$options->{nativearch}\";\n";
+ }
+ print $conf "Dir \"$options->{root}\";\n";
++ # not needed anymore for apt 1.3 and newer
++ print $conf "Dir::State::Status \"$options->{root}/var/lib/dpkg/status\";\n";
+ # for authentication, use the keyrings from the host
+ print $conf "Dir::Etc::Trusted \"/etc/apt/trusted.gpg\";\n";
+ print $conf "Dir::Etc::TrustedParts \"/etc/apt/trusted.gpg.d\";\n";
diff -Nru mmdebstrap-0.4.1/debian/patches/0004-chmod-0755-on-qemu-user-static-binary.patch mmdebstrap-0.4.1/debian/patches/0004-chmod-0755-on-qemu-user-static-binary.patch
--- mmdebstrap-0.4.1/debian/patches/0004-chmod-0755-on-qemu-user-static-binary.patch 1970-01-01 01:00:00.000000000 +0100
+++ mmdebstrap-0.4.1/debian/patches/0004-chmod-0755-on-qemu-user-static-binary.patch 2019-05-04 23:44:13.000000000 +0200
@@ -0,0 +1,23 @@
+From ca37c4e89e5f6d6fd8c77e44e243d4c25a95928c Mon Sep 17 00:00:00 2001
+From: Johannes 'josch' Schauer <josch@mister-muffin.de>
+Date: Thu, 25 Apr 2019 08:54:31 +0200
+Subject: [PATCH 4/6] chmod 0755 on qemu-user-static binary
+
+---
+ mmdebstrap | 5 +++++
+ 1 file changed, 5 insertions(+)
+
+--- a/mmdebstrap
++++ b/mmdebstrap
+@@ -1496,6 +1496,11 @@ sub setup {
+ error "cannot find $qemubin";
+ }
+ copy $qemubin, "$options->{root}/$qemubin" or error "cannot copy $qemubin: $!";
++ # File::Copy does not retain permissions but on some
++ # platforms (like Travis CI) the binfmt interpreter must
++ # have the executable bit set or otherwise execve will
++ # fail with EACCES
++ chmod 0755, "$options->{root}/$qemubin" or error "cannot chmod $qemubin: $!";
+ } else {
+ error "unknown mode: $options->{mode}";
+ }
diff -Nru mmdebstrap-0.4.1/debian/patches/0005-chdir-to-parent-of-root-before-remove_tree-to-preven.patch mmdebstrap-0.4.1/debian/patches/0005-chdir-to-parent-of-root-before-remove_tree-to-preven.patch
--- mmdebstrap-0.4.1/debian/patches/0005-chdir-to-parent-of-root-before-remove_tree-to-preven.patch 1970-01-01 01:00:00.000000000 +0100
+++ mmdebstrap-0.4.1/debian/patches/0005-chdir-to-parent-of-root-before-remove_tree-to-preven.patch 2019-05-04 23:44:15.000000000 +0200
@@ -0,0 +1,24 @@
+From c734a59f86e0c2a596cf7f41cdf98bc007f3abd7 Mon Sep 17 00:00:00 2001
+From: Johannes 'josch' Schauer <josch@mister-muffin.de>
+Date: Thu, 25 Apr 2019 08:56:42 +0200
+Subject: [PATCH 5/6] chdir to parent of root before remove_tree to prevent
+ 'cannot stat initial working directory' of File::Path
+
+---
+ mmdebstrap | 5 +++++
+ 1 file changed, 5 insertions(+)
+
+--- a/mmdebstrap
++++ b/mmdebstrap
+@@ -2350,6 +2350,11 @@ sub main() {
+ # Since this is still inside the unshared namespace, there is
+ # no risk of removing anything important.
+ $pid = get_unshare_cmd {
++ # File::Path will produce the error "cannot stat initial
++ # working directory" if the working directory cannot be
++ # accessed by the unprivileged unshared user. Thus, we first
++ # navigate to the parent of the root directory.
++ chdir "$options->{root}/.." or error "unable to chdir() to parent directory of $options->{root}: $!";
+ remove_tree($options->{root}, {error => \my $err});
+ if (@$err) {
+ for my $diag (@$err) {
diff -Nru mmdebstrap-0.4.1/debian/patches/0006-Only-error-out-on-W-and-Err-lines-on-apt-get-update.patch mmdebstrap-0.4.1/debian/patches/0006-Only-error-out-on-W-and-Err-lines-on-apt-get-update.patch
--- mmdebstrap-0.4.1/debian/patches/0006-Only-error-out-on-W-and-Err-lines-on-apt-get-update.patch 1970-01-01 01:00:00.000000000 +0100
+++ mmdebstrap-0.4.1/debian/patches/0006-Only-error-out-on-W-and-Err-lines-on-apt-get-update.patch 2019-05-04 23:44:18.000000000 +0200
@@ -0,0 +1,51 @@
+From da489c6a532ddafff61a35a3a77a2eb5da3e0570 Mon Sep 17 00:00:00 2001
+From: Johannes 'josch' Schauer <josch@mister-muffin.de>
+Date: Tue, 30 Apr 2019 00:07:35 +0200
+Subject: [PATCH 6/6] Only error out on W: and Err: lines, on "apt-get update"
+
+Otherwise, maintainer scripts that output W: or Err: lines will break
+package installation on "apt-get install".
+---
+ mmdebstrap | 24 +++++++++++++++---------
+ 1 file changed, 15 insertions(+), 9 deletions(-)
+
+--- a/mmdebstrap
++++ b/mmdebstrap
+@@ -681,14 +681,18 @@ sub run_apt_progress {
+ '-oDpkg::Use-Pty=false',
+ @debs
+ )};
+- my $line_has_error = sub {
+- # apt-get doesn't report a non-zero exit if the update failed. Thus, we
+- # have to parse its output. See #778357, #776152, #696335 and #745735
+- if ($_[0] =~ /^(W: |Err:)/) {
+- return 1;
+- }
+- return 0;
+- };
++ my $line_has_error = sub { return 0; };
++ if ($options->{FIND_APT_WARNINGS}) {
++ $line_has_error = sub {
++ # apt-get doesn't report a non-zero exit if the update failed.
++ # Thus, we have to parse its output. See #778357, #776152, #696335
++ # and #745735
++ if ($_[0] =~ /^(W: |Err:)/) {
++ return 1;
++ }
++ return 0;
++ };
++ }
+ my $line_handler = sub {
+ if ($_[0] =~ /(pmstatus|dlstatus):[^:]+:(\d+\.\d+):.*/) {
+ return $2;
+@@ -1135,7 +1139,9 @@ sub setup {
+ run_hooks('setup', $options);
+
+ info "running apt-get update...";
+- run_apt_progress({ ARGV => ['apt-get', 'update'], CHDIR => $options->{root} });
++ run_apt_progress({ ARGV => ['apt-get', 'update'],
++ CHDIR => $options->{root},
++ FIND_APT_WARNINGS => 1 });
+
+ # check if anything was downloaded at all
+ {
diff -Nru mmdebstrap-0.4.1/debian/patches/series mmdebstrap-0.4.1/debian/patches/series
--- mmdebstrap-0.4.1/debian/patches/series 1970-01-01 01:00:00.000000000 +0100
+++ mmdebstrap-0.4.1/debian/patches/series 2019-05-04 23:43:51.000000000 +0200
@@ -0,0 +1,6 @@
+0001-cleanup-auxfiles-after-running-apt-get-update-or-oth.patch
+0002-chdir-before-apt-get-update-to-accomodate-for-apt-1..patch
+0003-add-Dir-State-Status-to-apt-config-for-apt-1.3.patch
+0004-chmod-0755-on-qemu-user-static-binary.patch
+0005-chdir-to-parent-of-root-before-remove_tree-to-preven.patch
+0006-Only-error-out-on-W-and-Err-lines-on-apt-get-update.patch
--- End Message ---