Re: Bug#807940: dpkg-deb: unwanted files in data.tar with tar (>= 1.28)
Control: tags 807940 = confirmed
Control: merge 807940 807943
Control: retitle 807940 dpkg-deb: unwanted files in data.tar with tar (>= 1.28)
Control: reassign 807940 dpkg
Control: found 807940 1.17.25
Control: fixed 807940 1.18.2
Control: affects 807940 game-data-packager
This bug has already been fixed in dpkg. I'm reassigning it as well
as closing it, so we have proper version-tracking.
Briefly, the symptoms of #807940 are "a .deb generated by
game-data-packager contains ./DEBIAN in its data.tar", and the symptoms
of #807943 are "a .deb generated by game-data-packager is many times
the size it should be, causing it to exceed the hard limit on the size
of ar members". They seem to have the same root cause: combining an old
dpkg with a new tar.
Both bugs were reported against game-data-packager, but I've been able
to reproduce similar issues with just dpkg-deb. They were already fixed
in dpkg 1.18.2, 20 days before the version of tar that triggers them
was uploaded to unstable, which is why Debian developers didn't
previously encounter them.
Perhaps tar should have a versioned Breaks on older dpkg releases?
Steps to reproduce:
* have a directory ready for dpkg-deb, containing ./DEBIAN
and ./usr (for your convenience,
https://www.pseudorandom.co.uk/~smcv/soltys-en-data_1.0+43_all.zip
contains what game-data-packager would produce for a smallish
free-to-download game, in a directory named soltys-en-data_1.0+43_all)
* have dpkg (<< 1.18.2); the bug reporter has placed dpkg on hold,
as suggested by apt-listbugs, to avoid #801156 (I used a jessie chroot
to reproduce the underlying bug)
* have tar (>= 1.28) (I installed tar from stretch into the jessie chroot)
* fakeroot dpkg-deb -Znone -b soltys-en-data_1.0+43_all
Expected result:
* soltys-en-data_1.0+43_all/DEBIAN becomes the control.tar
* Files in soltys-en-data_1.0+43_all/DEBIAN are not added to the
data.tar
* Each file in soltys-en-data_1.0+43_all/usr appears in the data.tar
exactly once
Actual result:
* DEBIAN is also copied into the control.tar
* Each file in usr appears multiple times in the data.tar
This appears to have been fixed by this dpkg 1.18.2 change:
> * Move tar option --no-recursion before -T in dpkg-deb. With tar > 1.28 the
> --no-recursion option is now positional, and needs to be passed before
> the -T option, otherwise the tarball will end up with duplicated entries.
Further notes:
By running under strace:
fakeroot strace -eexecve -f dpkg-deb -Znone -b soltys-en-data_1.0+43_all
we can see that tar is indeed invoked with --no-recursion after -T:
[pid 20033] execve("/bin/tar", ["tar", "-cf", "-", "--format=gnu", "--null", "--no-unquote", "-T", "-", "--no-recursion"], [/* 40 vars */] <unfinished ...>
This causes the original symptom of #807940 (./DEBIAN in the data.tar)
because "./DEBIAN" is included in the recursive addition of ".".
This also causes the original symptom of #807943 (game-data-packager
fails to package quake4 or doom3) because adding 1-2 GB of game assets
to the data.tar multiple times causes them to exceed the 32-bit
limit on the size of an ar file member.
Regards,
S
Reply to: