Bug#1104854: binNMUs can cause ma-same violations in eg manpages
Package: debian-policy
User: reproducible-builds@lists.alioth.debian.org
Usertags: timestamps
Hi everyone. I'm filing this against policy but it's not clear which
package(s) need to be changed. We might decide to change sbuild
and/or dpkg-source and/or dh and/or lintian; we might decide to change
individual packages and/or policy. I have tried to capture the irc
conversation as I understand it.
Background: We usually use binNMUs for rebuilds. That involves a
binNMU-specfic arch-specific d/changelog fragment which is logically
added to the source changelog. That has the date of the binNMU
request. (Let's call that the "bdate" and the original source package
changelog date the "sdate".)
Since #843773, SOURCE_DATE_EPOCH (henceforth, S_D_E) has been set to
the binNMU date in binNMUs. (In normal uploads its set to the sdate.)
This is because otherwise we have this scenario:
libfoo_1_m68.deb contains /usr/lib/m68k-triplet/foo.so. We ask for a
binNMU on m68k. Now we have foo_1+b1_m68k.deb containing a
*different* foo.so. If we set S_D_E to the sdate, then 1+b1's foo.so
has the same mtime as 1's foo.so. This is a big problem because it
can mislead programs which rely on file mtimes as a way to detect
changes.
Examples of programs which rely on the mtime include backup systems
(which is how I noticed this) but also build tools like ccache and
make. So reusing the sdate mtime might cause an attempted rebuild of
*another* program to not pick up the changed libfoo - potentially a
very serious (and hard to detect and debug) problem.
However, if S_D_E influences the *contents* of shared files in an
ma-same package, we end up with a ma-same violation.
Apparently this has happened in libopts25-dev, in manpages.
Requirements:
1. *Contents* of shared files in ma-same packages must not change as
a result of binNMUs. If they embed a date, they must use the
sdate. Otheerwise it's a ma-same violation.
2. Arch-specific files whose contents change in binNMUs must have
new timestamps - ie, the timestamps must be the bdate.
Otherwise tools like ccache and make can badly malfunction.
3. Ideally, arch-specific files which are archives which embed dates,
would embed the bdate. Otherwise, build systems which use
archive member timestamps for rebuild-needed detection will
malfunction. I don't know if any such build systems exist.
Options that I can see:
A. Shared files in ma-same packages are not allowed to embed S_D_E.
For example, no dates in libopts25-dev's manpages. Other
situations (eg, other docs) will be handled similarly.
A.0. Implement the above by patching out the date requests in the
manpages in the individual package. (Dates in manpages aren't
really very useful anyway.) Add a lintian lint which spots a
ma-same package with dates in manpages.
A.1. Implement the above by having strip-nondeterminism strip all
build dates out of all manpages. (Does making this change involve
an archive-wide rebuild to ensure we can do future binNMUs?)
B. Set S_D_E to the sdate but use the bdate for file mtimes in
the .deb. We would need a separate solution for arch-specific
archives containing mtimes (and IDK how we would even detect
them - and detecting them seems like it might be important to
avoid the bugs described in requirement 3). Transition plan may
be complicated. Also how would the build obtain the *two* dates?
C. Say that affected packages may not be binNMU'd, but instead
require a no-changes source-only upload. It's not clear how we
would avoid cosntantly making the mistake of doing a binNMU
anyway, producing an ma-same violation.
Ian.
--
Ian Jackson <ijackson@chiark.greenend.org.uk> These opinions are my own.
Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.
Reply to: