[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Bug#769844: linux: please make linux build reproducibly



Control: unblock -1 by 759886

Jérémy Bobbio:
> Bastian Blank:
> > On Mon, Nov 17, 2014 at 12:46:45AM +0100, Jérémy Bobbio wrote:
> > > The first patch adds call to `dh_strip_nondeterminism` and
> > > `dh_fixmtimes`, both being part of the custom toolchain currently used
> > > for reproducible builds. Hence not tagging the bug with “patch” until
> > > they are integrated in debhelper.
> > 
> > Why does this need new tool instead of being integrated into the
> > existing ones?
> 
> I am not sure which ones you specifically have in mind, but the whole
> project is still at the experimental stage. We try to work in
> unintrusive ways.

We are currently experimenting with fixing mtimes in `dh_builddeb`
instead of requiring a new helper. I have also done my latest
experiments without `dh_strip_nondeterminism`. The attached patch adds
the `-n` flag to gzip to compensate.

> > > The second patch changes the value of KBUILD_BUILD_TIMESTAMP to a
> > > timestamp parseable by `date`.
> > 
> > Well, no.  The string is this way for a reason.
> 
> Would a patch against `scripts/gen_initramfs_list.sh` to make it parse
> Debian's KBUILD_BUILD_TIMESTAMP be acceptable then? Any other
> suggestions?

Implemented in the attached patch.

> > > An unclear aspect is where to add a call to `dh_genbuildinfo` which
> > > generates the .buildinfo [2]. It should be called after all binary
> > > packages have been created.
> > 
> > Not possible, dh_* acts on single binary packages.
> 
> Mh… I'm not sure we had realized that. It makes a case to move the
> generation of the .buildinfo closer to dpkg-genchanges.

That's how the experimental toolchain now works: the .buildinfo is
generated by dpkg-genbuildinfo, called by dpkg-buildpackage.

> > >  * linux-source: mtimes of many files differ. Would it be ok to just
> > >    create the tarball with a single timestamp (`tar --mtime=`)?
> >
> > Looks like a way.
> 
> Good. :) I will experiment with this approach and probably add another
> patch to this bug report.

I've used find+touch instead. See attached patch.


With the attached patch, my latest build+rebuild showed similar
differences in linux-doc and linux-manual. It is probably related to the
way API documentation is currently extracted from the source code.

I also stumbled on a variation of the Installed-Size field in one of the
.deb. But this is a different topic.

-- 
Lunar                                .''`. 
lunar@debian.org                    : :Ⓐ  :  # apt-get install anarchism
                                    `. `'` 
                                      `-   
diff -Nru linux-3.16.7-ckt2/debian/changelog linux-3.16.7-ckt2/debian/changelog
--- linux-3.16.7-ckt2/debian/changelog	2014-12-08 21:03:19.000000000 +0100
+++ linux-3.16.7-ckt2/debian/changelog	2015-01-05 18:24:20.000000000 +0100
@@ -1,3 +1,11 @@
+linux (3.16.7-ckt2-1.0~reproducible1) UNRELEASED; urgency=low
+
+  * Add a patch to make gen_initramfs.sh parse Debian's KBUILD_BUILD_TIMESTAMP.
+  * Adjust file mtimes before creating source tarball.
+  * Stop recording current time in gzip headers when compressing documentation.
+
+ -- Jérémy Bobbio <lunar@debian.org>  Mon, 05 Jan 2015 14:55:50 +0100
+
 linux (3.16.7-ckt2-1) unstable; urgency=high
 
   * New upstream stable update:
--- linux-3.16.7-ckt2/debian/patches/debianparse-debian-build-timestamp.patch	1970-01-01 01:00:00.000000000 +0100
+++ linux-3.16.7-ckt2/debian/patches/debianparse-debian-build-timestamp.patch	2015-01-05 14:50:58.000000000 +0100
@@ -0,0 +1,26 @@
+Description: parse Debian KBUILD_BUILD_TIMESTAMP in gen_initramfs_list.sh
+ gen_initramfs_list.sh assumes that KBUILD_BUILD_TIMESTAMP only contains a
+ date. In Debian, it's a longer string looking like:
+ “Debian 3.16.7-2 (2014-11-06)”. Here we adapt the script to parse Debian
+ format.
+ .
+ This change is necessary to have reproducible builds as the date will
+ then be used as mtime for files in initramfs archive instead of the
+ current time.
+Author: Jérémy Bobbio <lunar@debian.org>
+Bug-Debian: https://bugs.debian.org/769844
+Forwarded: not-needed
+
+--- linux-3.16.7-ckt2.orig/scripts/gen_initramfs_list.sh
++++ linux-3.16.7-ckt2/scripts/gen_initramfs_list.sh
+@@ -301,7 +301,9 @@ if [ ! -z ${output_file} ]; then
+ 	if [ -z ${cpio_file} ]; then
+ 		timestamp=
+ 		if test -n "$KBUILD_BUILD_TIMESTAMP"; then
+-			timestamp="$(date -d"$KBUILD_BUILD_TIMESTAMP" +%s || :)"
++			source_date=$(echo "$KBUILD_BUILD_TIMESTAMP" |
++					sed -e 's/.*(\([0-9-]\+\)).*/\1/')
++			timestamp="$(date -d"$source_date" +%s || :)"
+ 			if test -n "$timestamp"; then
+ 				timestamp="-t $timestamp"
+ 			fi
diff -Nru linux-3.16.7-ckt2/debian/patches/series linux-3.16.7-ckt2/debian/patches/series
--- linux-3.16.7-ckt2/debian/patches/series	2014-12-08 21:00:20.000000000 +0100
+++ linux-3.16.7-ckt2/debian/patches/series	2015-01-05 14:44:13.000000000 +0100
@@ -484,3 +484,4 @@
 debian/iovec-fix-abi-change-in-3.16.7-ckt1.patch
 debian/truncate-fix-abi-change-in-3.16.7-ckt1.patch
 debian/perf-fix-abi-change-in-3.16.7-ckt2.patch
+debianparse-debian-build-timestamp.patch
--- linux-3.16.7-ckt2/debian/rules.real	2014-11-04 05:41:34.000000000 +0100
+++ linux-3.16.7-ckt2/debian/rules.real	2015-01-05 18:23:52.000000000 +0100
@@ -87,6 +87,8 @@
 	rm -rf '$@' '$(DIR)'
 	$(call copy_source,$(DIR))
 	chmod -R u+rw,go=rX '$(DIR)'
+	find '$(DIR)' -depth -newermt '$(SOURCE_DATE)' -print0 | \
+		xargs -0r touch --no-dereference --date='$(SOURCE_DATE)'
 	cd '$(BUILD_DIR)'; tar -caf 'linux-source-$(UPSTREAMVERSION).tar.xz' 'linux-source-$(UPSTREAMVERSION)'
 	rm -rf '$(DIR)'
 
@@ -202,7 +204,7 @@
 	cd $(DIR)/Documentation/DocBook; \
 	find * -name '*.html' -print | \
 	cpio -pd --preserve-modification-time '$(CURDIR)/$(OUT_DIR)/html'
-	gzip -9qfr $(OUT_DIR)/Documentation
+	gzip -9nqfr $(OUT_DIR)/Documentation
 	+$(MAKE_SELF) install-base
 
 install-manual: PACKAGE_NAME = linux-manual-$(VERSION)

Attachment: signature.asc
Description: Digital signature


Reply to: