Hi, for those who don't read planet.d.o... # Reproducible builds? I never did any - manually :) I've never done a reproducible build attempt of any package, manually, ever. But what I've done now is setting up [reproducible builds] (https://wiki.debian.org/ReproducibleBuilds) on [jenkins.debian.net](https://jenkins.debian.net) which will build hundreds or thousands of packages, hopefully reproducibly, regularily in the future. Thanks to Lunar's and many other peoples work, this was actually rather easy. If you want to do this manually, it should take you just a few minutes to setup a suitable build environment. So three days ago when I wasn't exactly bored I decided that it was a good moment to implement some [reproducible build jobs on jenkins.d.n] (https://jenkins.debian.net/view/reproducible), and so I gave it a try and two hours later the basic implementation was working, and then it was an evening and morning of fine tuning until I was mostly satisfied. Since then there has been some polishing, but the basic setup is done and has been working since. What's the result? One job, [reproducible_setup](https://jenkins.debian.net/view/reproducible/job/reproducible_setup/) will just create a suitable environment for [pbuilding reproducible packages as documented so well](https://wiki.debian.org/ReproducibleBuilds#Usage_example) on the Debian wiki. And as that job runs 3.5 minutes only (to debootstrap from scratch), it's run daily. And then there are currently 16 other jobs, which test reproducible builds in different areas: [d-i](https://jenkins.debian.net/view/reproducible/job/reproducible_build_d- i/), [core](https://jenkins.debian.net/view/reproducible/job/reproducible_build_core/), [some](https://jenkins.debian.net/view/reproducible/job/reproducible_build_gnome/) [six](https://jenkins.debian.net/view/reproducible/job/reproducible_build_kde/) [major](https://jenkins.debian.net/view/reproducible/job/reproducible_build_xfce/) [desktops](https://jenkins.debian.net/view/reproducible/job/reproducible_build_lxde/) [and](https://jenkins.debian.net/view/reproducible/job/reproducible_build_mate/) [some](https://jenkins.debian.net/view/reproducible/job/reproducible_build_cinnamon/) selected [desktop applications] (https://jenkins.debian.net/view/reproducible/job/reproducible_build_desktop-apps/), some [security + privacy] (https://jenkins.debian.net/view/reproducible/job/reproducible_build_security-privacy/) related packages, some [build chains] (https://jenkins.debian.net/view/reproducible/job/reproducible_build_build-tools) we have in Debian, [libreoffice] (https://jenkins.debian.net/view/reproducible/job/reproducible_build_libreoffice) and [X.org](https://jenkins.debian.net/view/reproducible/job/reproducible_build_xorg/). Most of these jobs run several hours, but luckily not days. And they discover packages which still fail to build reproducibly, which already has caused some bugs to be filed, eg. [#762732 "libdebian-installer: please do not write timestamps in Doxygen generated documentation"](https://bugs.debian.org/762732). So this is the [output from testing the reproducibilty of all debian-installer packages](https://jenkins.debian.net/view/reproducible/job/reproducible_build_d- i/lastBuild/console): 72 packages were successfully built reproducibly, while 6 packages failed to do so. I was quite impressed by these numbers as AFAIK noone tried to build d-i reproducibly before. <pre> 72 packages successfully built reproducibly: userdevfs user-setup usb-discover udpkg tzsetup rootskel rootskel-gtk rescue preseed pkgsel partman-xfs partman-target partman- partitioning partman-nbd partman-multipath partman-md partman-lvm partman-jfs partman-iscsi partman-ext3 partman-efi partman-crypto partman-btrfs partman-basicmethods partman-basicfilesystems partman-base partman-auto partman-auto-raid partman-auto-lvm partman-auto-crypto partconf os-prober oldsys-preseed nobootloader network-console netcfg net-retriever mountmedia mklibs media-retriever mdcfg main-menu lvmcfg lowmem localechooser live-installer lilo-installer kickseed kernel-wedge kbd-chooser iso-scan installation-report installation-locale hw-detect grub-installer finish-install efi-reader dh-di debian-installer-utils debian-installer-netboot-images debian-installer- launcher clock-setup choose-mirror cdrom-retriever cdrom-detect cdrom-checker cdebconf-terminal cdebconf-entropy bterm-unifont base-installer apt-setup anna 6 packages failed to built reproducibly: win32-loader libdebian-installer debootstrap console-setup cdebconf busybox </pre> What's also impressive: all packages for the newly introduced [Cinnamon Desktop build reproducibly] (https://jenkins.debian.net/view/reproducible/job/reproducible_build_cinnamon/1/console) from the start! The jenkins setup is configured via just three small files: - [job-cfg/reproducible.yaml](http://anonscm.debian.org/cgit/qa/jenkins.debian.net.git/tree/job-cfg/reproducible.yaml) - defines which jobs exist and which source packages are to be build by them. - [bin/reproducible_setup.sh](http://anonscm.debian.org/cgit/qa/jenkins.debian.net.git/tree/bin/reproducible_setup.sh) - creates a base-reproducible.tgz for pbuilder with the (currently) five packages which need patches to support reproducible building. - [bin/reproducible_build.sh](http://anonscm.debian.org/cgit/qa/jenkins.debian.net.git/tree/bin/reproducible_build.sh) - tests whether given source packages from sid build reproducibly. That's it and that's enough to keep several cores busy for days. :-) But as each job only takes a few _hours_ each is scheduled twice a month and more jobs and packages shall be added in future (with some heuristics to schedule known good packages less often...) I guess it's an appropriate opportunity to say "many thanks to [Profitbricks](https://www.profitbricks.com)", who have been donating the powerful virtual machine jenkins.debian.net is running on since October 2012. I also want to say "many many thanks to Helmut" (Grohne) who has recently joined me in maintaining this jenkins setup. And then I'd like to thank "the KGB trio" (Gregor, Tincho and Dam!) for providing those KGB bots on IRC, which are very helpful for providing notifications on IRC channels and last but not least thanks to everybody who contributed so that reproducible builds got this far! Keep up the jolly good work! And If you happen to know failing packages not included in [job-cfg/reproducible.yaml](http://anonscm.debian.org/cgit/qa/jenkins.debian.net.git/tree/job- cfg/reproducible.yaml) I'd like to hear about those, so they'll get regularily tested and appear on the radar, until finally bugs are filed, fixed and migrated to stable. So one day all binary packages in Debian stable will be build reproducibly. An important step on this road is probably to have this defined as an release goal for Jessie+1. And then for jessie+1 hopefully the first 10k packages will build reproducibly? Or whooping 23k maybe? ;-) And maybe release jessie+2 with 100%?!? We will see! Even Jessie already has quite some packages (someone needs to count them...) which build reproducibly with just modified dpkg(-dev) and debhelper packages alone... So let's [fix all the bugs](https://bugs.debian.org/cgi-bin/pkgreport.cgi?usertag=reproducible-builds@lists.alioth.debian.org)! That said, an easier start for most of you is probably [the list of useful things you (yes, you!) can do!](https://wiki.debian.org/ReproducibleBuilds#Useful_things_you_.28yes.2C_you.21.29_can_do) :-) Oh, and last but surely not least in my book: many thanks too to the nice people hosting me so friendly in the last days! Keep on rockin'! cheer, Holer
Attachment:
signature.asc
Description: This is a digitally signed message part.