System for automatically forward porting local changes to new versions from Debian.
As part of maintaining Raspbian a common task is bringing changes
forward to new versions of the Debian packages. While I do try to push
high quality changes up to Debian one inevitablly has some changes that
either you don't want to push upstream because they are dirty hacks or
cases where the maintainers are unresponsive or changes to a stable
release where release policy would prevent pushing them upstream for
that release or whatever.
The process of bringing forward changes to new versions from Debian
isn't massively complicated but it is tedious and error prone to do
manually, so I decided to put together a system to (semi-) automate it.
I expect there are other deriviatives with similar needs though the
exact details of what they want will vary. There are probablly also
people who want such a system just for their private packages.
Right now the system has a number of hardcoded raspbian specific
assumptions. It is likely that other users will want to change them. I'm
not sure if it's worth trying to introduce a config file or if it's
simpler to just have people running the system maintain branches
customised to their setup and processes.
It's likely that there will be further improvements to the code. For
example specific code for handling debian/patches/series is high on the
todo list (right now it's just handled as a generic file which leads to
a high probability of patch failure for packages that add quilt patches).
============obtaining the code=============
The code is available from https://github.com/plugwash/autoforwardporter
you will also need svn://svn.debian.org/raspbian/raspbian-tools see later.
========basic process description============
1: generate debdiffs for existing modified versions.
2: work out what packages need changes to be forward ported and what the
version number of the new package should be.
3: Seperate the changes to debian/changelog from the rest of the
debdiff, extract the changelog entries and combine them together into a
new changelog entry which is then recombined with the debdiff.
4: Combine the processed debdiff with the new source package from debian
and build a new source package.
5: Use sbuild to build the new package.
=========== Assumptions=====================
There will be a working repository in /build/workingrepo managed by
reprepro. For each release this will contain three suites <codename> is
the main suite for the release of Raspbian. <codename>-staging is the
staging suite for codename (where we import and build packages before
doing some checks and pushing them to the main suite) and <codename>-deb
will reflect what is currently in Debian. With the current code the repo
only strictly needs to contain source packages, right now on my setup it
also contains all binaries for raspbian and arch all binaries for Debian
but the forwardporter code doesn't use them for anything. The suites are
expected to have update configurations in place so the autoforwarporter
can update them with reprepro update
There will be an archive of source packages in /build/sourcearchive .
The files are arranged in debian pool style paths (e.g. main/f/foo ,
main/libf/libfoo). The code will add source packages to this archive but
it is likely it will need to be seeded by manually adding source
packages when first setting the system up.
There will be a directory called /build/debdiffs for storing debdiffs
There will be a directory called /build/autoforwardporter. This will
contain files called <codename>.whitelist containining a whitelist of
packages to forward port for each release
There will be a directory called /build/autoforwardporter/output. This
will contain the output of the autoforwardporter.
Packages modified by raspbian will be marked by +rpi<n> in the version
number. This will usually be at the end but in some cases it may be
earlier to maintain the required version ordering. These will have their
changes forward ported (if they are in the whitelist)
Packages previously modified by raspbian but reverted to unmodified
source from Debian may have +rvt in their version number for version
ordering reasons. These will also be processed by the forward porter
where needed for version ordering reasons (e.g. if openssl is previously
at 1.0.1e-2+rvt+deb7u16 and debian releases 1.0.1e-2+deb7u17 then we
need to generate 1.0.1e-2+rvt+deb7u17)
sbuild chroots should be available for the staging suites, we use btrfs
snapshot mode but you should be able to use any mode.
The contents of svn://svn.debian.org/raspbian/raspbian-tools need be
downloaded and built in /home/plugwash/oodfinder (yes I KNOW this path
is ugly.......)
The autoforwardporter binaries need to be on the path (I put them in
~/bin of the user who will run them)
========usage========
The forward port process is started by running
forwardportdriver <codename>
I currently handle checking the resulting packages for armv7
contamination and uploading them to raspbian manually. I may change this
later.
Reply to: