Re: Playing with git-buildpackage
To add a different story to git-buildpackage, here are a couple of stories of
updating packages to new upstream versions that I hope are helpful. I'm
concentrating on the updating stage here because the importing/creation of a
new git repo for packaging should be a trivial step (if it's not, there's
something very wrong), while the process of updating to a new upstream version
is always going to be the step that requires thought.
My approach here could perhaps be described as a lowest common denominator
approach but my intention here is just to keep it simple. I feel that we have
a tendency to overthink our solutions to deal with every single corner case
imaginable and as a result we construct extraordinarily complex wrappers
around wrappers around helpers around wrappers with command line options for
everyday use that make a PhD thesis look short. They become a massive barrier
to new contributors and, much of the time, don't help us as much as we'd like
to believe.
So... in summary, below you will find:
* git for the source package: I need to work on the source so the full
source needs to be there.
* git-buildpackage imports tarballs, builds and tags things.
* quilt patches are managed by quilt. (If I'm sending patches to an
upstream, I'm probably attaching them to a bug tracker anyway, so I can
just attach the file from debian/patches/ -- if I'm importing a patch from
upstream, I can use either `quilt import` or `patch` or even just copy
it into debian/patches/. And I do this so rarely that it's not worth
learning lots of new tools to cover the case where the exact same
patch has been applied both upstream and in the package)
I'm sure there are some whose interactions with upstreams, workflows and
packages warrant more sophistication in the packaging helpers. Great! Let's
not push very complicated helpers at people who don't necessarily need them.
Let's not require everyone to become a git guru to be able to work on a simple
package. Ideally, let's try to make sure that it actually doesn't matter which
of the 3 workflows we've recently outlined are used because the artefacts in
the git repo on alioth are the same in all cases.
(Thanks for the inspiration, Barry)
## i18nspector
I have explicitly set the keyid in ~/.gbp.conf and have a debian/gbp.conf
requesting signed tags so I am prompted for my passphrase during importing to
sign the tags. I know that this upstream is sane and won't have put crap into
the tarball that I need to keep out of git so I can just let pristine-tar
commit it straight away.
$ uscan
[…]
$ git-import-orig ../i18nspector-0.16.tar.gz
[…]
(yes, I should learn to type `gbp import-orig` instead)
I like to check what crazy^Wcool things upstream has been doing in this
release, and also to look for new files, new copyright statements, bugs that
are fixed etc. The following is why I really like having the upstream source in
the git repo rather than having it separate.
$ git diff --stat upstream/0.15 upstream/0.16
[…]
44 files changed, 1571 insertions(+), 717 deletions(-)
$ git diff upstream/0.15 upstream/0.16
This upstream is good to work with -- he makes packaging easy. I don't
actually have any patches to update anyway. I did notice in my review that the
new version fixes a bug.
$ dch -v 0.16-1 "New upstream version."
$ dch --closes 753946
$ git commit debian/changelog -m"Add changelog for new upstream version"
I always do out-of-tree builds -- it's a way of avoiding the different
behaviour of the vcs build helpers and it's also means that if I've got stuff
removed in the clean target I don't have to worry about making the tree dirty.
I could (should?) probably figure out the correct incantation for git-
buildpackage to have this happen for me as Barry did (`--git-export=WC` by the
looks of it)... but I just run what amounts to `dpkg-buildpackage -nc -S -us -
c; sbuild ../foo.dsc` (yes, I have a short script that does that for me in the
same way as everyone has enormously complicated aliases for svn-bp -- it also
suppresses signing of the test builds). sbuild is configured to run lintian for
me too.
Now because I know this particular upstream produces good releases and the diff
looked pretty safe, I'll take an "optimistic" approach here and assume that
what I've got is going to be ok and build and tag at the same time. I can
always come back and `git commit --amend` if I need to later since I've not
pushed this work:
$ dch -r unstable
$ git commit debian/changelog -m"Prepare changelog for release"
$ git-buildpackage --git-tag
[…]
As Barry noted, gbp throws up a dbus notification that this has finished. I
quite like that, especially when I'm working on packages that take a long time
to build (lots of C++ stuff or piles of documentation that requires all of
texlive to be installed etc). Yes, Barry, you can disable that with --git-
notify=off (or in your ~/.gbp.conf).
Now this package has some autopkgtest tests, so I'll throw the thing I've just
built at adt-run too, once again with my own wrapper script because I'm
uninterested in remembering the right options to adt-run:
$ adt-quick ../build-area/i18nspector_0.16-1_amd64.changes
[…]
All looks good! (and yes, I also install the package and test it out locally
now :)
$ dput ftp-master !$
[…]
$ git push --tags origin :
[…]
To git+ssh://stuart@git.debian.org/git/collab-maint/i18nspector.git
1664fc6..4c8fb76 master -> master
59b3e3f..03c0ae1 pristine-tar -> pristine-tar
2c90ca7..074e3fe upstream -> upstream
* [new tag] debian/0.16-1 -> debian/0.16-1
* [new tag] upstream/0.16 -> upstream/0.16
yay!
## translate-toolkit
In other packages, I normally use quilt to work through the patches as needs
be. The approach here is no different to working with packages in svn at this
stage -- if the patches still apply, then it's only a matter of
$ while quilt push; do quilt refresh -p ab; done
But of course, it's not always that easy and no amount of fancy VCS helper is
going to change that. So, as a counter-point, here's another package I've just
updated with a new upstream version. This upstream tends to release larger
lumps of major work and I generally find I've got to very carefully look over
the released tarballs each time. I tend to unpack the source first, check what
new things have appeared in it that need stripping and then use mk-origtargz
to repack the tarball for me.
$ uscan
[…]
$ (cd ..; tar xf translate-toolkit-1.12.0.tar.bz2)
$ diff -r ../translate-toolkit-1.12.0 . | diffstat
360 files changed, 11390 insertions(+), 10753 deletions(-)
*eek* (although that diffstat does tell me some files were removed and that will
actually close some bugs against the package too)
I want to carefully look over things to look for new copyright statements, new
licences and new sourceless files. For a diff this big, I'd personally rather
use a graphical tool:
$ kdiff3 ../translate-toolkit-1.12.0 . &
And yes, I found some new dependencies that need packaging, some new copyright
holders to add to d/copyright and some bugs to fix. I updated debian/copyright
(including Files-Excluded), debian/control and debian/changelog as I went. I
made some notes about things in debian/rules that needed checking.
** some time passes **
I don't really want to commit changes for the new upstream release prior to
importing its tarball. Probably overly pedantic.
$ git stash
$ git-import-orig […] as above
$ git stash apply
This package carries quite a few patches. Some are Debian-specific and just
need to be refreshed, a couple will have been applied upstream.
$ quilt push
Applying patch disable-sphinx-theme.patch
patching file docs/conf.py
Hunk #1 succeeded at 158 (offset 6 lines).
Now at patch disable-sphinx-theme.patch
$ quilt refresh
$ quilt push
Applying patch po2web2py_missing_import.patch
patching file translate/convert/po2web2py.py
Hunk #1 FAILED at 26.
1 out of 1 hunk FAILED -- rejects in file translate/convert/po2web2py.py
Patch po2web2py_missing_import.patch can be reverse-applied
(this is a patch I sent upstream that has been applied -- this is a place
where git-dpm would just eat the patch for me and I'd move on.)
$ quilt delete -n
Removed patch po2web2py_missing_import.patch
$ quilt push -f
Applying patch storage_bzr
patching file translate/storage/versioncontrol/bzr.py
Hunk #1 FAILED at 65.
Hunk #2 FAILED at 103.
2 out of 2 hunks FAILED -- saving rejects to file
translate/storage/versioncontrol/bzr.py.rej
Applied patch storage_bzr (forced; needs refresh)
$ git diff upstream/1.11.0+dfsg upstream/1.12.0+dfsg1 --
translate/storage/versioncontrol/bzr.py
(the upstream code has changed here sufficiently that the patch needs
reworking... even git's 3 way merge is going to fail here since the exact same
lines are touched by the patch and by upstream pep8 changes)
*hack* *hack*
$ quilt refresh
Refreshed patch storage_bzr
$ git diff debian/patches
(yep, looks sane, keep going)
etc.
I'll note here that there are a few cases where git would get a merge right
where quilt is going to fail, because git can do it as a three-way merge and
so with more information, it might succeed. That wouldn't be true in this case
though. I don't believe that git-dpm or git-pq is going to be any simpler or
faster for these steps -- in one you use a few quilt commands and an editor,
the others you use a few git checkout/merge commands and an editor.
$ quilt pop -a
$ git commit debian/copyright -m"Update copyright with new and removed files"
$ git commit debian/patches -m"Refresh patches for new upstream version"
I still need to finish reworking some bits of d/rules to deal with changes in
the upstream build system. That's a job for another day though ... I know I'll
end up iterating on a build-test-debug cycle here a few times.
build, test, upload, push, enjoy -- all as before.
## configuration
For reference, some configuration I use with this:
~/.gitconfig:
[merge "dpkg-mergechangelogs"]
name = debian/changelog merge driver
driver = dpkg-mergechangelogs -m %O %A %B %A
~/.gbp.conf:
[DEFAULT]
keyid = 0x1396F2F7
debian/gbp.conf (in the source package -- applies to everyone)
[DEFAULT]
pristine-tar = True
sign-tags = True
.git/gbp.conf (in my working copy -- applies only to me or could actually go
into ~/.gbp.conf):
[DEFAULT]
builder=sbuild
[git-buildpackage]
export-dir=../build-area
--
Stuart Prescott http://www.nanonanonano.net/ stuart@nanonanonano.net
Debian Developer http://www.debian.org/ stuart@debian.org
GPG fingerprint 90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7
Reply to: