Re: Standardizing the layout of git packaging repositories
On 08/17/2014 03:13 AM, Raphael Hertzog wrote:
> Hi Thomas,
>
> On Sat, 16 Aug 2014, Thomas Goirand wrote:
>>> The goal here is to be able to host in the same repository the branches for
>>> multiple cooperating distributions (at least so that downstream can
>>> clone the debian respository and inject their branches next to the
>>> debian branches).
>>
>> I generally use debian/unstable, but sometimes, it's best to follow
>> upstream (for OpenStack, I use debian/icehouse, debian/juno, etc.), so
>> there's no one-size-fit-all here.
>
> Branches are cheap so you can easily have both if the upstream-based
> branches bring you value. But it's disconcerting for random Debian
> contributors to not have a clear mapping with Debian releases.
I do have debian/unstable, debian/wheezy and so on as well. Though for
this specific case, debian/icehouse and debian/juno (the last 2
examples) makes a lot more sense. Also, I do expect any new contributor
to at least know the names of upstream releases...
>> Why would you tag the upstream release? I mean, it's upstream's job to
>> do so, and it's natural to use their git tagging and their git
>> repository, no?
>
> Ideally, yes, but:
> - not all upstreams use git (and in those case we want to create our own
> tags pointing to synthetic commits created by tools like gbp import-orig)
> - among upstream using git, some are not creating proper tags/releases
> (and we are doing releases based on snapshot tarballs where we are
> creating our own version like "0~20140812")
> - among upstream who are creating tags, there's no single naming
> convention that is shared (and it can be useful to duplicate the
> upstream tags in a namespace of our own)
>
> Obviously, when upstream are already doing everything correctly, creating
> the upstream/<version> tag should not become some administrative chore but
> it could be done automatically as part of a some "gbp upstream-merge
> <upstream-tag>" command for example.
Ah, that's an interesting topic! :)
Often, upstream are using v1.2.3. Previously, I was doing, in
debian/gbp.conf:
upstream-tag = v%(version)s
But then I decided it was really annoying, and since, I just do:
git tag 1.2.3 v1.2.3
The very good thing is that, since the tag points to the same object,
version 1.2.3 is also PGP signed, just like v1.2.3! :)
>>> - where should the HEAD point to in the public repository?
>>
>> IMO, it should point to the packaging branch for Sid, but YMMV. Why is
>> this important?
>
> It's the default branch one gets when you do "git clone", it's better if
> the user ends up on some useful branch. I agree with you, it should
> probably be <vendor>/master (assuming we keep that branch naming).
Thanks to the awesome vcswatch of Myon, I have checked and all my
package VCSes are pointing to the correct debian/unstable branch by
default. git symbolic-ref is your friend here. Again, please don't
settle on anything containing "master"... :)
>>> - the above layout is for the traditional case of non-native packages,
>>> what would be the layout for native packages? how can be differentiate
>>> between native/non-native layout?
>>
>> Sorry, which layout are you talking about? With pristine-tar? Well, I
>> don't think using pristine-tar is in any way "traditional", it's just
>> one of the workflow, which I always avoid if upstream is using Git and
>> has correct tagging.
>
> This question had nothing to do with pristine-tar. It's just that if you look
> at the dpkg repository, we are doing upstream development in Debian and
> are thus using the master branch directly (and it would not make sense for
> us to start using debian/master). Similarly, since we are also upstream,
> it would not make much sense for us to create upstream/<version> tags...
>
> Hence the question of how to adapt the conventions for this specific case.
In such case, just use "unstable" or "wheezy", IMO. Or continue to use
"debian/unstable" or "debian/wheezy" (which is what I do and would prefer).
>>> - are there other important things to standardize?
>>
>> Yes. Producing orig.tar.xz out of upstream tag should be industrialized,
>> and written in "some" tools, which we would all be using. I currently
>> do: ./debian/rules gen-orig-xz, but that shouldn't be specific to my own
>> packages.
>
> Elsewhere you seem to imply that "git archive" should be enough. So what
> is there to industrialize here?
What I do is:
git archive --prefix=$(DEBPKGNAME)-$(VERSION)/ $(GIT_TAG) \
| xz >../$(DEBPKGNAME)_$(VERSION).orig.tar.xz
DEBPKGNAME, VERSION, and GIT_TAG are all coming from dpkg-parsechangelog
with some parsing (sed, awk, grep...). Because I don't want to write
this in all of my debian/rules, I do some (non-mandatory, so prefixed
with a dash) include. I'd like to see this go into a Debian standard
tool, so that I don't have to do such include.
Also, fetching the branches from upstream VCS should also be automated.
Adding a field for upstream Git in debian/control sounds like something
we should do:
Upstream-Vcs-Git: git://<somesite>.com/<whatever>
Then something like (any better command name would be ok):
gbp fetch-upstream
would do:
git add remote upstream git://<somesite>.com/<whatever>
git fetch upstream
Then you only have to do:
git merge -X <upstream-version>
to get upstream sources in your Debian packaging branch. That's what I
have set in openstack-pkg-tools (from which I include
/usr/share/openstack-pkg-tools/pkgos.make). What I wrote is not rocket
science, could be improved, etc. But it is necessary helpers. There's no
way I'd do this by hand on a daily basis.
Cheers,
Thomas Goirand (zigo)
Reply to: