Re: Notes from the DebConf Source Format BoF
Hi Roger,
On Sun, Aug 15, 2010 at 10:28:52PM +0100, Roger Leigh wrote:
> On Tue, Aug 10, 2010 at 08:27:24PM -0700, Russ Allbery wrote:
[..snip..]
> > Best practices for Git repository layout?
> > - git-buildpackage documentation is closest to that
>
> I would have to disagree here, the git-buildpackage default layout is
> far too "Debian-centric". By naming the Debian and Upstream branches
> "master" and "upstream" it's only really useful if you're importing
> upstream release tarballs. We should really be using a "debian" branch
> for Debian-specific changes, and possibly even using multiple branches
> for tracking oldstable/stable/unstable/experimental work.
When tracking upstreams VCS I usually put upstream's branches into their
own "namespace" like:
upstream/master
upstream/2.30
upstream/2.28
I then use master for sid and create branches for the other
distributions like
master (current sid develoment)
lenny (lenny stable updates)
bpo-lenny (lenny-backports)xi
experimental (experimental)
etc. We could also move all the debian work into it's own namespace
like:
debian/sid
debian/lenny
debian/bpo-lenny
debian/experimental
but I think that's rather overkill and more typing. When cloning from
git.debian.org I'd like to have the current development on master,
because that's what one expects to happen in a git repository of a
Debian package.
Cheers,
-- Guido
>
> If upstream is already using git, you might want to skip the tarball
> step and use their git branches directly (and they might have their
> own master branch). Also potentially annoying for our downstreams as
> well.
>
> > git push as an upload mechanism
> > - Attractive because over time it builds a Git repository for the package
> > - However, it assumes binaryless uploads, which we currently don't allow.
>
> This is something to think about for the future though; dropping
> binary uploads (by maintainers, not buildds) has been on the cards
> for some time now hasn't it? Is this still planned?
>
> > If you're implementing 3.0 format, please don't hard-code the extensions that
> > you "know" will be found in source packages, because as we add additional
> > files listed in *.dsc, we may add other types of files.
>
> We already found this out the hard way in sbuild; hopefully it's now
> completely clean--we removed all assumptions about the expected
> extensions.
>
> > What about repository size bloat if revision control history is included?
>
> In practice, a shallow clone is typically only half the size of a
> complete clone, so it's not going to eat too much extra archive space.
> For schroot:
>
> % du -sk schroot-shallow schroot schroot-full
> 4372 schroot-shallow
> 7556 schroot [cloned --depth 1 and then fetched all history]
> 6008 schroot-full
> % du -sk schroot-shallow/.git schroot/.git schroot-full/.git
> 1720 schroot-shallow/.git
> 4904 schroot/.git [cloned --depth 1 and then fetched all history]
> 3356 schroot-full/.git
> After repack and gc:
> % du -sk schroot-shallow/.git schroot/.git schroot-full/.git
> 1520 schroot-shallow/.git
> 2920 schroot/.git
> 2916 schroot-full/.git
> Packaged .git (after repack and gc):
> ls -l schroot*.bz2
> -rw-r--r-- 1 rleigh rleigh 2765372 Aug 15 21:50 schroot-full.tar.bz2
> -rw-r--r-- 1 rleigh rleigh 1403301 Aug 15 21:50 schroot-shallow.tar.bz2
> -rw-r--r-- 1 rleigh rleigh 2764894 Aug 15 21:50 schroot.tar.bz2
>
> So a five year history in this case is slightly less than double the
> packed size--not a bad tradeoff for the entire project history (IMO).
> Obviously for exceptional cases such as the Linux kernel this might
> not be quite so optimal. Not sure why there's a size difference if
> you shallow clone then fetch all, rather than cloning the entire
> thing--any history differences or just packed slightly differently?
>
> > Currently in 3.0 (git), origin points to the bundle and doesn't embed the
> > actual repository, but Joey is working on fixing that. (Setting origin
> > based on Vcs-Git.)
>
> As I mentioned above, would it make sense to set Vcs-Git based on origin
> on packing? On unpack after debcheckout the opposite may be useful as
> you say above.
>
> > source.debian.org is working on importing source packages into a Git
> > repository and storing the history as one revision per new source package
> > upload.
>
> While useful, don't we already have that if you're properly tagging
> all Debian releases in your git repository already? A central resource
> would be useful in case the original repos go offline, but given the
> space requirements, storing all the history should be possible, in which
> case why not simply track the upstream(s)?
>
>
> Lastly, one thing I'd like to push with git usage in Debian is
> better integration with upstreams. Rather than repeating it
> all here, this is detailed in these mails:
>
> http://lists.alioth.debian.org/pipermail/vcs-pkg-discuss/2009-August/000663.html
> http://thread.gmane.org/gmane.comp.sysutils.automake.general/10936
>
> Essentially, *everything* stays in git from upstream to distributed
> releases to debian work and releases and also to downstreams. There's
> no import of release tarballs because they are in git too, and there's
> no pristine tar because the GPG-signed tag of the distribution *is* the
> release. Currently what an upstream releases as the tarball might not
> exactly match the release in the VCS (due to autotools bootstrap, other
> generated files etc.) so here "make dist" actually makes a separate
> "distribution" branch (as opposed to release) so you have a natural
> set of branches:
> development → release → distribution → debian →→ downstream
> and at each step you have GPG-signed tags giving you an auditable
> chain of trust along the path.
>
> This makes pulling changes from upstream and downstream trivial since
> you have a contiguous history from end-to-end and the same applies to
> pushing changes back e.g. with rebasing and format-patch. This is
> lost if importing tarballs. Here, you can merge back a bugfix from
> a downstream or debian straight back to the current upstream
> development branch and submit it in a jiffy. And upstreams/downstreams
> can directly track what we're doing and cherry-pick changes at well.
>
>
> Regards,
> Roger
>
> --
> .''`. Roger Leigh
> : :' : Debian GNU/Linux http://people.debian.org/~rleigh/
> `. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/
> `- GPG Public Key: 0x25BFB848 Please GPG sign your mail.
Reply to: