[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Notes from the DebConf Source Format BoF

Hi Roger,
On Sun, Aug 15, 2010 at 10:28:52PM +0100, Roger Leigh wrote:
> On Tue, Aug 10, 2010 at 08:27:24PM -0700, Russ Allbery wrote:
> > Best practices for Git repository layout?
> > - git-buildpackage documentation is closest to that
> I would have to disagree here, the git-buildpackage default layout is
> far too "Debian-centric".  By naming the Debian and Upstream branches
> "master" and "upstream" it's only really useful if you're importing
> upstream release tarballs.  We should really be using a "debian" branch
> for Debian-specific changes, and possibly even using multiple branches
> for tracking oldstable/stable/unstable/experimental work.

When tracking upstreams VCS I usually put upstream's branches into their
own "namespace" like:


I then use master for sid and create branches for the other
distributions like

    master        (current sid develoment)
    lenny         (lenny stable updates)
    bpo-lenny     (lenny-backports)xi
    experimental  (experimental)

etc. We could also move all the debian work into it's own namespace


but I think that's rather overkill and more typing. When cloning from
git.debian.org I'd like to have the current development on master,
because that's what one expects to happen in a git repository of a
Debian package.
 -- Guido

> If upstream is already using git, you might want to skip the tarball
> step and use their git branches directly (and they might have their
> own master branch).  Also potentially annoying for our downstreams as
> well.
> > git push as an upload mechanism
> > - Attractive because over time it builds a Git repository for the package
> > - However, it assumes binaryless uploads, which we currently don't allow.
> This is something to think about for the future though; dropping
> binary uploads (by maintainers, not buildds) has been on the cards
> for some time now hasn't it?  Is this still planned?
> > If you're implementing 3.0 format, please don't hard-code the extensions that
> > you "know" will be found in source packages, because as we add additional
> > files listed in *.dsc, we may add other types of files.
> We already found this out the hard way in sbuild; hopefully it's now
> completely clean--we removed all assumptions about the expected
> extensions.
> > What about repository size bloat if revision control history is included?
> In practice, a shallow clone is typically only half the size of a
> complete clone, so it's not going to eat too much extra archive space.
> For schroot:
> % du -sk schroot-shallow schroot schroot-full
> 4372	schroot-shallow
> 7556	schroot [cloned --depth 1 and then fetched all history]
> 6008	schroot-full
> % du -sk schroot-shallow/.git schroot/.git schroot-full/.git
> 1720	schroot-shallow/.git
> 4904	schroot/.git [cloned --depth 1 and then fetched all history]
> 3356	schroot-full/.git
> After repack and gc:
> %  du -sk schroot-shallow/.git schroot/.git schroot-full/.git 
> 1520	schroot-shallow/.git
> 2920	schroot/.git
> 2916	schroot-full/.git
> Packaged .git (after repack and gc):
>  ls -l schroot*.bz2                             
> -rw-r--r-- 1 rleigh rleigh 2765372 Aug 15 21:50 schroot-full.tar.bz2
> -rw-r--r-- 1 rleigh rleigh 1403301 Aug 15 21:50 schroot-shallow.tar.bz2
> -rw-r--r-- 1 rleigh rleigh 2764894 Aug 15 21:50 schroot.tar.bz2
> So a five year history in this case is slightly less than double the
> packed size--not a bad tradeoff for the entire project history (IMO).
> Obviously for exceptional cases such as the Linux kernel this might
> not be quite so optimal.  Not sure why there's a size difference if
> you shallow clone then fetch all, rather than cloning the entire
> thing--any history differences or just packed slightly differently?
> > Currently in 3.0 (git), origin points to the bundle and doesn't embed the
> > actual repository, but Joey is working on fixing that.  (Setting origin
> > based on Vcs-Git.)
> As I mentioned above, would it make sense to set Vcs-Git based on origin
> on packing?  On unpack after debcheckout the opposite may be useful as
> you say above.
> > source.debian.org is working on importing source packages into a Git
> > repository and storing the history as one revision per new source package
> > upload.
> While useful, don't we already have that if you're properly tagging
> all Debian releases in your git repository already?  A central resource
> would be useful in case the original repos go offline, but given the
> space requirements, storing all the history should be possible, in which
> case why not simply track the upstream(s)?
> Lastly, one thing I'd like to push with git usage in Debian is
> better integration with upstreams.  Rather than repeating it
> all here, this is detailed in these mails:
>   http://lists.alioth.debian.org/pipermail/vcs-pkg-discuss/2009-August/000663.html
>   http://thread.gmane.org/gmane.comp.sysutils.automake.general/10936
> Essentially, *everything* stays in git from upstream to distributed
> releases to debian work and releases and also to downstreams.  There's
> no import of release tarballs because they are in git too, and there's
> no pristine tar because the GPG-signed tag of the distribution *is* the
> release.  Currently what an upstream releases as the tarball might not
> exactly match the release in the VCS (due to autotools bootstrap, other
> generated files etc.) so here "make dist" actually makes a separate
> "distribution" branch (as opposed to release) so you have a natural
> set of branches:
>   development → release → distribution → debian →→ downstream
> and at each step you have GPG-signed tags giving you an auditable
> chain of trust along the path.
> This makes pulling changes from upstream and downstream trivial since
> you have a contiguous history from end-to-end and the same applies to
> pushing changes back e.g. with rebasing and format-patch.  This is
> lost if importing tarballs.  Here, you can merge back a bugfix from
> a downstream or debian straight back to the current upstream
> development branch and submit it in a jiffy.  And upstreams/downstreams
> can directly track what we're doing and cherry-pick changes at well.
> Regards,
> Roger
> -- 
>   .''`.  Roger Leigh
>  : :' :  Debian GNU/Linux             http://people.debian.org/~rleigh/
>  `. `'   Printing on GNU/Linux?       http://gutenprint.sourceforge.net/
>    `-    GPG Public Key: 0x25BFB848   Please GPG sign your mail.

Reply to: