A while ago there was discussion onlist about making 3.0 (git) use git-bundle as its repository serialization format. I think that is a great improvement from the old ad-hoc format. Attached patch implements it. Notes: * I have not tried to keep compatability with old 3.0 (git) packages. The format is experimental.. * The bundle file has a .git extension. There is no real standard for extensions of git bundles, though .bundle is sometimes used. I choose .git for namespace reasons. Also, a .git bundle file can be cloned just like a .git bare repo directory. * All tags and local branches are included in the bundle, and are restored from the bundle. Whatever branch was checked out when the source package was built will be the one checked out when the source package is unpacked. * Since it clones the bundle to unpack, the bundle is set as the origin repository. The remote git repo configuration is not preserved. (Maybe enhance it so Vcs-Git is automatically set up as a remote?) * All history is currently included (via the --all switch to git-bundle), but I plan to add a format-specific dpkg-source option, to allow filtering of what is included in the bundle. Demo: joey@gnu:~/src/ttyrec>cat debian/source/format 3.0 (git) joey@gnu:~/src/ttyrec>git branch * debian master upstream joey@gnu:~/src/ttyrec>dpkg-source -b . dpkg-source: info: using source format `3.0 (git)' Counting objects: 204, done. Delta compression using up to 2 threads. Compressing objects: 100% (99/99), done. Writing objects: 100% (204/204), 79.94 KiB, done. Total 204 (delta 102), reused 160 (delta 95) dpkg-source: info: building ttyrec in ttyrec_1.0.8-4.dsc joey@gnu:~/src/ttyrec>cat ../ttyrec_1.0.8-4.dsc Format: 3.0 (git) Source: ttyrec Binary: ttyrec Architecture: any Version: 1:1.0.8-4 Maintainer: NIIBE Yutaka <gniibe@fsij.org> Standards-Version: 3.8.0 Build-Depends: debhelper (>= 5), sharutils Checksums-Sha1: d1e2f70b56c041a4360ebb8f2a9d45c8d28faf0f 83369 ttyrec_1.0.8-4.git Checksums-Sha256: 3f489ba4f49606369adb8c17ed45885bb7ead9e151b174ec955198ced1867f8b 83369 ttyrec_1.0.8-4.git Files: ec5322b0da83278f42471cc35d8bd41c 83369 ttyrec_1.0.8-4.git joey@gnu:~/src/ttyrec>file ../ttyrec_1.0.8-4.git ../ttyrec_1.0.8-4.git: Git bundle joey@gnu:~/src/tmp>dpkg-source -x ../ttyrec_1.0.8-4.dsc dpkg-source: warning: extracting unsigned source package (../ttyrec_1.0.8-4.dsc) dpkg-source: info: extracting ttyrec in ttyrec-1.0.8 dpkg-source: info: cloning ttyrec_1.0.8-4.git Initialized empty Git repository in /home/joey/src/tmp/ttyrec-1.0.8/.git/ joey@gnu:~/src/tmp>cd ttyrec-1.0.8 joey@gnu:~/src/tmp/ttyrec-1.0.8>ls Makefile README debian/ io.c io.h ttyplay.1 ttyplay.c ttyrec.1 ttyrec.c ttyrec.h ttytime.1 ttytime.c joey@gnu:~/src/tmp/ttyrec-1.0.8>git branch -a * debian remotes/origin/HEAD -> origin/debian remotes/origin/debian remotes/origin/master remotes/origin/upstream joey@gnu:~/src/tmp/ttyrec-1.0.8>git tag debian/1.0.8-5 upstream/1.0.8 -- see shy jo
From 4ea8342f75f1fd5119b1a0732fb0227b0ab06382 Mon Sep 17 00:00:00 2001
From: Joey Hess <joey@kitenet.net>
Date: Tue, 1 Jun 2010 16:01:35 -0400
Subject: [PATCH] modify 3.0 (git) to use git bundle
Much better than the old approach of a tarball of the .git repository,
the git bundle format is simple to understand and work with, and
doesn't need to be sanitized for security. Much code went away.
---
man/dpkg-source.1 | 28 ++++-
scripts/Dpkg/Source/Package/V3/git.pm | 205 +++------------------------------
2 files changed, 40 insertions(+), 193 deletions(-)
diff --git a/man/dpkg-source.1 b/man/dpkg-source.1
index 5e4d30b..8d52737 100644
--- a/man/dpkg-source.1
+++ b/man/dpkg-source.1
@@ -531,13 +531,33 @@ in the current directory. At least one file must be given.
The generated .dsc file will contain this value in its \fIFormat\fP field
and not "3.0 (custom)".
.
-.SS Format: 3.0 (git) and 3.0 (bzr)
-Those formats are experimental. They generate a single tarball
-containing the corresponding VCS repository.
+.SS Format: 3.0 (git)
+This format is experimental. It uses a bundle of a git repository to hold
+the source of a package.
.PP
.B Extracting
.PP
-The tarball is unpacked and then the VCS is used to checkout the current
+The bundle is cloned to a new git repository.
+.PP
+Note that by default the new repository will have the same branch checked
+out that was checked out in the original source. (Typically "master", but
+it could be anything.) Any other branches will be available, under as
+`remotes/origin/`
+.PP
+.B Building
+.PP
+Before going any further, some checks are done to ensure that we
+don't have any non-ignored uncommitted changes.
+.PP
+\fBgit-bundle\fP(1) is used to generate a bundle of the git repository.
+All branches and tags in the repository are included in the bundle.
+.SS Format: 3.0 (bzr)
+This format is experimental. It generates a single tarball
+containing the bzr repository.
+.PP
+.B Extracting
+.PP
+The tarball is unpacked and then bzr is used to checkout the current
branch.
.PP
.B Building
diff --git a/scripts/Dpkg/Source/Package/V3/git.pm b/scripts/Dpkg/Source/Package/V3/git.pm
index c8e3819..f2e2ea8 100644
--- a/scripts/Dpkg/Source/Package/V3/git.pm
+++ b/scripts/Dpkg/Source/Package/V3/git.pm
@@ -1,7 +1,7 @@
#
# git support for dpkg-source
#
-# Copyright © 2007 Joey Hess <joeyh@debian.org>.
+# Copyright © 2007,2010 Joey Hess <joeyh@debian.org>.
# Copyright © 2008 Frank Lichtenheld <djpig@debian.org>
#
# This program is free software; you can redistribute it and/or modify
@@ -22,21 +22,15 @@ package Dpkg::Source::Package::V3::git;
use strict;
use warnings;
-our $VERSION = "0.01";
+our $VERSION = "1.00";
use base 'Dpkg::Source::Package';
use Cwd;
-use File::Basename;
-use File::Find;
-use File::Temp qw(tempdir);
use Dpkg;
use Dpkg::Gettext;
-use Dpkg::Compression;
use Dpkg::ErrorHandling;
-use Dpkg::Source::Archive;
-use Dpkg::Exit;
use Dpkg::Source::Functions qw(erasedir);
our $CURRENT_MINOR_VERSION = "0";
@@ -70,42 +64,9 @@ sub sanity_check {
$srcdir);
}
- # Symlinks from .git to outside could cause unpack failures, or
- # point to files they shouldn't, so check for and don't allow.
- if (-l "$srcdir/.git") {
- error(_g("%s is a symlink"), "$srcdir/.git");
- }
- my $abs_srcdir = Cwd::abs_path($srcdir);
- find(sub {
- if (-l $_) {
- if (Cwd::abs_path(readlink($_)) !~ /^\Q$abs_srcdir\E(\/|$)/) {
- error(_g("%s is a symlink to outside %s"),
- $File::Find::name, $srcdir);
- }
- }
- }, "$srcdir/.git");
-
return 1;
}
-# Returns a hash of arrays of git config values.
-sub read_git_config {
- my $file = shift;
-
- my %ret;
- open(GIT_CONFIG, '-|', "git", "config", "--file", $file, "--null", "-l") ||
- subprocerr("git config");
- local $/ = "\0";
- while (<GIT_CONFIG>) {
- chomp;
- my ($key, $value) = split(/\n/, $_, 2);
- push @{$ret{$key}}, $value;
- }
- close(GIT_CONFIG) || syserr(_g("git config exited nonzero"));
-
- return \%ret;
-}
-
sub can_build {
my ($self, $dir) = @_;
return (-d "$dir/.git", _g("doesn't contain a git repository"));
@@ -114,24 +75,13 @@ sub can_build {
sub do_build {
my ($self, $dir) = @_;
my @argv = @{$self->{'options'}{'ARGV'}};
-#TODO: warn here?
-# my @tar_ignore = map { "--exclude=$_" } @{$self->{'options'}{'tar_ignore'}};
my $diff_ignore_regexp = $self->{'options'}{'diff_ignore_regexp'};
- $dir =~ s{/+$}{}; # Strip trailing /
- my ($dirname, $updir) = fileparse($dir);
-
if (scalar(@argv)) {
usageerr(_g("-b takes only one parameter with format `%s'"),
$self->{'fields'}{'Format'});
}
- my $sourcepackage = $self->{'fields'}{'Source'};
- my $basenamerev = $self->get_basename(1);
- my $basename = $self->get_basename();
- my $basedirname = $basename;
- $basedirname =~ s/_/-/;
-
sanity_check($dir);
my $old_cwd = getcwd();
@@ -170,166 +120,43 @@ sub do_build {
join(" ", @files));
}
- # git clone isn't used to copy the repo because the it might be an
- # unclonable shallow copy.
- chdir($old_cwd) ||
- syserr(_g("unable to chdir to `%s'"), $old_cwd);
-
- my $tmp = tempdir("$dirname.git.XXXXXX", DIR => $updir);
- push @Dpkg::Exit::handlers, sub { erasedir($tmp) };
- my $tardir = "$tmp/$dirname";
- mkdir($tardir) ||
- syserr(_g("cannot create directory %s"), $tardir);
-
- system("cp", "-a", "$dir/.git", $tardir);
- $? && subprocerr("cp -a $dir/.git $tardir");
- chdir($tardir) ||
- syserr(_g("unable to chdir to `%s'"), $tardir);
-
+ # Create a git bundle of the whole repository content.
# TODO support for creating a shallow clone for those cases where
# uploading the whole repo history is not desired
-
- # Clean up the new repo to save space.
- # First, delete the whole reflog, which is not needed in a
- # distributed source package.
- system("rm", "-rf", ".git/logs");
- $? && subprocerr("rm -rf .git/logs");
- system("git", "gc", "--prune");
- $? && subprocerr("git gc --prune");
-
- # .git/gitweb is created and used by git instaweb and should not be
- # transferwed by source package.
- system("rm", "-rf", ".git/gitweb");
- $? && subprocerr("rm -rf .git/gitweb");
-
- # As an optimisation, remove the index. It will be recreated by git
- # reset during unpack. It's probably small, but you never know, this
- # might save a lot of space. (Also, the index file may not be
- # portable.)
- unlink(".git/index"); # error intentionally ignored
-
+ my $basenamerev = $self->get_basename(1);
+ my $debianfile = "$basenamerev.git";
+ system("git", "bundle", "create", $old_cwd."/".$debianfile, "--all");
+ $? && subprocerr("git bundle");
+
chdir($old_cwd) ||
syserr(_g("unable to chdir to `%s'"), $old_cwd);
- # Create the tar file
- my $debianfile = "$basenamerev.git.tar." . $self->{'options'}{'comp_ext'};
- info(_g("building %s in %s"),
- $sourcepackage, $debianfile);
- my $tar = Dpkg::Source::Archive->new(filename => $debianfile,
- compression => $self->{'options'}{'compression'},
- compression_level => $self->{'options'}{'comp_level'});
- $tar->create('chdir' => $tmp);
- $tar->add_directory($dirname);
- $tar->finish();
-
- erasedir($tmp);
- pop @Dpkg::Exit::handlers;
-
$self->add_file($debianfile);
}
-# Called after a tarball is unpacked, to check out the working copy.
sub do_extract {
my ($self, $newdirectory) = @_;
my $fields = $self->{'fields'};
my $dscdir = $self->{'basedir'};
-
- my $basename = $self->get_basename();
my $basenamerev = $self->get_basename(1);
my @files = $self->get_files();
if (@files > 1) {
- error(_g("format v3.0 uses only one source file"));
+ error(_g("format v3.0 (git) uses only one source file"));
}
- my $tarfile = $files[0];
- if ($tarfile !~ /^\Q$basenamerev\E\.git\.tar\.$compression_re_file_ext$/) {
- error(_g("expected %s, got %s"),
- "$basenamerev.git.tar.$compression_re_file_ext", $tarfile);
+ my $bundle = $files[0];
+ if ($bundle !~ /^\Q$basenamerev\E\.git$/) {
+ error(_g("expected %s, got %s"), "$basenamerev.git", $bundle);
}
erasedir($newdirectory);
- # Extract main tarball
- info(_g("unpacking %s"), $tarfile);
- my $tar = Dpkg::Source::Archive->new(filename => "$dscdir$tarfile");
- $tar->extract($newdirectory);
-
+ # Extract git bundle.
+ info(_g("cloning %s"), $bundle);
+ system("git", "clone", $dscdir.$bundle, $newdirectory);
+ $? && subprocerr("git bundle");
sanity_check($newdirectory);
-
- my $old_cwd = getcwd();
- chdir($newdirectory) ||
- syserr(_g("unable to chdir to `%s'"), $newdirectory);
-
- # Disable git hooks, as unpacking a source package should not
- # involve running code.
- foreach my $hook (glob("./.git/hooks/*")) {
- if (-x $hook) {
- warning(_g("executable bit set on %s; clearing"), $hook);
- chmod(0666 &~ umask(), $hook) ||
- syserr(_g("unable to change permission of `%s'"), $hook);
- }
- }
-
- # This is a paranoia measure, since the index is not normally
- # provided by possibly-untrusted third parties, remove it if
- # present (git will recreate it as needed).
- if (-e ".git/index" || -l ".git/index") {
- unlink(".git/index") ||
- syserr(_g("unable to remove `%s'"), ".git/index");
- }
-
- # Comment out potentially probamatic or annoying stuff in
- # .git/config.
- my $safe_fields = qr/^(
- core\.autocrlf |
- branch\..* |
- remote\..* |
- core\.repositoryformatversion |
- core\.filemode |
- core\.logallrefupdates |
- core\.bare
- )$/x;
- my %config = %{read_git_config(".git/config")};
- foreach my $field (keys %config) {
- if ($field =~ /$safe_fields/) {
- delete $config{$field};
- }
- else {
- system("git", "config", "--file", ".git/config",
- "--unset-all", $field);
- $? && subprocerr("git config --file .git/config --unset-all $field");
- }
- }
- if (%config) {
- warning(_g("modifying .git/config to comment out some settings"));
- open(GIT_CONFIG, ">>", ".git/config") ||
- syserr(_g("unable to append to %s"), ".git/config");
- print GIT_CONFIG "\n# "._g("The following setting(s) were disabled by dpkg-source").":\n";
- foreach my $field (sort keys %config) {
- foreach my $value (@{$config{$field}}) {
- if (defined($value)) {
- print GIT_CONFIG "# $field=$value\n";
- } else {
- print GIT_CONFIG "# $field\n";
- }
- }
- }
- close GIT_CONFIG;
- }
-
- # .git/gitweb is created and used by git instaweb and should not be
- # transferwed by source package.
- system("rm", "-rf", ".git/gitweb");
- $? && subprocerr("rm -rf .git/gitweb");
-
- # git checkout is used to repopulate the WC with files
- # and recreate the index.
- system("git", "checkout", "-f");
- $? && subprocerr("git checkout -f");
-
- chdir($old_cwd) ||
- syserr(_g("unable to chdir to `%s'"), $old_cwd);
}
1;
--
1.7.1
Attachment:
signature.asc
Description: Digital signature