Re: dpkg-source's future and relation with VCS

[Guillem Jover <guillem@debian.org>]

Hi,

On Mon, 2008-02-11 at 14:46:33 +0100, Raphael Hertzog wrote:
> On Mon, 11 Feb 2008, Frank Lichtenheld wrote:
> > For me this sentence is the essence of your mail: That you dislike
> > Joey's idea since it includes the VCS in the source package itself and
> > would prefer that we worked on creating a solution on top of wig&pen
> > instead.
> > 
> > Do I understand that correctly?
> 
> No. I don't dislike it at all, I do like distributed VCS and the workflow
> and I believe that when upstream is using a DVCS, the $vcs.tar.gz is a
> really nice approach.

I've been also thinking about all this (but didn't comment on the
initial discussion as I hadn't make up my mind), I do like DVCS as
well, and agree that when upstream is using a DVCS ideally the Debian
maintainer would use that to keep track of the changes, but not on the
source packages produced! So I don't quite like Joey's idea in its
current form, for the reasons other people have stated on the thread:

 * have to commit generated files
 * have to reupload all previous history every time
 * have to use shallow clones (git) otherwise it bloats
 * we switch from a myriad patch systems to a myriad DVCS
   - implies needing to install the DVCS to unpack the sources
   - implies needing knowledge about the DVCS to work with the source
 * not 100% guaranteed future compatibility with repo format
 * it forces specific workflows on the DVCS

Also the fact that a source package is a release format, a snapshot.
I don't see usually upstreams release .git tarballs, it does not make
sense. One can always use the upstream VCS anyway (or the one
containing the Debian packaging).

If, for example, we'd want to fix the problem of having to reupload
the previous history on each upload, by supporting incremental deltas,
this is mostly reinventing 'git push' in the git case. And then there's
the problem (minor) that one has to increase the upstream version and
reuplod the whole thing in case one wants to switch DVCS.

Patches are the common denominators, any VCS will be able to import and
export from them. That does not mean people has to work directly with
the source package, I think it makes the most sense to consider it
mostly an export and import format, but still basic enough so that more
complex tools can be built on top of it to handle or convert from/to it.

So I think this is the wrong approach, it seems to be reinventing the
DVCS (cloning, pushing, etc), poorly, and probably something like the
NoMoreSourcePackages[0] makes more sense, and shields all those VCS
behind ftp-master/buildds, and we get one standardized source format as
well as output, and the benefits of maintainers directly using DVCS or
even Non-DVCS.

About Wig & Pen (and Anthony's comments about lack of implementation),
I've just written a really simple and stupid script (proof of concept)
that converts an existing package (using quilt) to it. Can be tried
with something like, and it's attached to this mail:

  apt-get source fbset ppmd
  dsc2wnp fbset*.dsc
  dsc2wnp ppmd*.dsc
  dpkg-source -x *.dsc

regards,
guillem

[0] http://lists.debian.org/debian-devel/2006/07/msg00544.html

#!/bin/bash
#
# Proof of concept Wig & Pen package converter
#
# Required: gpg, procmail, perl, patchutils

set -e

fdsc=$1

echo "Switching $fdsc to Wig&Pen format..."

cp -f $fdsc $fdsc.orig

echo " getting rid of gpg signature..."

if gpg -d $fdsc > $fdsc.asc 2>/dev/null; then
  mv $fdsc.asc $fdsc
fi

echo " extracting information..."

dscfiles=`formail -z -xFiles <$fdsc | cut -d' ' -f4`
pkgdir=`echo "$dscfiles" | sed -n -e 's:_:-:;s:\.orig\.tar\.gz::p'`
fdiff=`echo "$dscfiles" | grep 'diff\.gz$'`
fdebian=`echo $fdiff | sed -e 's:diff:debian.tar:'`

paths="`zcat $fdiff | lsdiff --strip 1 | grep -v '^debian' || true`"
if [ -n "$paths" ]; then
  echo " diff modifies files outside debian/ dir, bailing out."
  exit 1
fi

echo " extracting sources..."

dpkg-source -x $fdsc

echo " remove quilt series file..."

rm -f $pkgdir/debian/patches/series

echo " rename patches to run-parts expected names..."

cd $pkgdir
rename 's:\.patch$::' debian/patches/*
rename 'tr:_\.:-:' debian/patches/*
cd ..

echo " converting $fdiff to $fdebian..."

tar czf $fdebian -C $pkgdir/debian/ .

echo " fixing $fdsc to refer to $fdebian..."

fdebian_size=`stat -c %s $fdebian`
fdebian_md5=`md5sum $fdebian | sed -e "s:  : $fdebian_size :"`

sed -i -e "s:^.* $fdiff$: $fdebian_md5:" $fdsc

echo " fixing $fdsc Format..."

sed -i -e 's/^Format: 1\.0/Format: 2.0/' $fdsc

echo " cleaning up after conversion..."

rm -f "$fdiff"
rm -rf "$pkgdir"

echo "done."