Re: dpkg-source's future and relation with VCS
Anthony Towns <aj@azure.humbug.org.au> writes:
> A VCS is a system for controlling (managing, distributing) versions of a
> repository.
>
> A Debian source package is a repository.
It's a changeset or a set of changesets. This is not a repository any
more than a collection of objects is a balanced red/black tree. Only a
tiny subset of the changesets and metadata you'd need to make a repository
would be represented in wig&pen because wig&pen would only represent a cut
of the current state.
> It does have history:
>
> upstream: (unpack all the tarballs)
> upstream+1: (apply the first patch from .debian.tgz /patches)
> upstream+2: (apply the second patch from .debian.tgz /patches)
> ...
> debian: (apply the last patch from .debian.tgz /patches)
That isn't history. There isn't a chronological or derivational order,
and the patches are not based on each other in that fashion. That's
dependency management between changesets, which is something else (and
most of the dependencies in a series file are actually fake -- a
limitation of the representation format). This is a component that you
need to construct a VCS, but it isn't itself a VCS. In this case, it's
using that component to do something different than what a VCS is.
Yes, you can implement a VCS with quilt, but if you do, you're abusing
quilt and you'll be very unhappy because it doesn't have the tools that
you need to really do a VCS. But it's a great way of representing
changesets.
I think you're trying to simplify in order to see underlying patterns and
build a foundational framework for reasoning about this area, but in my
opinion you're being reductionist to the point of losing fundamental
distinctions.
>> It simply doesn't bundle together all of the changes into one blob of
>> "Debian work" and instead presents a bunch of separately appliable and
>> analyzable changes.
> Which is exactly what "history" is.
No, it isn't. A history would include the history of each change, for
example, as well as the changes that you had made previously and no longer
do. A history is something much larger than this, includes a
chronological component, and permits reconstruction of previous versions.
"Upstream source with one changeset applied" is not a previous version in
any meaningful definition of previous: it's not conceptually previous, and
since frequently that was the *last* changeset developed, it's not
chronologically previous either.
A patch management system stores only the current state, but with metadata
about how that state is conceptually assembled. It's a cut across a
history. It bears the same resemblence to a history as a core dump does
to a trace of a process over its whole lifetime.
I can switch to mathematics (somewhat) if that helps. A changeset
collection forms a bounded lattice. It is inherently constrained to a
bounded lattice: it describes how to form the top from the bottom. Each
new upstream release of a Debian package represented by a changeset
collection rebuilds the bounded lattice from a new bottom and does not
retain any information (history) about how the previous bounded lattice
was constructed.
A VCS repository constrained to be a bounded lattice is crippled to the
point of not being worth the name. A VCS repository is a generalized
partial order.
> The reason why Wig&Pen has never worked is precisely because it's not a
> real VCS: it's just a repository format, it doesn't have the tools to
> manage a repository. Which means it's not possible to create and
> maintain a package in Wig&Pen format, which is why it's not actually
> usable, inside or outside of Debian.
> For example, you talk about "separately analyzable" changes -- but
> wig&pen doesn't give you that: its changes are ordered to allow multiple
> changes to affect a single file, and unlike tools like darcs, there
> isn't any automated way to deal with dealing with corner cases when
> rearranging changesets; let alone merging changesets.
Right, because it's not a VCS. It doesn't need to provide any of those
things to be useful as an interchange format. If you want to perform
those operations, you import the interchange format into the structure of
a VCS and then use the VCS to handle that, or you go to the original VCS
which has history (in particular, merge history) and therefore a much
richer set of metadata to work from.
No one is questioning that you can't do everything you might potentially
want to do with wig&pen. The point is that wig&pen is a useful
interchange format that's useful now when we don't agree on what VCS to
use, because every VCS and many things (like quilt and dpatch) that are
not VCSes but only changeset manipulators can already usefully use wig&pen
as an interchange format.
You can't work usefully work on a Git source package using bzr or quilt
without losing the conceptual clarity of the changesets. You can work on
a wig&pen package in Git, bzr, or quilt while retaining that. That's the
difference, and the reason why it's not obvious to me that jumping
directly to a VCS is a good idea.
The VCS-based source format becomes very compelling once we all use the
same VCS, since it may be possible to us it to represent all of the data
in wig&pen plus real history and other useful information (it's not yet
clear to me that Git can usefully represent changesets via feature
branches, but that's another argument that is already ongoing elsewhere).
However, as long as we don't all use the same VCS, it pessimizes work on
the resulting packages using tools other than the specific VCS that
created the source package, whereas wig&pen gives all tools the same (if
lesser) metadata to work from.
Basically, a Git package is more useful than wig&pen if you're using Git,
and less useful than wig&pen if you're using anything else (except no
tools at all, in which case both are equally useful).
>> Am I able to upload a wig&pen source package right now? My impression
>> was that it was never tried.
>
> Are you able to use the .git source package format right now? Yes, you
> grab Joey's sources, create a package with git, and you're done. You
> can't upload it to the archive, because it's not supported by dpkg in
> stable, but that's it.
>
> By contrast, you can't even get that far with Wig&Pen. And the reason
> is that you've got a new VCS format, without the tools to go with it.
This doesn't mean anything other than no one was interested enough by the
idea to implement it. If Raphael is now, I for one am going to cheer the
effort on, since wig&pen is more useful to me immediately than the Git
package format is.
That doesn't mean that's all I want. I really want to see both. But I
will start using wig&pen *now*, whereas Git is going to take a while.
It's a useful intermediate step for me, and I don't think I'm alone. I'm
still far from convinced that Git actually replaces what quilt is capable
of it; quilt is, so far as I can tell, still a more powerful tool for
doing many of the tasks that I need to do as a Debian packager (as opposed
to an upstream developer, where Git is clearly superior).
wig&pen in the model in which we're currently discussing it is not
something invented specifically by Debian for Debian. It is, for all
intents and purposes, a quilt patch series, which is something invented by
Linux kernel developers for doing almost exactly the same sort of work as
Debian packagers do.
Out of curiosity, have you ever seriously used quilt? Similar to how
people don't "get" DVCSes until they've used them for a while, I'm finding
that people don't "get" what quilt is and how it differs from a VCS until
they've used it in anger to do serious work. It's a changeset
manipulation tool, not a VCS. It's fundamentally different than a VCS,
even a VCS built on changesets like arch (which I have also used). It is
useful in precisely those circumstances where changesets are useful
first-class objects rather than simply components in a VCS repository,
which is a different mindset than using a VCS.
--
Russ Allbery (rra@debian.org) <http://www.eyrie.org/~eagle/>
Reply to: