Re: RFC: DEP-14: Recommended layout for Git packaging repositories
Hi Raphael,
On Wed, Nov 12, 2014 at 10:15:27AM +0100, Raphael Hertzog wrote:
> Hi Scott,
>
> using your mail as an opportunity to explicity notify the respective
> package maintainers of this ongoing DEP.
>
> Guido, Bernhard, Ron, if you are not reading debian-devel, I would
> like to bring your attention to a discussion that I recently started
> about trying to standardize the layout of Git packaging repositories:
> https://lists.debian.org/debian-devel/2014/11/msg00444.html
> https://lists.debian.org/debian-devel/2014/11/threads.html#00444
>
> This mail is part of this discussion.
Thanks for the heads up on this, I wasn't aware of this before now.
Please keep me in the CC for anything people would like me to see
and/or reply to.
> On Tue, 11 Nov 2014, Scott Kitterman wrote:
> > Who's going to do patches to existing tools (e.g. git-dpm is the one I use and
> > care about) so they comply with this and similarly scripts to convert existing
> > git repos to match this recommendation?
> >
> > It doesn't make much sense to have an standard unless there's also a plan to
> > implement using it.
>
> My own idea was to involve the respective package maintainers in this
> discussion (now done with this mail) and once we have an agreed-upon DEP
> then to suggest a sprint (IRL or not) to update the respective tools.
So I've got a bunch of reading to do to fully catch up on everything
you've proposed here so far, but after a quick first pass, here's a
few first impression comments:
I think you probably need to be careful of overspecifying this. One
of the beautiful things about git is that if you understand how the
DAG works, it very easily lends itself to creating different ways of
working that are particularly suited to the needs of a particular
project. So while there are some rough guidelines as to some best
practices for certain situations, it invariably works best when the
workflow is shaped to suit the project and its developers, not when
the project and the developers are rigidly jammed into a framework
they need to contort themselves into.
Likewise, in the same "When in Rome" vein as submitting patches
upstream, exactly how a repository that is branched off the upstream
one will look, depends at least in part on how upstream has chosen to
work, and we definitely can't start telling them there are now rules
for how they need to manage their repo.
The design of gitpkg was based very much on this simple reality.
Rather than duplicating how cvs-buildpackage worked, or how svn-bp
evolved from it, gitpkg has a very simple rule:
- It doesn't care what your repository looks like or how you
choose to work in it, or what you name your tags and branches.
It's designed to be able to work with essentially any repository
that you find anywhere, without requiring you to first mash it into
some particular form, or put tool specific files into it.
Provided you can give it a treeish that contains the debian patched
source you want to export, and (optionally) a treeish that contains
the "orig" source, it can always make you a source package of that.
The "normal workflow" is simply: work in git exactly how you normally
would, whether you're making upstream changes or debian patches.
Export a source package with `gitpkg $debtag $upstreamtag`.
That in a nutshell is it. All of it. There aren't even any command
line options for you to forget to type, or to screw up in a way that
would make a different source package to the last time you called it.
There are hooks you can use if you want p-t tarballs, or quilt patches,
or to automatically build the package once it's exported (with cowpoke
or dpkg-bp etc.), but they are all optional extras that each local user
running gitpkg can choose for themselves, if they want them.
The ability to export a valid source package does not depend on knowing
some secret incantation or configuration the repo was made to depend on.
All you need to know is the treeish you want to export.
This means you can use it to export any version of a source package
from a repo that existed long before you'd ever heard of gitpkg.
Without needing to go back and rewrite or patch its history. You can
use it to export packages from a repo where someone used some other
tool previously. You can use it to branch your packaging off any
upstream repo no matter how it has been used or arranged. You can
use it to export packages that have been imported to git from
snapshot.d.o to build a full history from the pre-git epoch (with
git-debimport or any other tool you choose to do that with).
The good news from all that though, is that it would seem unlikely
that gitpkg itself would need any changes to be able to cope with
any repo design you could come up with that wasn't completely insane :)
The bad news perhaps, is I'm less convinced about how keen people
already using gitpkg will be to arbitrarily restructure their entire
existing repos to follow this. At least not without some much clearer
rationale about what compelling benefits they'd get in return for all
that work and disruption and rewriting of history.
Maybe I missed something somewhere (I should have been sound asleep
a few hours ago, so that's quite possible), but I see lots of
explanation of *what* you want to do, but not the killer reasoning
about *why* you want to do it, or what concrete things you think
will be gained from it.
More specifically, you have things like:
"Helper tools should usually rely on the output of dpkg-vendor
--query vendor to find out the name of the current vendor.
The retrieved string must be converted to lower case. This
allows users to override the current vendor by setting
DEB_VENDOR in their environment"
This seems like a git-bp specific hack to me, that has the assumption
and trained habits of using that tool so deeply embedded in it that
doesn't bother to explain why you would use this, when you would use
this, or even what you would expect it to do.
That seems completely inapplicable to a tool like gitpkg that lets
you export *any* version of a package, from *any* repo, at any time,
from any state of your working dir. gitpkg doesn't need to care what
your local vendor is, and shouldn't need to. You can export a Debian
package, and packages for any other vendor, all from the same tree,
automatically shipped off to suitable buildds in quick succession.
They can even all come from exactly the same tags if no special
patches are required for the other vendor distros.
Similarly the idea of tagging over the top of upstream tags just because
upstream didn't name them how you'd like seems a bit ... overkill.
What tool breaks because of such strict assumptions about naming?
The section on "Patch queue tags" also seems to be written from the
perspective of only knowing one tool and one workflow to use it.
With the work that David Bremner did on git-debcherry, gitpkg can
fully automatically extract a quilt patch series for a source package
simply from completely normal commits to your repo, in a form that
makes them trivial for upstream to cherry pick, and makes them
automatically vanish when you merge the upstream release that includes
them.
This again fits with the basic gitpkg philosophy of "just work in git
like you normally would". You make a patch, commit it to your tree,
send a pull request to upstream, and export a package that has it in
a quilt series (if you use format 3 and want that). If upstream
accepts it, it will be gone from the next package you export, with
no manual busywork needed to manage all that.
Anyway, there's probably more, and I don't want to sound too negative
about this. If you can take a step back from where you are now and
explain in some detail the rationale for what you really want to
achieve here (as opposed to the mechanisms you think might do that),
I'll certainly be happy to explore how that fits into the world
outside of git-bp as I've come to know it. I get the feeling at the
moment that you're mostly trying to formalise how *you* work with
git-bp. Which is as good a place as any to start, but the other
tools exist and evolved into their current state by looking at this
problem in different ways to what {cvs,svn,git}-buildpackage did,
and have tried to harness the full power that git gives to developers
rather than try to squeeze it into the same old box as the old tools.
> I would be willing to participate to such a sprint and do my share
> of the work (although I'm primarily using git-buildpackage and would
> likely start with this helper tool).
I am a bit curious about what patches you think you're going to need
to make to make things comply with this plan?
My first instinct would be to think that these are clues to bugs
and/or design flaws in the tools and/or this specification which
might reveal some interesting things if squinted at hard enough.
Sorry that all got longer than I planned it to for a few "quick
first thoughts" :) I'll be curious to see how Bernhard sees this
fit with his design for git-dpm too, which has a somewhat different
philosophy again for exploiting the real strengths of git, based on
his experience and the necessities of working with his upstreams.
Cheers,
Ron
Reply to: