Re: Survey: git packaging practices / repository format

["Enrico Weigelt, metux IT consult" <lkml@metux.net>]

On 29.05.19 13:50, Ian Jackson wrote:

Hi,

> Oh.  I think I have misunderstood.  I think you are describing a git
> workflow you use as a *downstream* of Debian, not as a maintainer
> *within* Debian.

Yes, something like that. I'm maintaining additional repos for certain
projects, usually backports or packages that aren't in Debian at all.
Pretty often it's pretty much rebasing Debianziation onto latest
upstream releases.

A very annoying aspect for me is the way upstream souces and patches
are managed, eg.

* I need to reproduce how exactly orig trees are constructed from the
  actual upstream (git) trees. We often have autogenerated stuff in
  here (eg. autotools stuff), often files are missing, stuff moved
  around, etc, etc.
  --> here we at least should have some fully automatic transformation
      system within git. probably not within the actual source packages
      themselves, possibly as a cross-package project.
* text-based patches are costly to reproduce into a git repository
  * many just don't apply via git-am --> at least we should fix them
    and add a policy that all patches need to be git-am compatible
    (no, quilt isn't so much helpful here, and I find it pretty complex
    to use - compared with git - I need rebase)
  * we don't have any clear (machine-readable) distinction between types
    of patches, eg. whether they're generic or really debian specific
  * somethimes we even have patches against autogenerated stuff
  * many patches lack any clear description
* sometimes we even have weird transformations on the source tree
  (usually on the orig tree, but sometimes also within rules file)

Few days ago, I tried to rebuild recent rustc (which I need for tbird),
but got a lot of strange fails. It also lacked the source code for some
library where that specific version even doesn't exist in the upstream
git repo anymore. I know that rust is an ugly beast, but those things
just should not happen. The rust toolchain seems to be a good candidate
for creating an fully automatic git transformation (eg. transform
submodules into merges, etc).

I'd like to propose some additions to the packaging policies:

* there shall be no source tree transformations in the build process,
  all necessary changes shall be done by patches
* the upstream build process shall be used as-is whenever possible,
  if necessary patch it. (eg. no manual build or install rules, etc)
* there shall be no conditional applying of patches - the queue shall
  be always applied a as a whole. if certain code changes are only
  applicable for certain build conditions (eg. different flavours like
  with the kernel), proper build-time config flags shall be introduced.
* all patches shall be git-am compatible and have a clear description
  of what they're actually doing
* patches shall be written in an generic/upstreamable way if possible
  (eg. introduce new build-time config flags if necessary)
* patches shall be grouped into generic/upstreamable and distro specific
  ones, the differenciation shall be easily machine-readable (eg.
  message headers), and the generic ones shall be first in the queue.
* no patching of autogenerated files
* autogenerated files shall be removed from source and always
  regenerated within the build process
* the debian/ directory shall contain a machine readable file for
  telling exactly the upstream source tree (eg. canonical version, url
  to tarball and gitrepo, tag name + commit id, etc)
* minimum required debhelper version shall be the one present in stable,
  (or better oldstable) - unless threre's really no other sane way

> And I think what you are saying is that you don't use source packages
> (.dsc) except maybe in the innards somewhere of your machinery.
> I think that is a good way for a downstream to work.  Certainly when I
> modify anything locally I don't bothere with that .dsc stuff.

Right, I've always found that very complicated and hard to maintain.
Actually, I'd like to propose phasing it out that old relic. With git
we just don't need that anymore.

> But my aim in this thread was to capture how people work *within*
> Debian, where a maintainer is still required to produce a .dsc.

I don't think that .dsc really makes the picture so different. It always
can be generated automatically. IMHO, it's only needed as an output
format for creating src repos and as an intermediate transport for
traditional tools like buildd.


--mtx

-- 
Enrico Weigelt, metux IT consult
Free software and Linux embedded engineering
info@metux.net -- +49-151-27565287