RFC: Standardizing source package artifacts build paths

[Guillem Jover <guillem@hadrons.org>]

Hi!

We currently have many built artifacts being dropped directly under
debian/ or under tool specific directories such as debian/.debhelper/.
These have at least the following problems:

  - Make cleaning, an operation that requires executing code from the
    source package itself.
  - Require keeping up updating things like .gitignore or artifacts
    might end up being committed into a VCS.
  - Can stomp over existing files or other tools files, as these are
    generally not namespaced.
  - Clutter the source tree, with debian/ mixing generated and manual
    files.
  - dpkg-dev tools cannot assume the location of the binary package
    to be built, as that depends on the packaging logic and how that
    might drive the various commands.

These files include for example:

  - debian/files
  - debian/*substvars
  - debian/<binary-pkgname>/
  - debian/*debhelper.log
  - debian/.debhelper/
  - debian/tmp/

We'd like to standardize on a new set of artifact build pathnames
for our deb toolchain. These would have the following form:

  - debian/.build/upstream*

    These could be used for out-of-tree builds replacing current
    build-tree/ and similar directories.

  - debian/.build/fsys* (debian/.build/{stage,installed}* ?)

    These would replace the current debian/tmp/ and variants.

  - debian/.build/binary/<pkgname>

    These would be equivalent to the current debian/<pkgname>/.

  - debian/.build/<tool>/<pathname>

    These would be tool specific, for example:

    debian/.build/dpkg/files
    debian/.build/dpkg/files.d/<artifact>
    debian/.build/dpkg/substvars/<pkgname>

    debian/.build/debhelper/<...>

The use of a hidden directory is to reduce clutter and stomping over any
existing directories (there's for example a 3D engine called «build» :).
Most of this could be transparent to most packages, as it would be hidden
behind debhelper logic on a new compat level. There are really few
packages not using debhelper (either directly or indirectly via cdbs).
dpkg-dev tools would preserve backwards compatibility, where applicable,
or could enable this by default only on hypothetical new dpkg-dev compat
levels <https://wiki.debian.org/Teams/Dpkg/Spec/DpkgDevCompatLevel>.

This would give us at least the following properties:

  - We could just (say) .gitignore a single pathname.
  - Cleaning the Debian artifacts would imply doing just
    «rm -rf debian/.build/» (even though this could be abstracted via a
    new dpkg-<something>clean<something> command).
  - Namespacing would avoid any collisions.
  - Would make parallelization way easier and faster.
  - dpkg-dev tools could start defaulting to use package specfific
    directories, instead of requiring the user/tool to specify them.
  - Could make it possible (but not necessary) to move parts of the logic
    from debian/rules (such as calling dpkg-deb) into the build driver
    (see the upcoming proposal), further simplifiying the build process.

Thanks,
Niels and Guillem