Re: Split Packages files based on new section "buildlibs"

[Fabrice BAUZAC-STEHLY <noon@mykolab.com>]

Hello,

(I thought I had sent this mail already, but it looks I haven't)

One of the basic things we want is that users can get the source of
their packages, modify a little, and make a new package for their own
use.  So in particular we want to be able to "apt source" and get all
the sources, in their exact versions that were used when the package was
built (pinned).

A wish we can have in the case of libraries is to separate libraries and
their lifecycles, so that e.g. if I have 10 packages relying on libc
(dynamically), one security upgrade of libc will be immediately taken
into account by the 10 packages that use it.  So in that case it's nice
to separate the library from the binary.  But Rust and Go basically use
static linking, so whatever they produce will embed the libraries.  So
it looks like this wish does not apply in the case of Rust and Go.

So let's say that we do not packages libraries as binary packages: we do
binary packages of only the applications.  We will have one binary
package built from e.g.:

    Application App 1.0
        depends on Library Lib1 1.2
            depends on Library Lib2 1.1
        depends on Library Lib3 1.0

I'm not sure what is the mechanism used in Go or Rust to "pin" the
versions of Lib1, Lib2 and Lib3, but it is necessary that if a user does
"apt source App" the correct versions of the dependencies are checked
out.  And I guess most Application release tarballs do not pin them
themselves, so it would need to be pinned somewhere else, such as inside
the debian/ directory, or in an aggregate upstream tarball that would
contain all the sources.

If we choose to make an aggregate upstream tarball, it would contain all
the sources of all the dependency tree inside one single tarball:

    App-aggregate-1.2.tar.gz:
        Application App 1.0 (contents of the source tarball)
        Library Lib1 1.2
        Library Lib2 1.1
        Library Lib3 1.0

App-aggregate would be versioned wholly instead of versioning the parts.
So if we want to bump e.g. Lib3 to 1.1, we would need to make a new
version of the upstream tarball.

The issue which this aggregate is that there will probably be lots of
duplicated source code.  For example, if some library Lib1 is very
popular and used by lots of applications, we will have its source code
embedded into the source packages of all these applications, and that
would be wasteful in terms of storage of the source packages.  Can we
find a way for source packages to indicate /several/ upstream tarballs,
so that each version of the Lib1 upstream tarball is stored only once?

App_1.0-1.dsc:
  Files:
   App-1.0.orig.tar.gz
   App-1.0-1.debian.tar.xz
   Lib1-1.2.orig.tar.gz
   Lib2-1.1.orig.tar.gz
   Lib3-1.0.orig.tar.gz

--
Fabrice BAUZAC-STEHLY
PGP 015AE9B25DCB0511D200A75DE5674DEA514C891D