Re: Let's shrink Packages.xz

[Ian Jackson <ijackson@chiark.greenend.org.uk>]

Russ Allbery writes ("Re: Let's shrink Packages.xz"):
> Ian Jackson <ijackson@chiark.greenend.org.uk> writes:
> > But the problem with lots of small packages is not that the Packages.xz
> > has too many bytes.
> >
> > It's that the packaging tools, UIs (for users and developers), and
> > humans, need to think about too many packages.
> >
> > This makes packaging tools slow, UIs cluttered, and humans confused.
> 
> So we need to figure out how to solve that problem.  But "don't package
> things" is not a good solution to that problem, obviously, and "don't
> package small things" or "don't use a packaging structure that works well
> for our tools" aren't much better.

I think the right answer is that we should try to avoid creating lots
of small packages even if that is a bit more work for the specific
package.

The reason this argument keeps coming up is that the people looking at
a particular package see almost entirely the costs of aggregating into
a single package.  The costs of disaggregating are diffused across the
whole of the user and developer population.

So there is a need to (a) exercise some self-restraint (b) educate
developers who have failed to restrain themselves.

> I think it's important, when looking at a problem like this, to
> distinguish between problems that are fixable via a change in policy
> versus problems that are only deferrable.  This is a problem that's
> deferrable by changing what and how we package, but not fixable.  The
> amount of software in the world is going to continue to grow, and Debian
> is hopefully going to continue to grow with it, which means that the
> package list is going to get longer regardless of our policies around
> packaging small things.  So all those problems are going to happen no
> matter what, which means we should find better solutions to them.

Yes, the overall costs of having lots of packages are going to grow
because there is always going to be more software and more complicated
software.

But improving our tools won't make this problem go away, either.  The
relationship between capability, degradation due to overload, and
effort put into scaling, is complicated, but we cannot expect to ever
make a system that will scale indefinitely.

So we will always need to compromise between having lots of packages
because that's convenient for those packages and having fewer packages
because that's convenient for the rest of the system.

The debate is simply where to put that boundary.  Personally I think
this should be spelled out more clearly in policy.  Having consistency
in approach across the archive would be valuable.

Ian.