How to make Packages file 50% smaller

[Brian May <bam@debian.org>]

Hello,

Does this file really *need* to contain long verbose descriptions of
packages?

Just by removing the long description (the short description remains),
I can cut the compressed file size by over 50%:

[1021] [snoopy:bam] ~/admin/linux >cat /var/lib/apt/lists/ftp.melb.apana.org.au_mirror_debian_dists_potato_main_binary-i386_Packages | gzip | wc 
   3309   19009  828014
[1022] [snoopy:bam] ~/admin/linux >cat /var/lib/apt/lists/ftp.melb.apana.org.au_mirror_debian_dists_potato_main_binary-i386_Packages | grep -v "^ " | gzip | wc 
   1424    8364  384015

which would have significant benefits for people like my who don't
have high speed ISDN connections and/or have to pay for bandwidth
used.

This is becoming more and more important, especially now that apt-get
can use all stable, testing, and unstable Packages in a useful manner.

I can't think of any useful role this long description serves to
apt-get, other then displaying of package information for users (with
apt-cache), in which case, you could use
<URL:http://packages.debian.org/$package instead>.

However, I imagine this might upset some people, so why not split the
description into a separate file, so you only have to download it if
you really want it? (perhaps even only for one distribution instead of
all of them).


Other proposals in the past to do similar things have failed, or
require more time for development before they are usable:

- rsync: uses too much load on the server.

- rproxy: requires Packages uncompressed, and these may be removed
from the servers in the future, see #81657. Furthermore I have had
serious problems with current Debian versions of rproxy, see #83603.

- rsync friendly gzip: (shows potential; still waiting).


I think this proposal has the benefit that it is simple, easy to
understand, and can be achieved without *now* without major changes,
except perhaps some way to use the separated description file.


Comments?
-- 
Brian May <bam@debian.org>