Re: dpkg --smallmem has a larger footprint than --largemem

[Ian Jackson <ian@davenant.greenend.org.uk>]

Ben Collins writes ("Re: dpkg --smallmem has a larger footprint than --largemem"):
> Simply calling stat() on all those files is a real performance jerk.

This is because the .../info directory is so large - because it
contains at least one file (and usually more) per package.  This is
fine for the maintainer scripts, only a few of which are touched for
each thing dpkg does.  But it's not so good for the .list files.  It
used to be OK when we only had a few '00 packages :-).

So I think this problem will go away if we amalgamate several packages
file listings' into one file, provided we put the new amalgamated
*.list files in a different directory to all the maintainer scripts.

Transition to this new on-disk format, in a way where the whole thing
doesn't go wrong if dpkg somehow dies halfway through, will be fun.

Also, of course, the new format will break all those programs that
parse the *.list files directly like they're not supposed to (though I
suppose we *could* keep them up to date too).

Ian.