dpkg and speed (was Re: Request for package: Perl Cryptix (needs non-us maintainer))

[Craig Sanders <cas@taz.net.au>]

On Sun, May 02, 1999 at 12:30:34PM +0200, Turbo Fredriksson wrote:

> The reason why I split it up so much, was because a person that only
> uses blowfish, don't have the use for the other ones etc.
>
> And if someone makes a program that uses DES encryption only, it only
> have to depend on one small package, not one big chunk...

given the tiny size of the packages, any savings from splitting
it up would probably be used up by the extra records in
/var/lib/dpkg/{status,available}

it also means more records for dpkg to parse...at >3200 packages
(including main, contrib, non-free, and non-us), dpkg takes over a
minute to start up even on a reasonably fast system (e.g. my cyrix-166
with 200MB RAM).


the packages database is just too big these days:

$ ls -l /var/lib/dpkg/{status,available}
-rw-r--r--   1 root     root      2530227 May  2 20:05 /var/lib/dpkg/available
-rw-r--r--   1 root     root      1275022 May  2 20:05 /var/lib/dpkg/status

$ wc /var/lib/dpkg/{status,available}
  40469  165611 1275022 /var/lib/dpkg/status
  68777  309380 2530227 /var/lib/dpkg/available
 109246  474991 3805249 total

$ grep "^Package:" /var/lib/dpkg/status | wc -l
   3793
$ grep "^Package:" /var/lib/dpkg/available | wc -l
   3319

dpkg has to read all of this in and parse it every time it does
anything. it also has to read in the files in /var/lib/dpkg/info to read
in the lists of files and conffiles owned by packages:

$ wc -l /var/lib/dpkg/info/{*.list,*.conffiles} | grep total
 110297 total

$ du -sck /var/lib/dpkg/info/{*.list,*.conffiles} | grep total
4951	total

in total, dpkg has to read in and process nearly 10MB of data just to
start up.

IMO, dpkg needs an overhaul...it should do something like apt does,
using a binary "cache" database for speed, and an authoritative text
database for safety.

craig

--
craig sanders