Re: quantification of cost of outdated dependencies
Joey Hess <joeyh@debian.org> writes:
> I've been hearing a bit lately about removing dependencies that are no
> longer needed for stable upgrade paths. The most common reason seems
> that this will make apt need less memory[1].
>
> So then, someone must have measured the memory use. Unless this is a
> kind of premature optimation. But, I've not seen a citation of any such
> measurement. I can make a rough estimate: apt-cache stats suggests that
> the average dependency takes 29 bytes.
>
> If that estimate were right, then to save 100kb, which seems an easy
> minimum amount to care about, 3500 dependencies would have to be
> removed. On average, one package in ten would need to be modified.
>
> That's a lot of skull sweat. It's larger than all but the biggest
> transitions. It does not seem, to me, to be worthwhile to save 100 kb of
> ram. Perhaps others would disagree?
>
> But my estimate is *not* right. apt-get may show a RSS of 20 mb
> but 16 of that is mmaped cache files. So, removing a dependency probably
> saves closer to 0 bytes of memory than 20 bytes.
The mmaped chache file IS where the memory is spend. apt doesn't keep
the dependencies in memory and cache. It just mmapes the cache. But even
though that are files you still need the memory to swap in the data.
Also the dependency resolver needs to process the extra depends. I'm
certain I could construct you a case that has exponentially many nearly
solution that the resolve can only exclude at the end of a long chain.
> Where then is the savings? It's not in Packages file download time;
> pdiff optimizes that away. It's not Package file bloat on the mirrors
> or on disk; the compressed text of *all* Dependency lines takes only
> some 500 kb on-disk. The main bloat seems to be 6 mb used in
> pkgcache.bin, and some amount of CPU time spent by apt-get update to
> parse the dependencies and populate the file.
So 6 out of 16MB are depends. That is nearly 40%. How much of those are
obsolete? 1%, 10%, 50%?
> So, I've convinved myself this is probably a false optimisation.
I wouldn't jump to conclusions either way that fast.
Why not write little script that checks the Packages file for obsolete
depends. Simply remove all entries that are not available in squeeze
main. That shouldn't be to hard to script and should be a fair
approximation. Then feed that to apt and compare memory and runtime for
a few things.
MfG
Goswin
Reply to: