Re: lintian.debian.org update plan?
Hideki Yamane <henrich@debian.or.jp> writes:
> On Thu, 09 Apr 2009 07:33:59 +0100
> "Adam D. Barratt" <adam@adam-barratt.org.uk> wrote:
>> There's an update run currently in progress. We were hoping it might
>> have finished by now, but it isn't quite there yet. Hopefully it won't
>> be too much longer - it's currently at package 36,842 of 38,093 (source
>> and binary packages) and processing binary packages with names starting
>> "x".
It's now done.
> Great, and thanks for your reply, Adam :-)
> And question, is it so huge process? It means that the machine has
> enough resource? If not, I'll ask some guys to help it (if that is the
> thing I can do).
There are a couple of reasons why it takes so long.
One of them is that Lintian somewhere has either a memory leak or is
accumulating data across all packages that it shouldn't. As a result, the
process grows to about 1GB of resident memory over the course of doing the
whole archive run, which slows it down a lot due to overflowing caches and
probably other bad effects.
We've spent a fair bit of time looking for this memory leak and have
plugged a few contributing causes, but much of it is still there. My
current suspicion is that part of it is the version comparison cache (plus
the slowness from forking dpkg --compare-versions). My intention in a
release relatively soon is to replace that code with something that uses
libapt-pkg-perl to do the version comparisons internally, which will
eliminate the need for the cache.
The other reason is that Lintian is extremely disk-I/O-intensive, and
several new checks have been added recently that make it even more so
(checks that require running strings on ELF binaries and file on all
source package files, for instance). It's hard to do very much about
this, since that disk I/O is part of finding problems. (strings on ELF
binaries is used to find embedded copies of zlib, for instance.)
--
Russ Allbery (rra@debian.org) <http://www.eyrie.org/~eagle/>
Reply to: