Re: Enhancing lintian performance for large source packages
Hi,
On 2024-05-23 19:16, Andrius Merkys wrote:
This week I gave a shot at profiling lintian. I took a large source
package at hand (~400 MB) [1], built it and ran lintian on .changes file
under NYTProf:
$ PERL5OPT=-d:NYTProf bin/lintian -EviI --pedantic ../*.changes
NYTProf highlighted that over 70 % of the runtime was spent in searching
for longest lines in source files as part of
Lintian::Check::Files::SourceMissing and
Lintian::Check::Files::Contents::LineLength checks. I modified them to
use better performing solutions for the same task and managed to
decrease their time share to 32 % for the package in question. This
reduced lintian runtime from 2278s to 820s for the package in question.
[snip]
[1] this is a local package with non-free content which I use
Redid the test with a source package from the main archive section,
promod3 3.4.0+ds-1 (~400 MB):
Wallclock time: 536s (old) vs. 262s (new)
Part spent in long line checks: 60% (old) vs. 25% (new)
Where 'old' is lintian git commit 48d9e6f and 'new' is the one with all
three MRs (!506, !508 and !509) merged in.
Best,
Andrius
Reply to: