Re: Couple of patches
Russ Allbery wrote:
[...]
>
> I don't want to dismiss using threads, since I know that making disk
> access asynchronous can help quite a bit, but I'm very nervous about
> introducing the complexity of thread management into the Lintian code.
> Threads tend to cause a lot of obscure and difficult-to-debug concurrency
> problems.
>
> If we can avoid adding threads and still get reasonable performance, I
> think it will save us headaches later.
At least the collection scripts could be run in parallel. All the collection
scripts with the same Order should run in parallel just fine, and once the
dependencies are done or all the scripts of the same Order are done the
next ones could be run. That ain't complex, it would be just like running
<command> & in a shell.
>
>> * Turning the *desc files into perl code to avoid the extra overhead of
>> parsing them everytime lintian is run. This could be done in a private/
>> script which would be run at some step during the package build process.
>> Only the final .deb package would really benefit from this; although we
>> have to be careful if we use make to know when a .desc file has changed
>> due to git's lack of timestamps preservation.
>
> I suspect that this (and precompiling the data files for Lintian::Data
> objects, which is a similar problem) would save less than 1% of Lintian's
> run time, and hence not be particularly worth it, but I could very well be
> wrong). If I'm wrong, though, it's a good idea. I like the idea of
> precompiling for the final package.
I just finished a PoC, just need to find out why some white spaces won't go
away, because with them the final files are actually a lot larger than the
original files. According to the profiling data, read and parse
_dpkg_control were consuming a bit over 2% of the total time required to
check oo.o-core. With my PoC read_dpkg_control only consumes 1%
>
> There are some more obvious things that we can do first, such as
> eliminating Tag and its second read of all the tag metadata by merging it
> into the new Lintian::Tag::Info module. I'm slowly working on that.
>
Sure. I already tried to remove the add_tag and display_tags from
frontend/lintian; but I ended up with something useless.
Cheers,
--
Raphael Geissert - Debian Maintainer
www.debian.org - get.debian.net
Reply to: