Re: Bits from the Testing Security team
On Wed, Oct 17, 2007 at 03:00:55PM +0100, Ian Jackson wrote:
> > > Indeed on modern multicore systems running the decompression in a
> > > separate process allows it to be run on a separate CPU, in parallel to
> > > the other processing done by dpkg proper. So it might be faster.
> > >
> > > (I haven't done any benchmarking but I think these conclusions are
> > > pretty obvious.)
> >
> > What I think running it in a separate process does it creating 2 pipes
> > you don't need. You can just do everything in the local memory of 1
> > process.
> >
> > Anyway, like you said, someone needs to do benchmarks to see if either
> > solution is faster than the other. I'm not convinced either way.
>
> The overhead (cpu cost) of reading or writing the contents of the .deb
> a few times through some pipes is irrelevant compared to both the cpu
> cost of decompressing it and of course the cpu cost of processing each
> file during installation.
Ian,
You were saying that by having 2 processes (or threads) you might gain
something. I'm saying that by doing it with 2 processes you also
create a new overhead which might or might not be bigger than what you
gain. In either case, I think it's unlikely that you'll notice such a
change.
If you want to take advantages of multiple core, it's properly more
likely that you can gain something with spreading the decompression
itself over multiple cores. You could for instance have several dpkg
subprocesses extracting a different .deb at the same time. But I'm
not conviced this is going to gain you anything either.
> > On Tue, Oct 16, 2007 at 02:34:36PM +0100, Ian Jackson wrote:
> > > Having dpkg invoke the decompresser as a separate program provides
> > > several IMO important advantages. It simplifies the code in dpkg, and
> > > it greatly simplifies the management of cross-version compatibility
> > > and multiple optional decompression schemes (including experimental
> > > ones), by decoupling dpkg from the decompressers.
> >
> > I see no reason why we can't have both available.
>
> dpkg has to be compiled one way in the distribution.
>
> And if there is no good reason to have the decompressors bound in then
> having that facility wired into the code is just extra complexity to
> no useful purpose.
If there is an advantage to use a library instead of doing it with an
external program, it makes sense to do it. But not everything you want
to use might have a library which you can use, so it would make sense
to atleast have the option to decompress with an external program.
Kurt
Reply to: