Bug#764204: apt-cache calls fcntl() on 65536 FDs
Control: tag -1 pending
2015-05-25 16:22 GMT+02:00 Sebastian Boehm <sebastian@sometimesfood.org>:
> Hi,
>
> On 25 May 2015 at 15:32, Julian Andres Klode <jak@debian.org> wrote:
>> I don't really see the performance issue here, though. I don't know
>> why the submitter thinks that there is a performance issue here. The
>> closing takes a few milliseconds. An apt-cache show runs in 0.040s
>> here, and has never taken longer for me.
>
> on my fastest machine, a single apt-cache show takes 0.06s on Jessie.
> However, the same apt-cache show takes less than 0.006s on Wheezy.
>
> For a single run on a fast machine, this isn't too bad, but on a slow
> machine this really hurts a lot. When running docker on my laptop, a
> single apt-cache show takes about 2.7 seconds on jessie, whereas it
> took only 1.4 seconds on wheezy.
>
> Jessie:
>
> root@b49835d0cc64:~# time apt-cache show mg > /dev/null
> real 0m2.708s
> user 0m1.830s
> sys 0m0.850s
>
> Wheezy:
>
> root@df40952198bb:~# time apt-cache show mg > /dev/null
> real 0m1.441s
> user 0m1.260s
> sys 0m0.190s
Those seem to be uncached or on compressed indices, and thus not
relevant for this bug.
>
> Even the 0.06s are quite noticeable when using tools such as Chef,
> which produce one apt-cache call per package check. On my fastest
> machine, checking the status for ~70 packages used to take
> approximately 400ms on wheezy, but takes more than four seconds with
> jessie. Obviously the situation is much worse with docker and on
> slower systems.
>
>> So I'd prefer to just close this, as it's not worth the extra effort
>> to read out /proc/self/fd.
>
> How about using the O_CLOEXEC flag when opening files?
We do, AFAIK.
I'm wondering a bit because show only does about 38 fcntl() calls for
me, it does not seem to ForkExec() here at all (and I don't know why
it should do that at all).
I just tested the closing call, and it took about 4 ms (or 10-12 in
jessie) for the program to start up, call fcntl() on the 65533 file
descriptors, and exit. I cannot believe that this is the major
performance difference you are experiencing between wheezy and jessie.
Anyway, I fixed this in
http://anonscm.debian.org/cgit/apt/apt.git/commit/?id=870a2b6d683e58c0584bbd3614a76cf25a055928
to use /proc/self/fd where available. This brings us down to 0.72 ms
in my optimal test case.
--
Julian Andres Klode - Debian Developer, Ubuntu Member
See http://wiki.debian.org/JulianAndresKlode and http://jak-linux.org/.
Reply to: