[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Bug#896790: marked as done (apt on jessie (pre-1.1) closes millions of files on ExecFork (makes Docker slow))

Your message dated Wed, 11 Jul 2018 12:24:54 +0200
with message-id <20180711122226.GA28985@debian.org>
and subject line Re: Bug#896790: apt on jessie (pre-1.1) closes millions of files on ExecFork (makes Docker  slow)
has caused the Debian Bug report #896790,
regarding apt on jessie (pre-1.1) closes millions of files on ExecFork (makes Docker  slow)
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact owner@bugs.debian.org
immediately.)


-- 
896790: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=896790
Debian Bug Tracking System
Contact owner@bugs.debian.org with problems
--- Begin Message --- 
Package: apt
Version: 1.0.9.8.4
Severity: important

Dear Maintainer,

I'm sorry for flagging this as 'important', but in the Docker eco-system
this bug is quite troublesome for everyone doing apt on Jessie.

The issue which is reported (and fixed) here --
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=764204 --
still applies to Jessie:

  Apt ExecFork() tries to close all possible FDs instead of just the
  open ones.

  **In docker builds, this generally defaults to 1 million(!) files**

  So, you'd be looking at this (from 3 to 1024*1024), instead of doing
  apt updates/upgrades.
[pid 17831] getrlimit(RLIMIT_NOFILE, {rlim_cur=1024*1024, rlim_max=1024*1024}) = 0 [pid 17831] fcntl(148924, F_SETFD, FD_CLOEXEC) = -1 EBADF (Bad file descriptor) 

  This caused a simple apt-get update/upgrade run to go from around 15
  secs on Wheezy to a whopping 200 seconds on Jessie.


Apt versions:

  Wheezy runs: apt 0.9.7.9+deb7u7 (fast, closes max. 40 FDs)
  Jessie runs: apt 1.0.9.8.4 (slow, closes max. nofile FDs)
  Stretch runs: apt 1.4.8 (fast, closes exactly open-FD-count)

Fix history:

  Old (broken after 0.9.13), November 2013:
https://github.com/Debian/apt/commit/61f954bff040809e7ab57b3adec2fe95339ffb94#diff-44ca32a60c136bb8155104e94ca99e0a 

  Better (fixed in 1.0.9.10+), April 2015:
https://github.com/Debian/apt/commit/15901516326737a67f2a9af26cd7e434162de019 

  Best (fixed in 1.1+), May 2015:
https://github.com/Debian/apt/commit/be4d908fb5d56f8a331bb88e878a6fb8d82a77a6 


Would you be willing to patch the Jessie Apt versions with the two
latter commits? I could file a patch if you want.


(For the record: workarounds for Docker include setting the nofile limit
in daemon.json {"default-ulimits": {"nofile":"512:1024"}} or passing
--ulimit nofile=512 to 'docker build' (not available for docker-compose).) 


Cheers,
Walter Doekes
OSSO B.V.



-- Package-specific info:

-- (no /etc/apt/preferences present) --


-- (/etc/apt/sources.list present, but not submitted) --


-- System Information:
Debian Release: 8.9
  APT prefers oldstable
  APT policy: (500, 'oldstable')
Architecture: amd64 (x86_64)

Kernel: Linux 4.4.0-116-generic (SMP w/16 CPU cores)
Locale: LANG=C.UTF-8, LC_CTYPE=C.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages apt depends on:
ii  debian-archive-keyring  2017.5~deb8u1
ii  gnupg                   1.4.18-7+deb8u3
ii  libapt-pkg4.12          1.0.9.8.4
ii  libc6                   2.19-18+deb8u10
ii  libgcc1                 1:4.9.2-10
ii  libstdc++6              4.9.2-10

apt recommends no packages.

Versions of packages apt suggests:
pn  apt-doc                      <none>
pn  aptitude | synaptic | wajig  <none>
ii  dpkg-dev                     1.17.27
ii  python-apt                   0.9.3.12

-- no debconf information

--- End Message ---
--- Begin Message --- 
On Tue, Apr 24, 2018 at 11:54:07AM +0200, wjdoekes@osso.nl wrote:
> Package: apt
> Version: 1.0.9.8.4
> Severity: important
> 
> Dear Maintainer,
> 
> I'm sorry for flagging this as 'important', but in the Docker eco-system
> this bug is quite troublesome for everyone doing apt on Jessie.
> 
> The issue which is reported (and fixed) here --
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=764204 --
> still applies to Jessie:

Which means reporting another bug for it is wrong.

> 
>   Apt ExecFork() tries to close all possible FDs instead of just the
>   open ones.
> 
>   **In docker builds, this generally defaults to 1 million(!) files**
> 
>   So, you'd be looking at this (from 3 to 1024*1024), instead of doing
>   apt updates/upgrades.
> 
>     [pid 17831] getrlimit(RLIMIT_NOFILE, {rlim_cur=1024*1024,
> rlim_max=1024*1024}) = 0
>     [pid 17831] fcntl(148924, F_SETFD, FD_CLOEXEC) = -1 EBADF (Bad file
> descriptor)
> 
>   This caused a simple apt-get update/upgrade run to go from around 15
>   secs on Wheezy to a whopping 200 seconds on Jessie.
> 
> 
> Apt versions:
> 
>   Wheezy runs: apt 0.9.7.9+deb7u7 (fast, closes max. 40 FDs)
>   Jessie runs: apt 1.0.9.8.4 (slow, closes max. nofile FDs)
>   Stretch runs: apt 1.4.8 (fast, closes exactly open-FD-count)
> 
> Fix history:
> 
>   Old (broken after 0.9.13), November 2013:
> https://github.com/Debian/apt/commit/61f954bff040809e7ab57b3adec2fe95339ffb94#diff-44ca32a60c136bb8155104e94ca99e0a
> 
>   Better (fixed in 1.0.9.10+), April 2015:
> https://github.com/Debian/apt/commit/15901516326737a67f2a9af26cd7e434162de019
> 
>   Best (fixed in 1.1+), May 2015:
> https://github.com/Debian/apt/commit/be4d908fb5d56f8a331bb88e878a6fb8d82a77a6
> 
> 
> Would you be willing to patch the Jessie Apt versions with the two
> latter commits? I could file a patch if you want.

No. The problem is well known and people don't have to start running
jessie in docker now. There will be no more fixes for jessie except for
release critical bugs.

-- 
debian developer - deb.li/jak | jak-linux.org - free software dev
ubuntu core developer                              i speak de, en

--- End Message ---
Reply to: