Bug#745487: apt python library leak memory and file descriptors
On Tue, Apr 22, 2014 at 11:23:24AM +0200, Petter Reinholdtsen wrote:
>
> Package: python-apt
> Version: 0.8.8.2
> Severity: important
>
> Since I wrote the isenkram package, it have been plagued with a nasty
> resourse leak. It leak memory and file descriptors every time some
> hardware is inserted. The amount of leaked memory varies, but seem to
> be 30-40 MiB every time. The amount of file descriptors depend on the
> number of APT sources listed in /etc/apt/.
> <URL: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=730704 >
> is an example on what happen when the process run out of file
> descriptors, and
> <URL: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=719837 > is the
> initial report about leaking memory.
>
> I've been able to track down the problem to the python-apt library, and
> the following test program demonstrate the leak:
>
> #!/usr/bin/python
> import apt
> while True:
> cache = apt.Cache()
> cache.open(None)
>
> When left alone running for a while, it crashes like this:
>
> % ./apt-leak.py
> Traceback (most recent call last):
> File "./apt-leak.py", line 6, in <module>
> cache.open(None)
> File "/usr/lib/python2.7/dist-packages/apt/cache.py", line 147, in open
> self._records = apt_pkg.PackageRecords(self._cache)
> SystemError: E:Could not open file /var/lib/apt/lists/security.debian.org_dists_wheezy_updates_contrib_binary-amd64_Packages - open (24: Too many open files), E:Could not open file /var/lib/apt/lists/security.debian.org_dists_wheezy_updates_main_binary-amd64_Packages - open (24: Too many open files), E:Could not open file /var/lib/apt/lists/ftp.skolelinux.org_skolelinux_dists_wheezy-test_local_binary-amd64_Packages - open (24: Too many open files), E:Could not open file /var/lib/apt/lists/http.debian.net_debian_dists_wheezy-backports_main_i18n_Translation-en - open (24: Too many open files), E:Could not open file /var/lib/apt/lists/http.debian.net_debian_dists_wheezy-backports_main_binary-amd64_Packages - open (24: Too many open files), E:Could not open file /var/lib/apt/lists/http.debian.net_debian_dists_wheezy_non-free_i18n_Translation-en - open (24: Too many open files), E:Could not open file /var/lib/apt/lists/http.debian.net_debian_dists_wheezy_main_i18n_Translation-en - open (24: Too many open files), E:Could not open file /var/lib/apt/lists/http.debian.net_debian_dists_wheezy_contrib_i18n_Translation-en - open (24: Too many open files), E:Could not open file /var/lib/apt/lists/http.debian.net_debian_dists_wheezy_contrib_binary-amd64_Packages - open (24: Too many open files), E:Could not open file /var/lib/apt/lists/http.debian.net_debian_dists_wheezy_non-free_binary-amd64_Packages - open (24: Too many open files), E:Could not open file /var/lib/apt/lists/http.debian.net_debian_dists_wheezy_main_binary-amd64_Packages - open (24: Too many open files)
> %
>
> Am I using the library wrong (ie should I do something to release the
> resources when I am done with the cache), or is it a bug in the library
> leaking memory and file descriptors?
There are indeed multiple leaks in python-apt, some even cannot be
fixed (IIRC for both parts of the sentences). I'll take a closer
look at this soon, unless someone else does first.
--
Julian Andres Klode - Debian Developer, Ubuntu Member
See http://wiki.debian.org/JulianAndresKlode and http://jak-linux.org/.
Please do not top-post if possible.
Reply to: