Bug#674690: apt: "apt-cache search" is slow because of listing /var/lib/apt/lists 45227 times
Package: apt
Version: 0.9.5.1
Severity: normal
Dear Maintainer,
I've recognized that "apt-cache search" has become quite slow on my system, and when stracing it it shows that it lists the contents of /var/lib/apt/lists a lot (45227 times!):
openat(AT_FDCWD, "/var/lib/apt/lists/", O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_DIRECTORY|O_CLOEXEC) = 3
getdents64(3, /* 156 entries */, 32768) = 13920
getdents64(3, /* 0 entries */, 32768) = 0
close(3) = 0
% grep -c 'openat(AT_FDCWD, "/var/lib/apt/lists/"' apt-cache-search-strace.txt
45227
(this count is reproducible with a second run)
% time apt-cache search tomahawk
apt-cache search tomahawk 3.84s user 4.54s system 99% cpu 8.427 total
I have reported the bug against Ubuntu at:
https://bugs.launchpad.net/ubuntu/+source/apt/+bug/1004947
I have tested it to be still an issue with apt from unstable.
-- Package-specific info:
-- apt-config dump --
APT "";
APT::Architecture "i386";
APT::Build-Essential "";
APT::Build-Essential:: "build-essential";
APT::Install-Recommends "false";
APT::Install-Suggests "0";
APT::Authentication "";
APT::Authentication::TrustCDROM "true";
APT::NeverAutoRemove "";
APT::NeverAutoRemove:: "^firmware-linux.*";
APT::NeverAutoRemove:: "^linux-firmware$";
APT::NeverAutoRemove:: "^linux-image.*";
APT::NeverAutoRemove:: "^kfreebsd-image.*";
APT::NeverAutoRemove:: "^linux-restricted-modules.*";
APT::NeverAutoRemove:: "^linux-ubuntu-modules-.*";
APT::NeverAutoRemove:: "^gnumach$";
APT::NeverAutoRemove:: "^gnumach-image.*";
APT::Never-MarkAuto-Sections "";
APT::Never-MarkAuto-Sections:: "metapackages";
APT::Never-MarkAuto-Sections:: "restricted/metapackages";
APT::Never-MarkAuto-Sections:: "universe/metapackages";
APT::Never-MarkAuto-Sections:: "multiverse/metapackages";
APT::Never-MarkAuto-Sections:: "oldlibs";
APT::Never-MarkAuto-Sections:: "restricted/oldlibs";
APT::Never-MarkAuto-Sections:: "universe/oldlibs";
APT::Never-MarkAuto-Sections:: "multiverse/oldlibs";
APT::Periodic "";
APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Download-Upgradeable-Packages "0";
APT::Periodic::AutocleanInterval "0";
APT::Periodic::Unattended-Upgrade "0";
APT::Update "";
APT::Update::Post-Invoke-Success "";
APT::Update::Post-Invoke-Success:: "touch /var/lib/apt/periodic/update-success-stamp 2>/dev/null || true";
APT::Update::Post-Invoke-Success:: "[ ! -f /var/run/dbus/system_bus_socket ] || /usr/bin/dbus-send --system --dest=org.debian.apt --type=signal /org/debian/apt org.debian.apt.CacheChanged || true";
APT::Archives "";
APT::Archives::MaxAge "30";
APT::Archives::MinAge "2";
APT::Archives::MaxSize "500";
APT::Changelogs "";
APT::Changelogs::Server "http://changelogs.ubuntu.com/changelogs";;
APT::Architectures "";
APT::Architectures:: "i386";
APT::Compressor "";
APT::Compressor::. "";
APT::Compressor::.::Name ".";
APT::Compressor::.::Extension "";
APT::Compressor::.::Binary "";
APT::Compressor::.::Cost "1";
APT::Compressor::gzip "";
APT::Compressor::gzip::Name "gzip";
APT::Compressor::gzip::Extension ".gz";
APT::Compressor::gzip::Binary "gzip";
APT::Compressor::gzip::Cost "2";
APT::Compressor::gzip::CompressArg "";
APT::Compressor::gzip::CompressArg:: "-9n";
APT::Compressor::gzip::UncompressArg "";
APT::Compressor::gzip::UncompressArg:: "-d";
APT::Compressor::bzip2 "";
APT::Compressor::bzip2::Name "bzip2";
APT::Compressor::bzip2::Extension ".bz2";
APT::Compressor::bzip2::Binary "bzip2";
APT::Compressor::bzip2::Cost "3";
APT::Compressor::bzip2::CompressArg "";
APT::Compressor::bzip2::CompressArg:: "-9";
APT::Compressor::bzip2::UncompressArg "";
APT::Compressor::bzip2::UncompressArg:: "-d";
APT::Compressor::xz "";
APT::Compressor::xz::Name "xz";
APT::Compressor::xz::Extension ".xz";
APT::Compressor::xz::Binary "xz";
APT::Compressor::xz::Cost "4";
APT::Compressor::xz::CompressArg "";
APT::Compressor::xz::CompressArg:: "-6";
APT::Compressor::xz::UncompressArg "";
APT::Compressor::xz::UncompressArg:: "-d";
APT::Compressor::lzma "";
APT::Compressor::lzma::Name "lzma";
APT::Compressor::lzma::Extension ".lzma";
APT::Compressor::lzma::Binary "xz";
APT::Compressor::lzma::Cost "5";
APT::Compressor::lzma::CompressArg "";
APT::Compressor::lzma::CompressArg:: "--format=lzma";
APT::Compressor::lzma::CompressArg:: "-9";
APT::Compressor::lzma::UncompressArg "";
APT::Compressor::lzma::UncompressArg:: "--format=lzma";
APT::Compressor::lzma::UncompressArg:: "-d";
APT::Compressor::::Name "";
APT::Compressor::::Extension ".";
APT::Compressor::::Binary "";
APT::Compressor::::Cost "100";
APT::Compressor::::CompressArg "";
APT::Compressor::::CompressArg:: "-9";
APT::Compressor::::UncompressArg "";
APT::Compressor::::UncompressArg:: "-d";
Dir "/";
Dir::State "var/lib/apt/";
Dir::State::lists "lists/";
Dir::State::cdroms "cdroms.list";
Dir::State::mirrors "mirrors/";
Dir::State::extended_states "extended_states";
Dir::State::status "/var/lib/dpkg/status";
Dir::Cache "var/cache/apt/";
Dir::Cache::archives "archives/";
Dir::Cache::srcpkgcache "srcpkgcache.bin";
Dir::Cache::pkgcache "pkgcache.bin";
Dir::Etc "etc/apt/";
Dir::Etc::sourcelist "sources.list";
Dir::Etc::sourceparts "sources.list.d";
Dir::Etc::vendorlist "vendors.list";
Dir::Etc::vendorparts "vendors.list.d";
Dir::Etc::main "apt.conf";
Dir::Etc::netrc "auth.conf";
Dir::Etc::parts "apt.conf.d";
Dir::Etc::preferences "preferences";
Dir::Etc::preferencesparts "preferences.d";
Dir::Etc::trusted "trusted.gpg";
Dir::Etc::trustedparts "trusted.gpg.d";
Dir::Bin "";
Dir::Bin::methods "/usr/lib/apt/methods";
Dir::Bin::solvers "";
Dir::Bin::solvers:: "/usr/lib/apt/solvers";
Dir::Bin::dpkg "/usr/bin/dpkg";
Dir::Bin::bzip2 "/bin/bzip2";
Dir::Bin::xz "/usr/bin/xz";
Dir::Media "";
Dir::Media::MountPath "/media/apt";
Dir::Log "var/log/apt";
Dir::Log::Terminal "term.log";
Dir::Log::History "history.log";
Dir::Ignore-Files-Silently "";
Dir::Ignore-Files-Silently:: "~$";
Dir::Ignore-Files-Silently:: "\.disabled$";
Dir::Ignore-Files-Silently:: "\.bak$";
Dir::Ignore-Files-Silently:: "\.dpkg-[a-z]+$";
Dir::Ignore-Files-Silently:: "\.save$";
Dir::Ignore-Files-Silently:: "\.orig$";
Acquire "";
Acquire::cdrom "";
Acquire::cdrom::mount "/media/cdrom/";
Acquire::http "";
Acquire::http::Proxy "http://localhost:8000";;
Acquire::Languages "";
aptitude "";
aptitude::Keep-Unused-Pattern "^linux-image.*$ | ^linux-restricted-modules.*$ | ^linux-ubuntu-modules.*$";
aptitude::Get-Root-Command "sudo:/usr/bin/sudo";
DPkg "";
DPkg::Pre-Invoke "";
DPkg::Pre-Invoke:: "if [ -x /usr/bin/etckeeper ]; then etckeeper pre-install; fi";
DPkg::Post-Invoke "";
DPkg::Post-Invoke:: "if [ -x /usr/bin/etckeeper ]; then etckeeper post-install; fi";
DPkg::Post-Invoke:: "if [ -d /var/lib/update-notifier ]; then touch /var/lib/update-notifier/dpkg-run-stamp; fi; if [ -e /var/lib/update-notifier/updates-available ]; then echo > /var/lib/update-notifier/updates-available; fi ";
DPkg::Pre-Install-Pkgs "";
DPkg::Pre-Install-Pkgs:: "/usr/bin/apt-listchanges --apt || test $? -ne 10";
DPkg::Pre-Install-Pkgs:: "/usr/sbin/dpkg-preconfigure --apt || true";
DPkg::Tools "";
DPkg::Tools::Options "";
DPkg::Tools::Options::/usr/bin/apt-listchanges "";
DPkg::Tools::Options::/usr/bin/apt-listchanges::Version "2";
RPM "";
RPM::Pre-Invoke "";
RPM::Pre-Invoke:: "if [ -x /usr/bin/etckeeper ]; then etckeeper pre-install; fi";
RPM::Post-Invoke "";
RPM::Post-Invoke:: "if [ -x /usr/bin/etckeeper ]; then etckeeper post-install; fi";
Unattended-Upgrade "";
Unattended-Upgrade::Allowed-Origins "";
Unattended-Upgrade::Allowed-Origins:: "${distro_id}:${distro_codename}-security";
CommandLine "";
CommandLine::AsString "apt-config dump";
-- (no /etc/apt/preferences present) --
-- /etc/apt/sources.list --
# See http://help.ubuntu.com/community/UpgradeNotes for how to upgrade to
# faster mirror
deb http://de.archive.ubuntu.com/ubuntu/ precise-proposed main restricted universe multiverse
deb http://de.archive.ubuntu.com/ubuntu/ precise-updates main restricted universe multiverse
deb http://de.archive.ubuntu.com/ubuntu/ precise main restricted universe multiverse
deb-src http://de.archive.ubuntu.com/ubuntu/ precise main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu/ precise-updates main restricted universe multiverse
deb-src http://archive.ubuntu.com/ubuntu/ precise-updates main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu/ precise-proposed main restricted universe multiverse
deb-src http://archive.ubuntu.com/ubuntu/ precise-proposed main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu/ precise-backports main restricted universe multiverse
deb-src http://archive.ubuntu.com/ubuntu/ precise-backports main restricted universe multiverse
deb http://security.ubuntu.com/ubuntu precise-security main restricted universe multiverse
deb-src http://security.ubuntu.com/ubuntu precise-security main restricted universe multiverse
deb http://archive.canonical.com/ubuntu precise partner
deb-src http://archive.canonical.com/ubuntu oneiric partner
## This software is not part of Ubuntu, but is offered by third-party
## developers who want to ship their latest software.
deb http://extras.ubuntu.com/ubuntu precise main
deb-src http://extras.ubuntu.com/ubuntu precise main
# deb http://download.virtualbox.org/virtualbox/debian natty contrib # disabled on upgrade to precise
deb-src http://ftp.de.debian.org/debian/ stable main contrib non-free
deb-src http://ftp.de.debian.org/debian/ unstable main contrib non-free
deb-src http://ftp.de.debian.org/debian/ testing main contrib non-free
deb http://ftp.de.debian.org/debian/ stable main contrib non-free
deb http://ftp.de.debian.org/debian/ unstable main contrib non-free
deb http://ftp.de.debian.org/debian/ testing main contrib non-free
deb http://ppa.launchpad.net/blueyed/ppa/ubuntu precise main # disabled on upgrade to precise
deb http://ppa.launchpad.net/blueyed/ppa/ubuntu oneiric main
deb-src http://ppa.launchpad.net/blueyed/ppa/ubuntu oneiric main
deb-src http://ppa.launchpad.net/blueyed/ppa/ubuntu natty main
deb-src http://ppa.launchpad.net/blueyed/ppa/ubuntu lucid main
# deb-src http://archive.ubuntu.com/ubuntu/ hardy main restricted universe multiverse
# deb-src http://archive.ubuntu.com/ubuntu/ hardy-updates main restricted universe multiverse
# deb-src http://archive.ubuntu.com/ubuntu/ hardy-proposed main restricted universe multiverse
# deb-src http://security.ubuntu.com/ubuntu hardy-security main restricted universe multiverse
# deb http://archive.ubuntu.com/ubuntu/ hardy main restricted universe multiverse
# deb http://archive.ubuntu.com/ubuntu/ hardy-updates main restricted universe multiverse
# deb http://archive.ubuntu.com/ubuntu/ hardy-proposed main restricted universe multiverse
# deb http://security.ubuntu.com/ubuntu hardy-security main restricted universe multiverse
# deb http://repository.spotify.com stable non-free # disabled on upgrade to precise
-- System Information:
Debian Release: wheezy/sid
APT prefers precise-updates
APT policy: (500, 'precise-updates'), (500, 'precise-security'), (500, 'precise-proposed'), (500, 'precise'), (100, 'precise-backports'), (50, 'unstable'), (50, 'testing'), (50, 'stable')
Architecture: i386 (i686)
Kernel: Linux 3.2.0-24-generic-pae (SMP w/4 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Versions of packages apt depends on:
ii debian-archive-keyring 2010.08.28
ii gnupg 1.4.11-3ubuntu2
ii libapt-pkg4.12 0.9.5.1
ii libc6 2.15-0ubuntu10
ii libgcc1 1:4.6.3-1ubuntu5
ii libstdc++6 4.6.3-1ubuntu5
apt recommends no packages.
Versions of packages apt suggests:
ii apt-doc <none>
ii aptitude 0.6.6-1ubuntu1
ii dpkg-dev 1.16.1.2ubuntu7
ii python-apt 0.8.3ubuntu7
ii synaptic 0.75.9ubuntu1
ii xz-utils 5.1.1alpha+20110809-3
-- no debconf information
Reply to: