Bug#766206: apt: Stale partial -> confusing bogus 404 (manual intervention needed, not obvious)
Package: apt
Version: 0.9.7.9+deb7u6
Severity: important
Dear Maintainer,
"apt-get update" with a stale partial Packages file displays a cryptic error
404 leading me to assume that the Packages file doesn't exist and the server
is broken. This is made worse by it giving the URL of the Packages file (not
including .bz2), which of course does give a 404. So the obvious solution is
to download the .bz2'ed version, unpack it and put it in lists - but this
bypasses some important security checks. The real solution is to delete the
old cached packages files, thus allowing the .bz2 fetch to succeed.
This is extremely confusing, even to a user with some technical knowledge!
Most likely the problem is apt-get is misunderstanding an error message from
the web server when doing a partial fetch; this ought to be fixable. If not,
it should delete the partial files itself, and if that isn't possible then at
least improve the error message! Don't return a URL (.../Packages) that is
guaranteed to fail, and thus probably confuse the user into doing something
foolish!
This is a "usable security" issue IMHO, in that the obvious solution to the
cryptic error messages is dangerous, and the bug usually only arises when the
user needs to do an update urgently to fix a security problem (presumably
known and exploitable). Even with support from the official IRC channel it
took quite some time to resolve this; it's not good enough to dump it on the
user, and it wasted some security@debian time too with me panicking about
security.debian.org apparently being broken.
Output:
==
dayna:~# apt-get update
Hit http://www.deb-multimedia.org wheezy Release.gpg
Hit http://www.deb-multimedia.org wheezy Release
Hit http://www.deb-multimedia.org wheezy/main Sources
Hit http://www.deb-multimedia.org wheezy/main i386 Packages
Hit http://security.debian.org wheezy/updates Release.gpg
Hit http://security.debian.org wheezy/updates Release
Hit ftp://ftp.fr.debian.org wheezy Release.gpg
Hit ftp://ftp.fr.debian.org wheezy Release
Hit http://security.debian.org wheezy/updates/contrib i386 Packages
Hit http://security.debian.org wheezy/updates/non-free i386 Packages
Ign http://www.deb-multimedia.org wheezy/main Translation-en_GB
Ign http://www.deb-multimedia.org wheezy/main Translation-en
Hit http://security.debian.org wheezy/updates/contrib Translation-en
Hit ftp://ftp.fr.debian.org wheezy/main Sources
Hit http://security.debian.org wheezy/updates/non-free Translation-en
Hit ftp://ftp.fr.debian.org wheezy/non-free Sources
Hit http://mirrors.kernel.org wheezy-updates Release.gpg
Ign http://security.debian.org wheezy/updates/main Translation-en
Hit ftp://ftp.fr.debian.org wheezy/contrib Sources
Err http://security.debian.org wheezy/updates/main i386 Packages
404 Not Found [IP: 212.211.132.250 80]
Hit ftp://ftp.uk.debian.org wheezy Release.gpg
Hit ftp://ftp.fr.debian.org wheezy/main i386 Packages
Hit ftp://ftp.uk.debian.org wheezy Release
Hit ftp://ftp.fr.debian.org wheezy/non-free i386 Packages
Hit ftp://ftp.uk.debian.org wheezy/main Sources
Hit ftp://ftp.uk.debian.org wheezy/non-free Sources
Hit ftp://ftp.fr.debian.org wheezy/contrib i386 Packages
Hit ftp://ftp.uk.debian.org wheezy/contrib Sources
Hit ftp://ftp.fr.debian.org wheezy/contrib Translation-en
Hit ftp://ftp.fr.debian.org wheezy/main Translation-en
Hit http://mirrors.kernel.org wheezy-updates Release
Hit ftp://ftp.fr.debian.org wheezy/non-free Translation-en
Hit ftp://ftp.uk.debian.org wheezy/main i386 Packages
Hit http://mirrors.kernel.org wheezy-updates/main Sources
Hit http://mirrors.kernel.org wheezy-updates/contrib Sources
Hit ftp://ftp.uk.debian.org wheezy/non-free i386 Packages
Hit ftp://ftp.uk.debian.org wheezy/contrib i386 Packages
Hit ftp://ftp.uk.debian.org wheezy/contrib Translation-en
Hit ftp://ftp.uk.debian.org wheezy/main Translation-en
Hit http://mirrors.kernel.org wheezy-updates/non-free Sources
Hit ftp://ftp.uk.debian.org wheezy/non-free Translation-en
Hit http://mirrors.kernel.org wheezy-updates/main i386 Packages/DiffIndex
Hit http://mirrors.kernel.org wheezy-updates/contrib i386 Packages
Hit http://mirrors.kernel.org wheezy-updates/non-free i386 Packages
Hit http://mirrors.kernel.org wheezy-updates/contrib Translation-en
Hit http://mirrors.kernel.org wheezy-updates/main Translation-en/DiffIndex
Hit http://mirrors.kernel.org wheezy-updates/non-free Translation-en
W: Failed to fetch http://security.debian.org/dists/wheezy/updates/main/binary-i386/Packages 404 Not Found [IP: 212.211.132.250 80]
E: Some index files failed to download. They have been ignored, or old ones used instead.
dayna:~#
==
IRC on 20 and 21 October 2014 on #debian on debian IRC server.
-- Package-specific info:
-- apt-config dump --
APT "";
APT::Architecture "i386";
APT::Build-Essential "";
APT::Build-Essential:: "build-essential";
APT::Install-Recommends "1";
APT::Install-Suggests "0";
APT::Authentication "";
APT::Authentication::TrustCDROM "true";
APT::NeverAutoRemove "";
APT::NeverAutoRemove:: "^firmware-linux.*";
APT::NeverAutoRemove:: "^linux-firmware$";
APT::NeverAutoRemove:: "^linux-image.*";
APT::NeverAutoRemove:: "^kfreebsd-image.*";
APT::NeverAutoRemove:: "^linux-restricted-modules.*";
APT::NeverAutoRemove:: "^linux-ubuntu-modules-.*";
APT::NeverAutoRemove:: "^gnumach$";
APT::NeverAutoRemove:: "^gnumach-image.*";
APT::Never-MarkAuto-Sections "";
APT::Never-MarkAuto-Sections:: "metapackages";
APT::Never-MarkAuto-Sections:: "restricted/metapackages";
APT::Never-MarkAuto-Sections:: "universe/metapackages";
APT::Never-MarkAuto-Sections:: "multiverse/metapackages";
APT::Never-MarkAuto-Sections:: "oldlibs";
APT::Never-MarkAuto-Sections:: "restricted/oldlibs";
APT::Never-MarkAuto-Sections:: "universe/oldlibs";
APT::Never-MarkAuto-Sections:: "multiverse/oldlibs";
APT::Update "";
APT::Update::Post-Invoke-Success "";
APT::Update::Post-Invoke-Success:: "[ ! -f /var/run/dbus/system_bus_socket ] || /usr/bin/dbus-send --system --dest=org.debian.apt --type=signal /org/debian/apt org.debian.apt.CacheChanged || true";
APT::Architectures "";
APT::Architectures:: "i386";
APT::Compressor "";
APT::Compressor::. "";
APT::Compressor::.::Name ".";
APT::Compressor::.::Extension "";
APT::Compressor::.::Binary "";
APT::Compressor::.::Cost "1";
APT::Compressor::gzip "";
APT::Compressor::gzip::Name "gzip";
APT::Compressor::gzip::Extension ".gz";
APT::Compressor::gzip::Binary "gzip";
APT::Compressor::gzip::Cost "2";
APT::Compressor::gzip::CompressArg "";
APT::Compressor::gzip::CompressArg:: "-9n";
APT::Compressor::gzip::UncompressArg "";
APT::Compressor::gzip::UncompressArg:: "-d";
APT::Compressor::bzip2 "";
APT::Compressor::bzip2::Name "bzip2";
APT::Compressor::bzip2::Extension ".bz2";
APT::Compressor::bzip2::Binary "bzip2";
APT::Compressor::bzip2::Cost "3";
APT::Compressor::bzip2::CompressArg "";
APT::Compressor::bzip2::CompressArg:: "-9";
APT::Compressor::bzip2::UncompressArg "";
APT::Compressor::bzip2::UncompressArg:: "-d";
APT::Compressor::xz "";
APT::Compressor::xz::Name "xz";
APT::Compressor::xz::Extension ".xz";
APT::Compressor::xz::Binary "xz";
APT::Compressor::xz::Cost "4";
APT::Compressor::xz::CompressArg "";
APT::Compressor::xz::CompressArg:: "-6";
APT::Compressor::xz::UncompressArg "";
APT::Compressor::xz::UncompressArg:: "-d";
APT::Compressor::lzma "";
APT::Compressor::lzma::Name "lzma";
APT::Compressor::lzma::Extension ".lzma";
APT::Compressor::lzma::Binary "xz";
APT::Compressor::lzma::Cost "5";
APT::Compressor::lzma::CompressArg "";
APT::Compressor::lzma::CompressArg:: "--format=lzma";
APT::Compressor::lzma::CompressArg:: "-9";
APT::Compressor::lzma::UncompressArg "";
APT::Compressor::lzma::UncompressArg:: "--format=lzma";
APT::Compressor::lzma::UncompressArg:: "-d";
APT::Compressor::::Name "";
APT::Compressor::::Extension ".";
APT::Compressor::::Binary "";
APT::Compressor::::Cost "100";
APT::Compressor::::CompressArg "";
APT::Compressor::::CompressArg:: "-9";
APT::Compressor::::UncompressArg "";
APT::Compressor::::UncompressArg:: "-d";
Dir "/";
Dir::State "var/lib/apt/";
Dir::State::lists "lists/";
Dir::State::cdroms "cdroms.list";
Dir::State::mirrors "mirrors/";
Dir::State::extended_states "extended_states";
Dir::State::status "/var/lib/dpkg/status";
Dir::Cache "var/cache/apt/";
Dir::Cache::archives "archives/";
Dir::Cache::srcpkgcache "srcpkgcache.bin";
Dir::Cache::pkgcache "pkgcache.bin";
Dir::Etc "etc/apt/";
Dir::Etc::sourcelist "sources.list";
Dir::Etc::sourceparts "sources.list.d";
Dir::Etc::vendorlist "vendors.list";
Dir::Etc::vendorparts "vendors.list.d";
Dir::Etc::main "apt.conf";
Dir::Etc::netrc "auth.conf";
Dir::Etc::parts "apt.conf.d";
Dir::Etc::preferences "preferences";
Dir::Etc::preferencesparts "preferences.d";
Dir::Etc::trusted "trusted.gpg";
Dir::Etc::trustedparts "trusted.gpg.d";
Dir::Bin "";
Dir::Bin::methods "/usr/lib/apt/methods";
Dir::Bin::solvers "";
Dir::Bin::solvers:: "/usr/lib/apt/solvers";
Dir::Bin::dpkg "/usr/bin/dpkg";
Dir::Bin::bzip2 "/bin/bzip2";
Dir::Bin::xz "/usr/bin/xz";
Dir::Media "";
Dir::Media::MountPath "/media/apt";
Dir::Log "var/log/apt";
Dir::Log::Terminal "term.log";
Dir::Log::History "history.log";
Dir::Ignore-Files-Silently "";
Dir::Ignore-Files-Silently:: "~$";
Dir::Ignore-Files-Silently:: "\.disabled$";
Dir::Ignore-Files-Silently:: "\.bak$";
Dir::Ignore-Files-Silently:: "\.dpkg-[a-z]+$";
Dir::Ignore-Files-Silently:: "\.save$";
Dir::Ignore-Files-Silently:: "\.orig$";
Acquire "";
Acquire::cdrom "";
Acquire::cdrom::mount "/media/cdrom/";
Acquire::Languages "";
Acquire::Languages:: "en";
Acquire::Languages:: "none";
DPkg "";
DPkg::Pre-Install-Pkgs "";
DPkg::Pre-Install-Pkgs:: "/usr/bin/apt-listchanges --apt || test $? -ne 10";
DPkg::Pre-Install-Pkgs:: "/usr/sbin/dpkg-preconfigure --apt || true";
DPkg::Tools "";
DPkg::Tools::Options "";
DPkg::Tools::Options::/usr/bin/apt-listchanges "";
DPkg::Tools::Options::/usr/bin/apt-listchanges::Version "2";
Unattended-Upgrade "";
Unattended-Upgrade::Origins-Pattern "";
Unattended-Upgrade::Origins-Pattern:: "origin=Debian,archive=stable,label=Debian-Security";
Unattended-Upgrade::Origins-Pattern:: "origin=Debian,archive=oldstable,label=Debian-Security";
CommandLine "";
CommandLine::AsString "apt-config dump";
-- (no /etc/apt/preferences present) --
-- /etc/apt/sources.list --
#deb http://www.mirrorservice.org/sites/ftp.debian.org/debian/ wheezy main contrib non-free
#deb-src http://www.mirrorservice.org/sites/ftp.debian.org/debian/ wheezy main contrib non-free
deb ftp://ftp.uk.debian.org/debian/ wheezy main non-free contrib
deb-src ftp://ftp.uk.debian.org/debian/ wheezy main non-free contrib
#deb ftp://ftp.fr.debian.org/debian/ wheezy main non-free contrib
#deb-src ftp://ftp.fr.debian.org/debian/ wheezy main non-free contrib
deb http://security.debian.org/ wheezy/updates main contrib non-free
deb-src http://security.debian.org/ wheezy/updates main contrib non-free
#deb http://mirrors.kernel.org/debian wheezy-updates main contrib non-free
#deb-src http://mirrors.kernel.org/debian wheezy-updates main contrib non-free
#deb http://www.deb-multimedia.org/ wheezy main
#deb-src http://www.deb-multimedia.org/ wheezy main
-- System Information:
Debian Release: 7.7
APT prefers stable
APT policy: (500, 'stable')
Architecture: i386 (i686)
Kernel: Linux 3.2.0-4-686-pae (SMP w/4 CPU cores)
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/bash
Versions of packages apt depends on:
ii debian-archive-keyring 2014.1~deb7u1
ii gnupg 1.4.12-7+deb7u6
ii libapt-pkg4.12 0.9.7.9+deb7u6
ii libc6 2.13-38+deb7u6
ii libgcc1 1:4.7.2-5
ii libstdc++6 4.7.2-5
apt recommends no packages.
Versions of packages apt suggests:
pn apt-doc <none>
ii aptitude 0.6.8.2-1
ii dpkg-dev 1.16.15
ii python-apt 0.8.8.2
ii xz-utils 5.1.1alpha+20120614-2
-- no debconf information
Reply to: