Bug#343985: apt: "Bad header line" and "404 Not Found"

["Manuel Bilderbeek" <bilm@oce.nl>]

Package: apt
Version: 0.6.42.1
Severity: normal


Since the new APT (which does the gnupg stuff), I have problems doing an
apt-get update here. "Here" is relevant, because we use a proxy server
here.
I do an "apt-get update" every morning, and the first time always fails:

$ sudo apt-get update
Password:
Ign http://ftp.nl.debian.org testing Release.gpg
Ign http://ftp.nl.debian.org testing Release
Ign http://security.debian.org stable/updates Release.gpg
Ign http://security.debian.org stable/updates Release
Ign http://ftp.nl.debian.org testing/main Packages
Ign http://ftp.nl.debian.org testing/contrib Packages
Ign http://ftp.nl.debian.org testing/non-free Packages
Err http://ftp.nl.debian.org testing/main Packages
Bad header line
Ign http://http.us.debian.org testing Release.gpg
Err http://ftp.nl.debian.org testing/contrib Packages
Bad header line
Err http://ftp.nl.debian.org testing/non-free Packages
Bad header line
Ign http://security.debian.org stable/updates/main Packages
Ign http://security.debian.org stable/updates/contrib Packages
Ign http://security.debian.org stable/updates/non-free Packages
Ign http://http.us.debian.org testing Release
Err http://security.debian.org stable/updates/main Packages
404 Not Found
Err http://security.debian.org stable/updates/contrib Packages
404 Not Found
Get:1 http://security.debian.org stable/updates/non-free
Packages [164kB]
Ign http://http.us.debian.org testing/main Packages
Ign http://http.us.debian.org testing/contrib Packages
Ign http://http.us.debian.org testing/non-free Packages
Err http://http.us.debian.org testing/main Packages
Bad header line
Err http://http.us.debian.org testing/contrib Packages
Bad header line
Err http://http.us.debian.org testing/non-free Packages
Bad header line
Fetched 164kB in 2s (63.1kB/s)
Failed to fetch http://ftp.nl.debian.org/debian/dists/testing/main/binary-i386/Packages.gz Bad header line
Failed to fetch http://ftp.nl.debian.org/debian/dists/testing/contrib/binary-i386/Packages.gz Bad header line
Failed to fetch http://ftp.nl.debian.org/debian/dists/testing/non-free/binary-i386/Packages.gz Bad header line
Failed to fetch http://security.debian.org/dists/stable/updates/main/binary-i386/Packages.gz 404 Not Found
Failed to fetch http://security.debian.org/dists/stable/updates/contrib/binary-i386/Packages.gz 404 Not Found
Failed to fetch http://http.us.debian.org/debian/dists/testing/main/binary-i386/Packages.gz Bad header line
Failed to fetch http://http.us.debian.org/debian/dists/testing/contrib/binary-i386/Packages.gz Bad header line
Failed to fetch http://http.us.debian.org/debian/dists/testing/non-free/binary-i386/Packages.gz Bad header line
Reading package lists... Done
E: Some index files failed to download, they have been ignored, or old ones used instead.


However, when I do it again, right after this, it does work:


$ sudo apt-get update
Get:1 http://security.debian.org stable/updates Release.gpg [189B]
Get:2 http://security.debian.org stable/updates Release [20.0kB]
Get:3 http://ftp.nl.debian.org testing Release.gpg [189B]
Get:4 http://ftp.nl.debian.org testing Release [35.9kB]
Get:5 http://http.us.debian.org testing Release.gpg [189B]
Get:6 http://http.us.debian.org testing Release [35.9kB]
Ign http://security.debian.org stable/updates/main Packages
Get:7 http://ftp.nl.debian.org testing/main Packages [2715kB]
Ign http://security.debian.org stable/updates/contrib Packages
Ign http://security.debian.org stable/updates/non-free Packages
Get:8 http://security.debian.org stable/updates/main Packages [164kB]
Hit http://security.debian.org stable/updates/contrib Packages
Get:9 http://http.us.debian.org testing/main Packages [2715kB]
Hit http://security.debian.org stable/updates/non-free Packages
Get:10 http://ftp.nl.debian.org testing/contrib Packages [43.7kB]
Get:11 http://ftp.nl.debian.org testing/non-free Packages [51.4kB]
Get:12 http://http.us.debian.org testing/contrib Packages [43.7kB]
Get:13 http://http.us.debian.org testing/non-free Packages [51.4kB]
Fetched 5876kB in 36s (161kB/s)
Reading package lists... Done

At home, where I don't use a proxy, I have no problems.
Hinted by bug #273447 I tried the

Acquire::http::Pipeline-Depth 0

but that didn't help a bit.

-- Package-specific info:

-- apt-config dump --

APT "";
APT::Architecture "i386";
APT::Build-Essential "";
APT::Build-Essential:: "build-essential";
Dir "/";
Dir::State "var/lib/apt/";
Dir::State::lists "lists/";
Dir::State::cdroms "cdroms.list";
Dir::State::userstatus "status.user";
Dir::State::status "/var/lib/dpkg/status";
Dir::Cache "var/cache/apt/";
Dir::Cache::archives "archives/";
Dir::Cache::srcpkgcache "srcpkgcache.bin";
Dir::Cache::pkgcache "pkgcache.bin";
Dir::Etc "etc/apt/";
Dir::Etc::sourcelist "sources.list";
Dir::Etc::vendorlist "vendors.list";
Dir::Etc::vendorparts "vendors.list.d";
Dir::Etc::main "apt.conf";
Dir::Etc::parts "apt.conf.d";
Dir::Etc::preferences "preferences";
Dir::Bin "";
Dir::Bin::methods "/usr/lib/apt/methods";
Dir::Bin::dpkg "/usr/bin/dpkg";
DPkg "";
DPkg::Pre-Install-Pkgs "";
DPkg::Pre-Install-Pkgs:: "/usr/sbin/dpkg-preconfigure --apt || true";
DPkg::Post-Invoke "";
DPkg::Post-Invoke:: "if [ -x /usr/sbin/localepurge ] && [ $(ps w -p $PPID | grep -c remove) != 1 ]; then /usr/sbin/localepurge; else exit 0; fi";
Acquire "";
Acquire::http "";
Acquire::http::Proxy "http://cache-rd.oce.nl:80";;
Acquire::http::Pipeline-Depth "0";

-- (no /etc/apt/preferences present) --


-- /etc/apt/sources.list --

#deb copy:///cdrom/ sarge main

#deb http://http.us.debian.org/debian/ testing main  contrib
#deb http://ftp.nl.debian.org/debian/ testing main  contrib
#deb-src http://ftp.nl.debian.org/debian/ testing main  contrib

deb http://ftp.nl.debian.org/debian testing main contrib non-free
#deb http://ftp.nl.debian.org/debian stable main contrib non-free
deb http://http.us.debian.org/debian testing main contrib non-free

#deb http://people.debian.org/~asac/testing ./
#deb-src http://people.debian.org/~asac/testing ./

#deb http://people.debian.org/~asac/experimental/ ./
#deb-src http://people.debian.org/~asac/experimental/ ./

deb http://security.debian.org/ stable/updates main contrib non-free

-- System Information:
Debian Release: testing/unstable
  APT prefers testing
  APT policy: (500, 'testing'), (500, 'stable')
Architecture: i386 (i686)
Shell:  /bin/sh linked to /bin/bash
Kernel: Linux 2.6.12-1-686
Locale: LANG=C, LC_CTYPE=C (charmap=ANSI_X3.4-1968)

Versions of packages apt depends on:
ii  libc6                         2.3.5-8    GNU C Library: Shared libraries an
ii  libgcc1                       1:4.0.2-5  GCC support library
ii  libstdc++6                    4.0.2-5    The GNU Standard C++ Library v3

apt recommends no packages.

-- no debconf information