--- Begin Message ---
Package: apt
Version: 0.6.46.4-0.1
Severity: important
Tags: patch
Using apt-proxy, apt-get update sometimes failed mysteriously to fetch files
known to exist.
>From the outside, this looks as follows:
APT tries to fetch a non-existent file (in our case, a DiffIndex).
You see a HTTP GET on the wire and a 404 response.
APT records the failure as Ign.
APT tries to fetch an existent URL.
You don't see anything on the wire.
APT accuses the HTTP server of having closed the connection
and records failure on the existent URL.
APT continues normally with the next URL.
>From the inside, the HTTP acquire method goes into great lengths to shoot
itself in the foot (or, more precisely, several feet) as follows:
ServerState::RunHeaders() sets Persistent = false
ServerState::HeaderLine() gets a Transfer-Encoding: chunked header.
It deduces from this, that a Body must be present, which in fact is correct,
only it might not to be the entirely right way to deduce this.
It also receives a Connection: close header.
HttpMethod::DealWithHeaders() recognizes the 404 and returns 4.
HttpMethod::Loop() calls Fail().
HttpMethod::Loop() then continues to skip the Body.
This is superflous because the connection is known to be closed afterwards.
ServerState::RunData() calls HttpMethod::Go().
HttpMethod::Go() calls CircleBuf::Read().
CircleBuf::Read() calls read() until this returns 0.
(because the server closed the connection).
HttpMethod::Go() then calls HttpMethod::ServerDie().
HttpMethod::ServerDie() calls _error and registers a closed connection.
It also closes the connection via ServerState::Close().
HttpMethod::Loop() closes the connection again because Server->Persistent
has been cleared (it doesn't check for the fd to be -1). This is harmless.
HttpMethod::Loop() then tries to re-open the connection via ServerState::Open().
ServerState::Open() calls Connect() (from methods/connect.cc).
Connect() notices a PendingError() and returns false.
The problem may be viewed as Fail() being called too early.
One could also avoid skipping the Body if the connection is already known
to be closed thereafter.
More generally, I have the impression that HttpMethod::Go() should not
call ServerDie() if Read() returns false if the connection may correctly
be closed by the server. I haven't looked into this.
The attached patch fixes our problem by moving the call to Fail() further down.
-- Package-specific info:
-- (no /etc/apt/preferences present) --
-- (/etc/apt/sources.list present, but not submitted) --
-- System Information:
Debian Release: 4.0
APT prefers stable
APT policy: (500, 'stable')
Architecture: i386 (i686)
Shell: /bin/sh linked to /bin/bash
Kernel: Linux 2.6.24.wap
Locale: LANG=de_DE@euro, LC_CTYPE=de_DE@euro (charmap=ISO-8859-15)
Versions of packages apt depends on:
ii debian-archive-keyring 2007.07.31~etch1 GnuPG archive keys of the Debian a
ii libc6 2.3.6.ds1-13etch4 GNU C Library: Shared libraries
ii libgcc1 1:4.1.1-21 GCC support library
ii libstdc++6 4.1.1-21 The GNU Standard C++ Library v3
apt recommends no packages.
-- no debconf information
--- methods/http.cc.orig 2006-12-04 15:37:36.000000000 +0100
+++ methods/http.cc 2008-02-12 17:37:52.000000000 +0100
@@ -1200,13 +1200,13 @@
// We need to flush the data, the header is like a 404 w/ error text
case 4:
{
- Fail();
-
// Send to content to dev/null
File = new FileFd("/dev/null",FileFd::WriteExists);
Server->RunData();
delete File;
File = 0;
+
+ Fail();
break;
}
--- End Message ---
--- Begin Message ---
Source: apt
Source-Version: 1.4~beta1
We believe that the bug you reported is fixed in the latest version of
apt, which is due to be installed in the Debian FTP archive.
A summary of the changes between this version and the previous one is
attached.
Thank you for reporting the bug, which will now be closed. If you
have further comments please address them to 465572@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.
Debian distribution maintenance software
pp.
Julian Andres Klode <jak@debian.org> (supplier of updated apt package)
(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512
Format: 1.8
Date: Fri, 25 Nov 2016 23:49:54 +0100
Source: apt
Binary: apt libapt-pkg5.0 libapt-inst2.0 apt-doc libapt-pkg-dev libapt-pkg-doc apt-utils apt-transport-https
Architecture: source
Version: 1.4~beta1
Distribution: unstable
Urgency: medium
Maintainer: APT Development Team <deity@lists.debian.org>
Changed-By: Julian Andres Klode <jak@debian.org>
Description:
apt - commandline package manager
apt-doc - documentation for APT
apt-transport-https - https download transport for APT
apt-utils - package management related utility programs
libapt-inst2.0 - deb package format runtime library
libapt-pkg-dev - development files for APT's libapt-pkg and libapt-inst
libapt-pkg-doc - documentation for APT development
libapt-pkg5.0 - package management runtime library
Closes: 272557 465572 644610 767891 837395 838779 840552 840757 841763 841874 842877 844724 845599
Changes:
apt (1.4~beta1) unstable; urgency=medium
.
[ Chris Leick ]
* Updated German documentation translation
* fix three typos in sources & manpages
* German translation proof read by Helge Kreutzmann
.
[ Frans Spiesschaert ]
* Dutch program translation update (Closes: #840552)
* Dutch manpages translation update (Closes: #840757)
.
[ David Kalnischkies ]
* don't install new deps of candidates for kept back pkgs
* keep Release.gpg on untrusted to trusted IMS-Hit (Closes: 838779)
* fix testcase expecting incorrect remove log from dpkg
* reset HOME, USER(NAME), TMPDIR & SHELL in DropPrivileges (Closes: 842877)
* show the conflicting distribution warning again (Closes: 841874)
* rename Checksum-FileSize to Filesize in hashsum mismatch
* improve SOCKS error messages for http slightly
* support 'apt build-dep .' (aka: without /)
* add TMP/TEMP/TEMPDIR to the TMPDIR DropPrivileges dance
* http: skip connection cleanup if we close it anyhow
* add hidden config to set packages as Essential/Important.
Thanks to Anthony Towns for initial patch (Closes: 767891)
* don't warn if untransformed distribution matches.
Thanks to Lukas Anzinger for initial patch (Closes: 644610)
* show distribution mismatch for changed codenames
* react to trig-pend only if we have nothing else to do
* correct cross & disappear progress detection
* improve arch-unqualified dpkg-progress parsing
* don't perform implicit crossgrades involving M-A:same
* do not configure unconfigured to be removed packages
* skip unconfigure for unconfigured to-be removed pkgs
* report apt-key errors via status-fd messages (LP: #1522988)
* add apt-key support for armored GPG key files (*.asc)
* document which keyring formats are supported by apt-key (Closes: 844724)
* get pdiff files from the same mirror as the index
* follow the googletest merge in build-depends
.
[ Michael Vogt ]
* Do not (re)start "apt-daily.system"
Thanks to Alexandre Detiste (Closes: #841763)
.
[ Johannes Schauer ]
* add support for Build-Depends/Conflicts-Arch (Closes: #837395)
.
[ Edgar Fuß ]
* http: clear content before reporting the failure (Closes: #465572)
.
[ James Clarke ]
* apt-ftparchive: Support NotAutomatic and ButAutomaticUpgrades fields
(Closes: #272557)
.
[ Julian Andres Klode ]
* TagSection: Split AlphaIndexes into AlphaIndexes and BetaIndexes
* TagSection: Extract Find() methods taking Pos instead of Key
* Squashed 'triehash/' content from commit 16f59e1
* TagSection: Introduce functions for looking up by key ids
* debListParser: Convert to use pkgTagSection::Key-based lookup
* Bump the cache major version for non-backportable changes
* Introduce tolower_ascii_unsafe() and use it for hashing
* Optimize VersionHash() to not need temporary copy of input
* Compare size before data when ordering cache bucket entries
* debListParser: Micro-optimize AvailableDescriptionLanguages()
* Do not use MD5SumValue for Description_md5()
* gpgv: Untrust SHA1, RIPE-MD/160, but allow downgrading to weak
.
[ Paul Wise ]
* show output as documented for APT::Periodic::Verbose 2 (Closes: 845599)
Checksums-Sha1:
9f49c6a174ec4da92b27041060cb3531dfa93337 2565 apt_1.4~beta1.dsc
bde0ba2ec01caa7856bb02255ab96ed0bba6d56d 2053936 apt_1.4~beta1.tar.xz
Checksums-Sha256:
59965ea9f2489feeb510d945f8ec681f8cedf6ff9ddc413c02423b2f48b0d9c1 2565 apt_1.4~beta1.dsc
7321ec058d2a7d664e1b5123a1d73fc0d63738eb97783df8dce45e8a2b2898c1 2053936 apt_1.4~beta1.tar.xz
Files:
0bd8495d92665d653fc0552a29a1e707 2565 admin important apt_1.4~beta1.dsc
08e36fb472189a177a8f1b55949e41ae 2053936 admin important apt_1.4~beta1.tar.xz
-----BEGIN PGP SIGNATURE-----
iQJDBAEBCgAtFiEEzeVhi4gF/W4gLOnC1zw55WWAs4YFAlg4wOQPHGpha0BkZWJp
YW4ub3JnAAoJENc8OeVlgLOGoDcP/0TuSlspjqIdw0V7mP8jvDg69jH2WDVv4GGJ
8/+JTSry8k7x7X07GaZX7bPmSUbFB1JMeP1+SWyFYa/J3e/GwqTWhKllojjvCo79
XnduRT6KsMrzznzB4CIlS9MwpIn6Wz3hxogJYiJcST+8243m59QsqhdnnD+fWrHG
IpLk8cPgxrVPnw+9cSXt/zuJVuuZL2ExMBk+QaRYpSXXCdMN7rUuBCoWP4RE5hTy
isGA2K53tHt7QYpo6175N3zwcuFyOTzp4XuuOlemG2FcIHjIpZ978EhA8bLdh5QW
OSbW8suti7IwX7+TJyOjYOOGmZSwa+34f6lq4ED0N79SRQAaCpOKqx6OMWFnvDhN
vEeMXpaWZSsf1QIgQ6V27fMWHVncQJ5SO+Dz8E1VTmgqVU/kMSb6JKW06WCkH5fO
ebNfv1l9UndWQQAvVrLTS8SduR334lZN2Z1FfJLPVHLZTqfSNZjLQWuVwbnZReDR
GIEgmXge1MEmHFDW33/xISOe3kLJfWtSjQxe2NbvTKKKGvwEzawVQNlJxAOGmhMW
ZbGXSx4j7YWlUmBVObHSIsHqCLZov4pLA08CsYu6g3DE/YyuQqrzmIRC/cS6yR5K
lZ+6sJLnaWaTxjEsBsT5AKNpdoOB6vnodNJY4bY0malBcCDfv8BpnuApmyFRzyCw
sUscE3+G
=eM/Q
-----END PGP SIGNATURE-----
--- End Message ---