Bug#68954: apt-get: fails on transparent HTTP proxy
Package: apt
Version: 0.3.19
When trying to do apt-get update (or any other "fetching" function),
apt-get fails with a "bad header line" message; it's not clear what
exactly the bad header line is.
Fact is that to the client it looks like you're connecting directly
to the site you specify, but the corporate firewall/etc. redirects
this through a transparent proxy, which is probably screwing things
up. I've included selected parts of strace output below to help
show what actually gets received.
10530 write(3, "GET /debian/dists/potato/main/bi"..., 890) = 890
| 00000 47 45 54 20 2f 64 65 62 69 61 6e 2f 64 69 73 74 GET /deb ian/dist |
| 00010 73 2f 70 6f 74 61 74 6f 2f 6d 61 69 6e 2f 62 69 s/potato /main/bi |
| 00020 6e 61 72 79 2d 69 33 38 36 2f 50 61 63 6b 61 67 nary-i38 6/Packag |
| 00030 65 73 2e 67 7a 20 48 54 54 50 2f 31 2e 31 0d 0a es.gz HT TP/1.1.. |
| 00040 48 6f 73 74 3a 20 77 77 77 2e 75 6b 2e 64 65 62 Host: ww w.uk.deb |
| 00050 69 61 6e 2e 6f 72 67 0d 0a 43 6f 6e 6e 65 63 74 ian.org. .Connect |
| 00060 69 6f 6e 3a 20 6b 65 65 70 2d 61 6c 69 76 65 0d ion: kee p-alive. |
| 00070 0a 55 73 65 72 2d 41 67 65 6e 74 3a 20 44 65 62 .User-Ag ent: Deb |
| 00080 69 61 6e 20 41 50 54 2d 48 54 54 50 2f 31 2e 32 ian APT- HTTP/1.2 |
[...]
10530 read(3, "HTTP/1.1 200 OK\r\nDate: Fri, 11 A"..., 65536) = 2896
| 00000 48 54 54 50 2f 31 2e 31 20 32 30 30 20 4f 4b 0d HTTP/1.1 200 OK. |
| 00010 0a 44 61 74 65 3a 20 46 72 69 2c 20 31 31 20 41 .Date: F ri, 11 A |
| 00020 75 67 20 32 30 30 30 20 31 31 3a 34 30 3a 31 30 ug 2000 11:40:10 |
| 00030 20 47 4d 54 0d 0a 53 65 72 76 65 72 3a 20 41 70 GMT..Se rver: Ap |
| 00040 61 63 68 65 2f 31 2e 33 2e 39 20 28 55 6e 69 78 ache/1.3 .9 (Unix |
| 00050 29 20 44 65 62 69 61 6e 2f 47 4e 55 20 6d 6f 64 ) Debian /GNU mod |
| 00060 5f 73 73 6c 2f 32 2e 34 2e 31 30 20 4f 70 65 6e _ssl/2.4 .10 Open |
| 00070 53 53 4c 2f 30 2e 39 2e 34 0d 0a 4c 61 73 74 2d SSL/0.9. 4..Last- |
| 00080 4d 6f 64 69 66 69 65 64 3a 20 53 61 74 2c 20 32 Modified : Sat, 2 |
| 00090 32 20 4a 75 6c 20 32 30 30 30 20 31 39 3a 34 36 2 Jul 20 00 19:46 |
| 000a0 3a 35 38 20 47 4d 54 0d 0a 45 54 61 67 3a 20 22 :58 GMT. .ETag: " |
| 000b0 32 32 32 66 30 36 2d 63 38 37 33 65 2d 33 39 37 222f06-c 873e-397 |
| 000c0 39 66 61 33 32 22 0d 0a 41 63 63 65 70 74 2d 52 9fa32".. Accept-R |
| 000d0 61 6e 67 65 73 3a 20 62 79 74 65 73 0d 0a 43 6f anges: b ytes..Co |
| 000e0 6e 6e 65 63 74 69 6f 6e 3a 20 63 6c 6f 73 65 0d nnection : close. |
| 000f0 0a 43 6f 6e 74 65 6e 74 2d 54 79 70 65 3a 20 61 .Content -Type: a |
| 00100 70 70 6c 69 63 61 74 69 6f 6e 2f 62 69 6e 61 72 pplicati on/binar |
| 00110 79 0d 0a 43 6f 6e 74 65 6e 74 2d 45 6e 63 6f 64 y..Conte nt-Encod |
| 00120 69 6e 67 3a 20 78 2d 67 7a 69 70 0d 0a 43 6f 6e ing: x-g zip..Con |
| 00130 6e 65 63 74 69 6f 6e 3a 20 63 6c 6f 73 65 0d 0a nection: close.. |
| 00140 50 72 6f 78 79 2d 43 6f 6e 6e 65 63 74 69 6f 6e Proxy-Co nnection |
| 00150 3a 20 63 6c 6f 73 65 0d 0a 41 63 63 65 70 74 2d : close. .Accept- |
| 00160 45 6e 63 6f 64 69 6e 67 3a 0d 0a 43 6f 6e 74 65 Encoding :..Conte |
| 00170 6e 74 2d 4c 65 6e 67 74 68 3a 20 38 32 31 30 35 nt-Lengt h: 82105 |
| 00180 34 0d 0a 0d 0a 1f 8b 08 00 b1 f5 79 39 02 03 d4 4....... ...y9... |
| 00190 5b 69 73 dc 46 92 fd de bf a2 c2 11 1b 92 62 fa [is.F... ......b. |
| 001a0 c0 7d 70 b5 0e d3 ba ac 59 69 ac 15 e5 b5 66 bf .}p..... Yi....f. |
[...]
[here at the end of the given file (the 821054 bytes), there is no
more data from the socket]
10530 write(1, "400 URI Failure\nURI: http://www.";..., 145) = 145
[...]
10527 write(1, "\r \rErr http"..., 74) = 74
10527 write(1, " Bad header line\n", 18) = 18
[...]
As a workaroung I currently use ftp, but that fetches the Packages.gz
etc. every time, even when nothing has been changed; the http method
correctly sees there's been no change.
The same thing happens if I explicitly define a proxy with
export http_proxy=http://my.proxy.name:80/
I'm assuming that a fix isn't too hard in this case?
FYI, the proxy is HTTP-GW p1.4/1 from the TIS firewall toolkit
( http://www.tis.com ) and Gauntlet
( http://www.tis.com/Home/NetworkSecurity/Gauntlet/Gauntlet.html )
as far as I can tell.
Paul Slootman
Reply to: