Bug#456499: apt-transport-https: Errors with private repository and GPG keys

["MLA (Peter Clark)" <mla@forrussia.org>]

The thought struck me that maybe the problem was server-side, rather than 
client-side, so I swapped out lighttpd for apache, and what do you know, the 
error disappeared. Which is a pity, since apache is a bloated resource hog. I 
still wasn't sure whether lighttpd or apt were to blame, so I dug farther.

Further discovery: When things work, 
/var/lib/apt/lists/packages.mydomain.org_dists_unstable_Release 
and
/var/lib/apt/lists/packages.mydomain.org_dists_unstable_Release.gpg
are 728 bytes and 191 bytes, respectively. However, the next time I run 
apt-get, their file sizes are 0 and 381 bytes, respectively, and then 
Release.gpg is deleted. Which is why the *next* times works again, because 
Release.gpg is fetched anew. So now I understand why it works exactly 50% of 
the time, but I still don't understand what's happening.

Here is what's odd; in the debug log for apt (running `apt-get -o 
Debug::Acquire::https=true -o Debug::Acquire::gpgv=true update 2>apt.log`), I 
noticed the following:

> GET /dists/unstable/Release.gpg HTTP/1.1
User-Agent: Debian APT-CURL/1.0 (0.7.6ubuntu14)
Host: packages.mydomain.org
Accept: */*
Cache-Control: max-age=0
Range: bytes=190-
If-Range: Fri, 14 Dec 2007 09:51:25 GMT

The "Range: bytes=190-" is what attracted my attention, since none of the 
other GPG keys get that. 190 + 191 (the size of my Release.gpg file) = 381, 
which is the same size as Release.gpg when apt doesn't work! So apparently, 
rather than 
overwrite /v/l/a/l/packages.mydomain.org_dists_unstable_Release.gpg, the same 
file is being downloaded and appended to itself!

So then I turned to the source: I'm pretty bad at reading C, but here's what I 
believe to be the relevant portion of https.cc:

   // if we have the file send an if-range query with a range header
   if (stat(Itm->DestFile.c_str(),&SBuf) >= 0 && SBuf.st_size > 0)
   {
      char Buf[1000];
      sprintf(Buf,"Range: bytes=%li-\r\nIf-Range: %s\r\n",
              (long)SBuf.st_size - 1,
              TimeRFC1123(SBuf.st_mtime).c_str());
      headers = curl_slist_append(headers, Buf);
   }

That's Greek to me, sorry. If someone can translate that into something 
sensible, I would appreciate it. But in any case, I at least have a rough 
understanding (correct me if I'm wrong) of what is happening:
1. The first time apt-get is run for a server using https, there is no copy of 
Release.gpg cached, so when it is downloaded, it matches the size that server 
gives (191 bytes) and it validates without a problem.
2. The second time apt-get is run, it downloads 190 bytes of Release.gpg and 
*appends* this to the cached Release.gpg, which then of course fails to 
validate. Release.gpg is then dutifully erased, since it is invalid.
3. There is no local copy of Release.gpg, so when apt-get is run for a third 
time, the cycle repeats.

I haven't looked at what's going on with Release (why it is 0 bytes), but I 
suspect that if we solve the problem with Release.gpg, Release will likewise 
be fixed.

So who is to blame? Lighttpd or apt? Why, when apache is serving, are there no 
such errors? 

NB: What is probably not to blame: the self-signed certificate. I put a copy 
of the server certificate in /etc/ssl/certs/ca-certificates.crt, and apt 
happily accepts it without complaint.