Bug#636292: MD5Sum mismatch is due to multiple DNS queries!
(cc's kept since I am not really sure everyone involved is in subscribed
to debian-mirrors. If you want me to start trimming them down, please
say so).
On Fri, 05 Aug 2011, Kurt Roeckx wrote:
> Even with multiple lines in the sources.list file I only see those
> 2 requests.
Hmm, a normal request like this is supposed to return a number of A or
AAAA records for, e.g. ftp.us.debian.org, and not just one.
Just so that we can close that door completely, does apt do the right
thing and use always the same A record or AAAA record from the returned
set, switching to the next one only if there are problems? I believe it
does it right, but it would be nice to have a definitive answer on it
(and I don't really grok apt to take a quick look at the source to check
it myself).
> (tested with apt 0.8.15.4, I doubt 0.8.15.5 behaves differently.)
>
> As far as I know the issues with hash sum mismatches is either one
> of:
> - They use an old version of the mirror script that didn't exclude
> InRelease in the first stage. As a result the InRelease file
> was already updated while the Packages/Sources file isn't for
> a long time. This has been a problem since ftp-master started
> generating those InRelease file, which was just after the
> squeeze release.
> - There is always a delay between updating the Release file and
> the Packages and Sources file, and the error should go away
> after a short time.
> - ftp-master generated broken files for some reason. It sometimes
> happen but not that often.
>
> So I suggest you make sure that all the mirrors that you see
> an issue with have updated their mirror script, since I think
> that's the biggest issue at the moment.
That is actually quite possible. However, it is also something we can
assert for sure:
> This was fixed with this commit in archvsync:
> commit 77223bb1af262e139a898020a05680e932d51888
> Author: Joerg Jaspert <joerg@debian.org>
> Date: Tue Feb 22 22:32:13 2011 +0100
>
> ftpsync
>
> update rsync_options1 to also exclude the newish InRelease files in the first run
>
> Signed-off-by: Joerg Jaspert <joerg@debian.org>
>
> This is part of the 80387 version that you can find in
> project/ftpsync/ on the Debian mirrors. 80387 was released
> the next day.
>
> If they are using this script to update the mirror, you should
> be able to see the version in project/trace/
>
> If there is no version in that file (only a date) they're probably
> using an even older script that's also broken.
So, it is time to inspect the project/trace/* files in every mirror on
the multi-mirror aliases that users have complained about.
> An other issue might be that you're behind some broken transparent
> proxy and your connection gets directed to a different servers for
> each file you get. As far as I know apt will only open 1
> connection to the server and requests all files over that single
> connection, so this really shouldn't happen.
That might not be true if it is a http/1.0 proxy, or if persistent
connections get disabled for whatever reason. In that case, apt would
have to make multiple connections, and therefore any proxy, transparent
or not, would likely round-robin over the multiple A and AAAA records.
The answer for that would be to update our repository format to have
something seqlock-like to allow apt to detect metadata generation
mismatch, and thus be able to automatically refetch things until it gets
all metadata with the same generation number:
http://en.wikipedia.org/wiki/Seqlock
Maybe using rsync or ftp can help, if it enforces the "get everything
using the same connection" that http might or might not allow apt to do.
But that does NOT scale well at the mirror server side, at all.
--
"One disk to rule them all, One disk to find them. One disk to bring
them all and in the darkness grind them. In the Land of Redmond
where the shadows lie." -- The Silicon Valley Tarot
Henrique Holschuh
Reply to: