[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Bug#743298: marked as done (apt-get corrupts Sources and Packages files on running unxz)

Your message dated Tue, 1 Apr 2014 19:35:20 +0200
with message-id <[🔎] 20140401173519.GA5341@crossbow>
and subject line Re: Bug#743298: apt-get corrupts Sources and Packages files on running unxz
has caused the Debian Bug report #743298,
regarding apt-get corrupts Sources and Packages files on running unxz
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact owner@bugs.debian.org
immediately.)


-- 
743298: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=743298
Debian Bug Tracking System
Contact owner@bugs.debian.org with problems
--- Begin Message --- 
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Package: apt
Version: 0.9.16.1

I have 2 hosts downloading sid from http://ftp.debian.org.
One host ("elmer") claims on "apt-get update"

W: Failed to fetch xz:/var/lib/apt/lists/partial/ftp.debian.org_debian_dists_sid_main_source_Sources  Hash Sum mismatch
W: Failed to fetch xz:/var/lib/apt/lists/partial/ftp.debian.org_debian_dists_sid_main_binary-amd64_Packages  Hash Sum mismatch
E: Some index files failed to download. They have been ignored, or old ones used instead.

The other host ("cecil") is fine on running "apt-get update".

Looking at the md5sums both hosts received the same files:

elmer:
# unxz -c /var/lib/apt/lists/partial/ftp.debian.org_debian_dists_sid_main_source_Sources | md5sum
556de395edcbe37e205962641ad0b04d  -
# unxz -c /var/lib/apt/lists/partial/ftp.debian.org_debian_dists_sid_main_binary-amd64_Packages | md5sum
540ce340af3a3f0106776a4ac9d2c4dc  -

cecil:
# cd /var/lib/apt/lists/
# md5sum ftp.debian.org_debian_dists_sid_main_source_Sources
556de395edcbe37e205962641ad0b04d  ftp.debian.org_debian_dists_sid_main_source_Sources
# md5sum ftp.debian.org_debian_dists_sid_main_binary-amd64_Packages
540ce340af3a3f0106776a4ac9d2c4dc  ftp.debian.org_debian_dists_sid_main_binary-amd64_Packages

Looking at the *.decomp.FAILED files on elmer I get:
# md5sum /var/lib/apt/lists/partial/ftp.debian.org_debian_dists_sid_main_source_Sources.decomp.FAILED
b5d614111ab0b408cc93ff0458157104  /var/lib/apt/lists/partial/ftp.debian.org_debian_dists_sid_main_source_Sources.decomp.FAILED
# md5sum /var/lib/apt/lists/partial/ftp.debian.org_debian_dists_sid_main_binary-amd64_Packages.decomp.FAILED
04b7a5d389eaa39ee160e0ebb3363a77  /var/lib/apt/lists/partial/ftp.debian.org_debian_dists_sid_main_binary-amd64_Packages.decomp.FAILED


The files uncompressed by apt-get on elmer appear to be cut off.
wc says:

cecil:
# wc ftp.debian.org_debian_dists_sid_main_binary-amd64_Packages ftp.debian.org_debian_dists_sid_main_source_Sources
  814399  2425128 33210072 ftp.debian.org_debian_dists_sid_main_binary-amd64_Packages
  639189  1863722 32532437 ftp.debian.org_debian_dists_sid_main_source_Sources
 1453588  4288850 65742509 total

elmer:
# wc partial/ftp.debian.org_debian_dists_sid_main_binary-amd64_Packages.decomp.FAILED partial/ftp.debian.org_debian_dists_sid_main_source_Sources.decomp.FAILED
  638218  1899565 26189824 partial/ftp.debian.org_debian_dists_sid_main_binary-amd64_Packages.decomp.FAILED
   22305    63063  1064960 partial/ftp.debian.org_debian_dists_sid_main_source_Sources.decomp.FAILED
  660523  1962628 27254784 total

Please note that I get this problem only with the files shown above. Other
Sources and Packages files are fine.


There is no proxy involved (AFAICT).

Harri
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQEcBAEBCAAGBQJTOsxlAAoJEAqeKp5m04HLVoIH/0J/oyQHbFiWT/U32RpZZIYg
mVpQeepaHx75ztiLnRtAlV/WZHh/S4qQDFuB23IP7miIzx2CS+1mM3Bs4dWtR6OV
VNvE0OPcwQmBy8XLxMMot1i9tn7xclgKF/06Y+pEyFyJthpLdleAm5to6oInI/UI
MxE0n2vmRuCDUcPb6eCrv3lQiCRcvlczMXDlvowM2PbOrNsccMbvryLmuONLKHrK
lKVAW4QNMX8fEyQIfuij3nl5zPYXTQO4rVn1sWZ7sP/OZ6cSFeOze+GnpIR+fAoB
VjtbkxhAr96nzj8NrbY2cMpz0Px7Zcw2oVMmAbNtuOuGAiWjPuCYs2NxEnVA0Eg=
=RiGW
-----END PGP SIGNATURE-----

--- End Message ---
--- Begin Message --- 
Version: 1.0

Hi,

On Tue, Apr 01, 2014 at 04:25:47PM +0200, Harald Dunkel wrote:
> I have 2 hosts downloading sid from http://ftp.debian.org.
> One host ("elmer") claims on "apt-get update"
> 
> W: Failed to fetch xz:/var/lib/apt/lists/partial/ftp.debian.org_debian_dists_sid_main_source_Sources  Hash Sum mismatch
> W: Failed to fetch xz:/var/lib/apt/lists/partial/ftp.debian.org_debian_dists_sid_main_binary-amd64_Packages  Hash Sum mismatch
> E: Some index files failed to download. They have been ignored, or old ones used instead.

Thanks for the report!
Unfortunately it's a (hidden) duplicate and fixed in a recent upload,
so I have to close it again… but let me explain a bit before that:

As noted in #624122 – our catch-all hashsum mismatch bugreport(s) –
I botched uncompressing of lzma/xz in 0.9.16 while introducing
(un)compression via liblzma instead of an optional external binary.

The problem is that it can happen that the code reads 4096 compressed
bytes, which do not produce any uncompressed output. The code then came
to the conclusion that it is done uncompressing the file… which happens
to be wrong in this case resulting in a too short file and therefore in
a hashsum mismatch as a too short file of course doesn't match.

This happens "rarely", so by any chance the next dinstall run will
generate a new file which doesn't expose this behavior – just like our
testcases do not expose this at all.


A fix is known for a while, but was stuck in git as nobody with
upload rights and sufficient time was around to upload it.
(We can argue now if that was so to punish me or just by chance –
 I opt for the former, which is what I deserved I guess)
On the upside, this has changed today, so that this fix is finally
included – and even in a very important release for us. ;)


In the meantime, the last workaround mentioned in #624122 is actually
the only one working in case the problem is this one. It doesn't help
for all the cases which usually end up in #624122. Just like the
config options mentioned before that about another issue ending up with
that error message as #624122 has no workaround if you get it for
"hashsum mismatch" rather than a "code practice vs. theory mismatch"
(and I can only break so many apt versions).


Best regards

David Kalnischkies

Attachment: signature.asc
Description: Digital signature


--- End Message ---
Reply to: