Bug#813000: apt complains about corrupted index files even after multiple updates
On 30 January 2016 at 13:42, Stefan Bühler <stbuehler@web.de> wrote:
> Hi,
>
> On Thu, 28 Jan 2016 12:53:18 +0000 Rohan Garg <rohangarg@kubuntu.org>
> wrote:
>> Package: apt
>> Version: 1.2.1
>> Severity: important
>>
>> Dear Maintainer,
>>
>> It seems that after the 1.2.1 update apt fails to install packages
>> even after multiple 'apt update' calls.
>>
>> I constantly see this error on the console output :
>>
>> E: The package index files are corrupted. No Filename: field for
>> package libpopt0.
>
> I can confirm similar problems. The affected package varies from time
> to time, sometimes it doesn't fail at the "Filename: " but claims a
> downloaded file has wrong file size or even downloads completely
> unrelated packages.
>
> All this happens when I try to install many packages at once in a fresh
> docker environment (created with debootstrap).
>
> I got some valgrind backtraces:
>
> First issue (patch to fix it attached):
>
> ==890== Mismatched free() / delete / delete []
> ==890== at 0x4C2B30B: operator delete(void*) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==890== by 0x4EDCA0A: DirectFileFdPrivate::~DirectFileFdPrivate() (in /usr/lib/x86_64-linux-gnu/libapt-pkg.so.5.0.0)
> [...]
> ==890== Address 0x709bb00 is 0 bytes inside a block of size 4,096 alloc'd
> ==890== at 0x4C2A8CF: operator new[](unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==890== by 0x4EDD6AB: FileFdPrivate::FileFdPrivate(FileFd*) (in /usr/lib/x86_64-linux-gnu/libapt-pkg.so.5.0.0)
Thanks, applied.
> [...]
>
> Second issue (with some offsets between):
>
> 0x4E36000: /usr/lib/x86_64-linux-gnu/libapt-pkg.so.5.0.0
> +0x1175d0: pkgCache::ReMap
> ==890== Conditional jump or move depends on uninitialised value(s)
> ==890== at 0x4F4D8A0: pkgCache::ReMap(bool const&) (in /usr/lib/x86_64-linux-gnu/libapt-pkg.so.5.0.0)
> at pkgCache::ReMap+0x2d0 (0x1178a0)
> ==890== by 0x4F55467: pkgCacheGenerator::pkgCacheGenerator(DynamicMMap*, OpProgress*) (in /usr/lib/x86_64-linux-gnu/libapt-pkg.so.5.0.0)
> ==890== by 0x4F575EF: pkgCacheGenerator::MakeStatusCache(pkgSourceList&, OpProgress*, MMap**, pkgCache**, bool) (in /usr/lib/x86_64-linux-gnu/libapt-pkg.so.5.0.0)
> ==890== by 0x4EBA907: pkgCacheFile::BuildCaches(OpProgress*, bool) (in /usr/lib/x86_64-linux-gnu/libapt-pkg.so.5.0.0)
> ==890== by 0x4EC0742: APT::CacheSetHelper::PackageFromPackageName(APT::PackageContainerInterface*, pkgCacheFile&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) (in /usr/lib/x86_64-linux-gnu/libapt-pkg.so.5.0.0)
> ==890== by 0x4EC0D40: APT::CacheSetHelper::PackageFrom(APT::CacheSetHelper::PkgSelector, APT::PackageContainerInterface*, pkgCacheFile&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (in /usr/lib/x86_64-linux-gnu/libapt-pkg.so
> .5.0.0)
> ==890== by 0x4EBD749: APT::CacheSetHelper::PackageFromString(APT::PackageContainerInterface*, pkgCacheFile&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (in /usr/lib/x86_64-linux-gnu/libapt-pkg.so.5.0.0)
> ==890== by 0x4EBED3B: APT::VersionContainerInterface::FromString(APT::VersionContainerInterface*, pkgCacheFile&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, APT::CacheSetHelper::VerSelector, APT::CacheSetHelper&, bool) (in /usr/lib/x86_64-linux-gnu/libapt-pkg.so.5.0.0)
> ==890== by 0x4EBFC07: APT::VersionContainerInterface::FromCommandLine(APT::VersionContainerInterface*, pkgCacheFile&, char const**, APT::CacheSetHelper::VerSelector, APT::CacheSetHelper&) (in /usr/lib/x86_64-linux-gnu/libapt-pkg.so.5.0.0)
> ==890== by 0x51E071A: ShowPackage(CommandLine&) (in /usr/lib/x86_64-linux-gnu/libapt-private.so.0.0.0)
> ==890== by 0x4ECE5A5: CommandLine::DispatchArg(CommandLine::Dispatch const*, bool) (in /usr/lib/x86_64-linux-gnu/libapt-pkg.so.5.0.0)
> ==890== by 0x51BFEE2: DispatchCommandLine(CommandLine&, std::vector<CommandLine::Dispatch, std::allocator<CommandLine::Dispatch> > const&) (in /usr/lib/x86_64-linux-gnu/libapt-private.so.0.0.0)
>
> I managed to track down the code (pkgCache::ReMap+0x2d0) in gdbs
> disassembly to pkgcache.cc:199:
> if (hash != HeaderP->CacheFileSize)
That's not an issue, it's expected. We mmap() the memory, and then
hash the entire memory we mapped, as that's what we are writing to the
disk. It does not matter that there are uninitialized values in there,
we just want to check that if we have written it to a file and read it
back, that it's the same.
>
> Disabling lz4 seems to fix the issue (but not the valgrind report):
>
> echo 'APT::Compressor::lz4::Cost "5000";' > /etc/apt/apt.conf.d/disable-lz4
> rm /var/lib/apt/lists/*.lz4
> apt-get update
OK. Not sure how that happens.
--
Julian Andres Klode - Debian Developer, Ubuntu Member
See http://wiki.debian.org/JulianAndresKlode and http://jak-linux.org/.
Reply to: