Bug#813000: apt complains about corrupted index files even after multiple updates

[Stefan Bühler <stbuehler@web.de>]

Hi,

On Sat, 30 Jan 2016 15:03:27 +0100
Julian Andres Klode <jak@debian.org> wrote:

> On 30 January 2016 at 13:42, Stefan Bühler <stbuehler@web.de> wrote:
> > [...]
> > ==890== Conditional jump or move depends on uninitialised value(s)
> > ==890==    at 0x4F4D8A0: pkgCache::ReMap(bool const&)
> > (in /usr/lib/x86_64-linux-gnu/libapt-pkg.so.5.0.0)> >
> > [...]
> > I managed to track down the code (pkgCache::ReMap+0x2d0) in gdbs
> > disassembly to pkgcache.cc:199:
> >   if (hash != HeaderP->CacheFileSize)  
> 
> That's not an issue, it's expected. We mmap() the memory, and then
> hash the entire memory we mapped, as that's what we are writing to the
> disk. It does not matter that there are uninitialized values in there,
> we just want to check that if we have written it to a file and read it
> back, that it's the same.

That is sad; I can see now that it should work, but it would be nice if
this could be fixed one day.

> > Disabling lz4 seems to fix the issue (but not the valgrind report):
> >
> >   echo 'APT::Compressor::lz4::Cost "5000";'
> > > /etc/apt/apt.conf.d/disable-lz4 rm /var/lib/apt/lists/*.lz4
> >   apt-get update  
> 
> OK. Not sure how that happens.

I might add that the docker/debootstrap combo sets (among others) the
following options by default:

    Acquire::GzipIndexes "true";
    Acquire::CompressionTypes::Order:: "gz";

I think in fileutl.cc Lz4FileFdPrivate::InternalOpen() needs to reset
next_to_load = APT_BUFFER_SIZE, as otherwise a Seek() after reaching
end of file won't read anything anymore due to next_to_load == 0.

Would this bug explain the issue we are seeing here?

Kind regards,
Stefan