Bug#737085: apt: Apt downloads arch all packages from wrong repo/checks wrong checksum
+++ Julian Andres Klode [2014-01-30 08:12 +0100]:
> On Thu, Jan 30, 2014 at 03:13:16AM +0000, Wookey wrote:
> > Package: apt
> > Version: 0.9.15
> > Severity: important
> >
> > In the sources I have my own bootstrap repository containing a lot of
> > (unstable) packages built for arm64, and plain debian unstable and saucy repos
> >
> > apt-get install <arch-all-package> (that is available in all 3 repos)
> > results in a size mismatch error. It seems that apt is using the
> > checksum from one repo but downloading the package from another.
> >
> > The packages used is just an example it seems to be the same for any arch all package
> >
> > (debian-arm64)# apt-cache policy x11proto-scrnsaver-dev
> > x11proto-scrnsaver-dev:
> > Installed: (none)
> > Candidate: 1.2.2-1
> > Version table:
> > 1.2.2-1 0
> > 500 http://people.debian.org/~wookey/bootstrap/debianrepo2/ debianstrap/main arm64 Packages
> > 500 http://ftp.uk.debian.org/debian/ unstable/main amd64 Packages
> > 500 http://ports.ubuntu.com/ubuntu-ports/ saucy/main arm64 Packages
> >
>
> Right, and that's a problem, as having two different packages with the
> same version is not really supported. APT differentiates packages with the
> same version by CRC-16 hashing the fields
> Installed-Size
> Depends
> Pre-Depends
> Conflicts
> Breaks
> Replaces
> in order to handle packages where those are the same APT would need to hash
> size or SHA hash as well, but this fails for installed packages, as this
> information is not provided in /var/lib/dpkg/status.
OK. That makes sense. I see what's going on now.
Which of course if why we do -B builds for other architectures and
carefully ensure there is only one copy of the arch all packages.
The problem is that in order to debootstrap you need all the packages in
one repo so leaving the arch all packages in ftp.uk.debian.org means you
can't debootstrap if you only uploaded the new-arch 'any' packages to
the 'bootstrap' repo. It's also important to test that the arch-all
build actually works, and not just the arch-any part so doing those
builds and testing the results can be good.
It's fine for apt to consider these packages to be functionally
equivalent, but it does need to check the correct checksum on download.
It seems to me that this can be fixed by either adding size/hash to the
hash as you suggest(making them 'different packages', or just separately
ensuring that the checksum for the repo/file that was downloaded is
used. Apt knows that there is more than one repo source for this
package, but doesn't record that there might be more than one checksum?
The fact that it can end up choosing one checksum and another source
does seem wrong. Perhaps the code/object structure makes it hard to fix
this this way and your fix is the only one that makes sense?
> An alternative would be to change the cache-building algorithms to look
> at SHA hashes and/or size and create different version entries in the cache
> if they are present in both versions, but different. SHA Hashes would require
> all repositories to use the same best checksum algorithm.
I think just adding size to the hash would be cheap and easy and would
largely solve this problem. Adding the hash would cover a few extra
cases where the size came out the same too, but if it's difficult I'd be
happy to have this mostly-solved, as it's a situation we normally try to
avoid anyway.
I am clueless about the apt codebase (and C++ if it's not fairly 'C'-ey)
but am prepared to take a stab at this if you give me a clue where to
look.
thanks for the quick response.
Wookey
--
Principal hats: Linaro, Emdebian, Wookware, Balloonboard, ARM
http://wookware.org/
Reply to: