[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Bug#346138: marked as done (apt: Weird stuff happening with cached index files)

Your message dated Thu, 13 Aug 2015 14:39:41 +0200
with message-id <20150813123941.GA16463@crossbow>
and subject line Re: Bug#346138: state of #346138
has caused the Debian Bug report #346138,
regarding apt: Weird stuff happening with cached index files
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact owner@bugs.debian.org
immediately.)


-- 
346138: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=346138
Debian Bug Tracking System
Contact owner@bugs.debian.org with problems
--- Begin Message --- 
Package: apt
Version: 0.6.43
Severity: normal


Apologies in advance that this is a bit woolly.

Since the key debacle (though I dunno what the point is of using keys when
the package has so many hundreds of open bugs) I am having trouble with the
index files. Watch this :

piwakawaka:~# aptitude update
Reading package lists... Done
Building dependency tree... Done
Reading extended state information
Initializing package states... Done
Building tag database... Done
Hit ftp://debian.paradise.net.nz sid Release.gpg
Get:1 ftp://debian.paradise.net.nz sid Release [42.0kB]
Get:2 http://http.us.debian.org sid Release.gpg [378B]
Hit http://http.us.debian.org sid Release
Hit http://http.us.debian.org sid/main Packages
Hit http://http.us.debian.org sid/contrib Packages
Hit http://http.us.debian.org sid/non-free Packages
Hit http://http.us.debian.org sid/main Sources
Hit http://http.us.debian.org sid/contrib Sources
Hit http://http.us.debian.org sid/non-free Sources
Hit ftp://debian.paradise.net.nz sid/main Packages
Hit ftp://debian.paradise.net.nz sid/non-free Packages
Hit ftp://debian.paradise.net.nz sid/contrib Packages
Fetched 42.4kB in 2s (16.9kB/s)
Reading package lists... Done
piwakawaka:~# aptitude update

So far so good, but I immediately repeated the update and this happened :

piwakawaka:~# aptitude update
Reading package lists... Done
Building dependency tree... Done
Reading extended state information
Initializing package states... Done
Building tag database... Done
Hit ftp://debian.paradise.net.nz sid Release.gpg
Get:1 ftp://debian.paradise.net.nz sid Release [42.0kB]
Ign http://http.us.debian.org sid Release.gpg
Ign http://http.us.debian.org sid Release
Ign http://http.us.debian.org sid/main Packages
Hit ftp://debian.paradise.net.nz sid/main Packages
Hit ftp://debian.paradise.net.nz sid/non-free Packages
Ign http://http.us.debian.org sid/contrib Packages
Get:2 http://http.us.debian.org sid/non-free Packages [52.1kB]
Hit ftp://debian.paradise.net.nz sid/contrib Packages
Get:3 http://http.us.debian.org sid/main Sources [60.5kB]
Get:4 http://http.us.debian.org sid/contrib Sources [1107kB]
Get:5 http://http.us.debian.org sid/non-free Sources [23.0kB]
Get:6 http://http.us.debian.org sid/main Packages [24.6kB]
Hit http://http.us.debian.org sid/contrib Packages
99% [6 Packages gzip 0] [4 Sources bzip2 3592192]
29.0kB/s 0s
gzip: stdin: not in gzip format
Err http://http.us.debian.org sid/main Packages
  Sub-process gzip returned an error code (1)
Fetched 1309kB in 49s (26.3kB/s)
Reading package lists... Done
piwakawaka:~#

Which is weird. Yesterday when the same thing happened I deleted the saved
files from /var/lib/apt/lists and /var/lib/apt/lists/partial/ (not including
the lock file or the partial directory entry) which made a fine *WORKAROUND*
so lets have a look at what's there now :

piwakawaka:/var/lib/apt# ls -al lists
total 37440
drwxr-xr-x 3 root root     4096 Jan  6 11:08 .
drwxr-xr-x 5 root root     4096 Jan  5 15:44 ..
-rw-r--r-- 1 root root    42010 Dec 28 10:02
debian.paradise.net.nz_debian_dists_sid_Release
-rw-r--r-- 1 root root      189 Dec 28 10:02
debian.paradise.net.nz_debian_dists_sid_Release.gpg
-rw-r--r-- 1 root root   221131 Dec 28 09:17
debian.paradise.net.nz_debian_dists_sid_contrib_binary-i386_Packages
-rw-r--r-- 1 root root 15726115 Dec 28 09:17
debian.paradise.net.nz_debian_dists_sid_main_binary-i386_Packages
-rw-r--r-- 1 root root   259513 Dec 25 09:16
debian.paradise.net.nz_debian_dists_sid_non-free_binary-i386_Packages
-rw-r--r-- 1 root root    38234 Jan  5 10:11
http.us.debian.org_debian_dists_sid_Release
-rw-r--r-- 1 root root      378 Jan  5 10:11
http.us.debian.org_debian_dists_sid_Release.gpg
-rw-r--r-- 1 root root   221235 Jan  5 09:16
http.us.debian.org_debian_dists_sid_contrib_binary-i386_Packages
-rw-r--r-- 1 root root    95554 Jan  5 09:42
http.us.debian.org_debian_dists_sid_contrib_source_Sources
-rw-r--r-- 1 root root 15829945 Jan  5 09:16
http.us.debian.org_debian_dists_sid_main_binary-i386_Packages
-rw-r--r-- 1 root root  5418824 Jan  5 09:42
http.us.debian.org_debian_dists_sid_main_source_Sources
-rw-r--r-- 1 root root   269480 Jan  5 09:16
http.us.debian.org_debian_dists_sid_non-free_binary-i386_Packages
-rw-r--r-- 1 root root    95554 Jan  5 09:42
http.us.debian.org_debian_dists_sid_non-free_source_Sources
-rw-r----- 1 root root        0 Jan  6 11:08 lock
drwxr-xr-x 2 root root     4096 Jan  6 11:13 partial
piwakawaka:/var/lib/apt# ls -al lists/partial/
total 3872
drwxr-xr-x 2 root root    4096 Jan  6 11:13 .
drwxr-xr-x 3 root root    4096 Jan  6 11:08 ..
-rw-r--r-- 1 root root 3951594 Jan  5 09:16
http.us.debian.org_debian_dists_sid_main_binary-i386_Packages
piwakawaka:/var/lib/apt#

Note that http.us.debian.org_debian_dists_sid_main_binary-i386_Packages was
downloaded during neither of the updates above, but that the second update
has crapped out on it.

So I remove the cached list files and update again

piwakawaka:/var/lib/apt# rm -f lists/d*
piwakawaka:/var/lib/apt# rm -f lists/h*
piwakawaka:/var/lib/apt# ls -al lists
total 12
drwxr-xr-x 3 root root 4096 Jan  6 11:34 .
drwxr-xr-x 5 root root 4096 Jan  5 15:44 ..
-rw-r----- 1 root root    0 Jan  6 11:08 lock
drwxr-xr-x 2 root root 4096 Jan  6 11:33 partial
piwakawaka:/var/lib/apt# rm lists/partial/*
rm: cannot lstat ists/partial/*': No such file or directory
piwakawaka:/var/lib/apt#

piwakawaka:~# l /var/lib/apt/lists/
total 37436
-rw-r--r-- 1 root root    42010 Dec 28 10:02
debian.paradise.net.nz_debian_dists_sid_Release
-rw-r--r-- 1 root root      189 Dec 28 10:02
debian.paradise.net.nz_debian_dists_sid_Release.gpg
-rw-r--r-- 1 root root   221131 Dec 28 09:17
debian.paradise.net.nz_debian_dists_sid_contrib_binary-i386_Packages
-rw-r--r-- 1 root root 15726115 Dec 28 09:17
debian.paradise.net.nz_debian_dists_sid_main_binary-i386_Packages
-rw-r--r-- 1 root root   259513 Dec 25 09:16
debian.paradise.net.nz_debian_dists_sid_non-free_binary-i386_Packages
-rw-r--r-- 1 root root    38234 Jan  5 10:11
http.us.debian.org_debian_dists_sid_Release
-rw-r--r-- 1 root root      378 Jan  5 10:11
http.us.debian.org_debian_dists_sid_Release.gpg
-rw-r--r-- 1 root root   221235 Jan  5 09:16
http.us.debian.org_debian_dists_sid_contrib_binary-i386_Packages
-rw-r--r-- 1 root root    95554 Jan  5 09:42
http.us.debian.org_debian_dists_sid_contrib_source_Sources
-rw-r--r-- 1 root root 15829945 Jan  5 09:16
http.us.debian.org_debian_dists_sid_main_binary-i386_Packages
-rw-r--r-- 1 root root  5418824 Jan  5 09:42
http.us.debian.org_debian_dists_sid_main_source_Sources
-rw-r--r-- 1 root root   269480 Jan  5 09:16
http.us.debian.org_debian_dists_sid_non-free_binary-i386_Packages
-rw-r--r-- 1 root root    99117 Jan  5 09:42
http.us.debian.org_debian_dists_sid_non-free_source_Sources
-rw-r----- 1 root root        0 Jan  6 11:35 lock
drwxr-xr-x 2 root root     4096 Jan  6 11:40 partial
piwakawaka:~# l /var/lib/apt/lists/partial/
total 0
piwakawaka:~#                                                                                       
 
Gotta love those hard to reproduce bugs. Perhaps it only happens with the
gzip file.

Drop me a line for any further information.

And as a bit of a side note, yesterday when it died in the same fashion it
appeared to have the main/Packages and contrib/Packages swapped during an
update prior to the problem above surfacing, but I can see no evidence of
the same problem today :

Get:6 http://http.us.debian.org sid/main Packages [24.6kB]
Get:7 http://http.us.debian.org sid/contrib Packages [3952kB]

-- Package-specific info:

-- apt-config dump --

APT "";
APT::Architecture "i386";
APT::Build-Essential "";
APT::Build-Essential:: "build-essential";
Dir "/";
Dir::State "var/lib/apt/";
Dir::State::lists "lists/";
Dir::State::cdroms "cdroms.list";
Dir::State::userstatus "status.user";
Dir::State::status "/var/lib/dpkg/status";
Dir::Cache "var/cache/apt/";
Dir::Cache::archives "archives/";
Dir::Cache::srcpkgcache "srcpkgcache.bin";
Dir::Cache::pkgcache "pkgcache.bin";
Dir::Etc "etc/apt/";
Dir::Etc::sourcelist "sources.list";
Dir::Etc::sourceparts "sources.list.d";
Dir::Etc::vendorlist "vendors.list";
Dir::Etc::vendorparts "vendors.list.d";
Dir::Etc::main "apt.conf";
Dir::Etc::parts "apt.conf.d";
Dir::Etc::preferences "preferences";
Dir::Bin "";
Dir::Bin::methods "/usr/lib/apt/methods";
Dir::Bin::dpkg "/usr/bin/dpkg";
DPkg "";
DPkg::Pre-Install-Pkgs "";
DPkg::Pre-Install-Pkgs:: "/usr/bin/apt-listchanges --apt || test $? -ne 10";
DPkg::Pre-Install-Pkgs:: "/usr/sbin/dpkg-preconfigure --apt || true";
DPkg::Tools "";
DPkg::Tools::Options "";
DPkg::Tools::Options::/usr/bin/apt-listchanges "";
DPkg::Tools::Options::/usr/bin/apt-listchanges::Version "2";
DPkg::Post-Invoke "";
DPkg::Post-Invoke:: "if [ -x /usr/bin/debsums ]; then /usr/bin/debsums --generate=nocheck -sp /var/cache/apt/archives; fi";
DPkg::Post-Invoke:: "test -f /var/run/med-common.usermenu && if [ -x /usr/sbin/cdd-update-usermenus ] ; then /usr/sbin/cdd-update-usermenus med ; fi ; rm -f /var/run/med-common.usermenu";

-- (no /etc/apt/preferences present) --


-- /etc/apt/sources.list --

# debian.paradise.net.nz
deb ftp://debian.paradise.net.nz/debian/ sid main non-free contrib

# main debian site
deb http://http.us.debian.org/debian sid main contrib non-free

# Source : Uncomment if you want the apt-get source function to work
deb-src http://http.us.debian.org/debian sid main contrib non-free

-- System Information:
Debian Release: testing/unstable
  APT prefers unstable
  APT policy: (500, 'unstable')
Architecture: i386 (i686)
Shell:  /bin/sh linked to /bin/bash
Kernel: Linux 2.6.11-1-k7
Locale: LANG=C, LC_CTYPE=C (charmap=ANSI_X3.4-1968)

Versions of packages apt depends on:
ii  libc6                         2.3.5-11   GNU C Library: Shared libraries an
ii  libgcc1                       1:4.0.2-5  GCC support library
ii  libstdc++6                    4.0.2-5    The GNU Standard C++ Library v3

apt recommends no packages.

-- no debconf information

--- End Message ---
--- Begin Message --- 
Version: 1.1~exp4

On Sun, Nov 16, 2008 at 10:55:34PM +0200, Eugene V. Lyubimkin wrote:
> Mark Robinson wrote:
> > I haven't seen exactly this effect for quite some time.
> > 
> > Every now and again I see a corruption of an index file causing bzip to
> > fail.
> Sorry, I am in some misunderstanding because of last phrase. Do you see this bug now, or
> this bug has gone and other appeared, or something else?

I am not to sure I understand either and after 7 years…

The initial bugreport at least mentions that removing the contents of lists/
helped, which sounds like a general bug we had to fight against multiple years
until we bit the bullet last year and started rewriting the acquire system,
which is what is currently in experimental and will be apt 1.1 heading for
stretch.

I am therefore optimistically close this bug as fixed with this version.
Eventually it was fixed previously with smaller reworks, too…


> > I suspect that this is either dodgy mirrors or a dodgy proxy operated by
> > my isp.
> > 
> > Further I think this is related to the sporadic and reasonably common
> > gpg errors which I see on the package lists and the occasional MD5 error.
> > 
> > I wonder if these errors could be trapped and the request them be
> > repeated with the appropriate options to force the proxy to fetch the
> > original file from the mirror again.
> This would require some work... Personally, I would not play with apt code to fix
> 3rd-party bugs though.

Also, our methods try to 'trick' proxies into behave nicely nowadays, but that
works only so far…


Best regards

David Kalnischkies

Attachment: signature.asc
Description: Digital signature


--- End Message ---
Reply to: