corrupt package index, backing up previous indexes automatically as to have smaller pdiffs.
Hi all,
Disclaimer 0 - This is going to be a long one, sorry again.
Disclaimer 1 - Wherever I have mentioned or used the word "you" , it
doesn't mean any person in particular, I simply mean a person who
doesn't have good bandwidth with her/im and some of the problems s/he
may face which I have also faced and am looking for solutions for
this.
And while people have suggested me that getting a fatter connection
would solve the problems and personally I could do that there are many
(in India and elsewhere) where they may not be able to either due to
financial costs or monopoly ISP's or both or any other number of
reasons. This is more of voice of the voiceless and what could be done
about it and Debian being a Universal OS should probably care about
them as well.
This has happened to me a few times and continues to happen now and
then is sometimes the package index becomes corrupted when you are
updating it. Some of the multiple reasons I have found so far are -
a. Sometimes the mirror stops responding in the middle of the update
and apt doesn't know other mirrors to use. But this is hit and miss,
sometimes it is able to get to the other mirror/s , sometimes not.
Dunno why this happens. I use the re-director as it is/was claimed to
be better and most times it is a better experience, but sometimes the
above happens.
b. Sometimes the bandwidth given is not enough and the package index
gets corrupted.
c. Sometimes, electricity goes out and if the system is not on a UPS
and you are in middle of a package update it gets corrupted.
d. At times, if you have a big sources.list even then sometimes the
package index gets corrupted.
There may be some more reasons why the package index gets corrupted
but I can't remember now atm.
Now for some reason apt doesn't know how to deal with corrupted
package index and that is what also JAK also shared in his
presentation when I asked him the question, or at least I believe that
is what he shared. I would be happy to be corrected if that is not the
case.
Apart from that, the only solution that we have is to rm all the
entries from /var/lib/apt/lists and then start from scratch again.
Now here even if you have a small sources.list it is going to test
your patience quite a bit and there is no guarantee that the package
index will be correctly updated. While sometimes it is able to sort of
jump over one of the index entries, recover and have the index updated
and you know that you need to just do the update once more and that
single entry would come in and you will get the nice "x number of
packages to be updated" or just a message which doesn't have any
error if no packages have been updated which is what you are looking
for the whole time.
If not, then you are out of luck and you have to try it a few times
till the time you succeed. Sometimes if you are out of luck and either
the mirror or the route in-between or any of the gazillion things
in-between (all the hops, packets etc.) do not work and the only
option is to go do some other work, play, sleep etc. and try doing
that in a few hours from whenever you tried last time.
Now the good apt and brethen maintainers did a great thing, just like
for package delta, they also used the same delta idea for package
index as well. So, if you are lucky and everything works you get a
small delta, say hardly 10-12 MB (or more or less depending on how
your sources.list is tailored and whether the sources you are
downloading from also makes delta of the package index) and you are
good to go.
Now the only solution to my mind to this complex issue (with many
moving parts) is to have a back-up of a successful package index
updated. But doing it manually is/can be a pain and as any manual
process is, it is and would be error-prone. Could something be done
about it ? While people could certainly write a script about it, but
could something be done, some function added and in the .conf file a
toggle of yes/no could have which enables this happening.
I don't know if there are or even is possible to find out how many
people and how much tries and time is used by people trying to get
their index updated, are not successful and they have to try again. If
we could get some stats about non-success or multiple tries than
probably it might be possible to try and figure out how we should go
about it.
Comments probably but apt maintainers but AFAIK it's the same with all
the package maintainer front-ends as the logic and methodology is more
or less the same/similar.
Looking forward to know, learn and hopefully share :) . Sorry again
for the longish tone.
--
Regards,
Shirish Agarwal शिरीष अग्रवाल
My quotes in this email licensed under CC 3.0
http://creativecommons.org/licenses/by-nc/3.0/
http://flossexperiences.wordpress.com
EB80 462B 08E1 A0DE A73A 2C2F 9F3D C7A4 E1C4 D2D8
Reply to: