Re: update anonftpsync
On Thu, Feb 28, 2008 at 11:02:20PM -0500, Justin Pryzby wrote:
> > ..how is then --del different from --delete-after when updating
> > a Debian mirror, other than that one big ass batch at the end?
> I think I was wrong about --delete-after; it really is needed.
> Otherwise the *old* Packages.gz file will be referencing nonextant
> .debs, which is the same situation that's trying to be avoided for the
> new Packages.gz by running rsync twice.
Exactly, there's a race condition window there. With users who are on a slow
link (e.g. modem, ISDN, slow wireless/DSL), they might start downloading the
old Packages file, churn at it for five or ten minutes, during which time
you can also remove the file from disk (the rsync can renew everything), but
Apache will keep serving it to them until their connection is done.
After that, they unknowingly run apt-get upgrade which tries to get the
previous revisions of packages, yet they get a bunch of 404s, because the
server's rsync process is deleting the files. Then they run apt-get update
again, and find no new changes (because the server isn't done deleting files
yet), which leaves them puzzled.
If we keep the old data files in place until the point where all indices are
updated, we don't really help much in the way of eliminating the race
condition, but we shorten its time and we eliminate the user confusion
(because we're sure that their subsequent apt-get update will get the new
indices).
(Yet, my example isn't necessarily exactly applicable every time, because dak
is known to have a setting which leaves the last few old .deb revisions in
the pool even if they are obsolete. However, this can't be relied upon,
so we still suggest some amount of leniency in the mirroring as well.
If one's mirror server is low on free space, obviously they can't follow
our suggestion, but it's still a decent default.)
--
Josip Rodin
mirrors@debian.org
Reply to: