Re: update anonftpsync
On Sat, Mar 01, 2008 at 09:37:43AM +0100, Arnt Karlsen wrote:
> On Fri, 29 Feb 2008 13:57:48 +0100, Josip wrote in message
> <20080229125747.GA25026@keid.carnet.hr>:
>
> > On Thu, Feb 28, 2008 at 11:02:20PM -0500, Justin Pryzby wrote:
> > > > ..how is then --del different from --delete-after when updating
> > > > a Debian mirror, other than that one big ass batch at the end?
> > > I think I was wrong about --delete-after; it really is needed.
> > > Otherwise the *old* Packages.gz file will be referencing nonextant
> > > .debs, which is the same situation that's trying to be avoided for
> > > the new Packages.gz by running rsync twice.
> >
> > Exactly, there's a race condition window there. With users who are on
> > a slow link (e.g. modem, ISDN, slow wireless/DSL), they might start
> > downloading the old Packages file, churn at it for five or ten
> > minutes, during which time you can also remove the file from disk
> > (the rsync can renew everything), but Apache will keep serving it to
> > them until their connection is done.
> >
> > After that, they unknowingly run apt-get upgrade which tries to get
> > the previous revisions of packages, yet they get a bunch of 404s,
> > because the server's rsync process is deleting the files. Then they
> > run apt-get update again, and find no new changes (because the server
> > isn't done deleting files yet), which leaves them puzzled.
> >
> > If we keep the old data files in place until the point where all
> > indices are updated, we don't really help much in the way of
> > eliminating the race condition, but we shorten its time and we
> > eliminate the user confusion (because we're sure that their
> > subsequent apt-get update will get the new indices).
> >
> > (Yet, my example isn't necessarily exactly applicable every time,
> > because dak is known to have a setting which leaves the last few
> > old .deb revisions in the pool even if they are obsolete. However,
> > this can't be relied upon, so we still suggest some amount of
> > leniency in the mirroring as well. If one's mirror server is low on
> > free space, obviously they can't follow our suggestion, but it's
> > still a decent default.)
>
> ..so, the way to do this, is run --delete-during on the pool/ tree
That's not good enough since the *old* Packages file will then reference
debs that have now been removed. When /pool debs are synced, no --del
options can be used. Removing anything at that stage will leave the
possibility of unavailability of some referenced debs.
> and --delete-after on everything else, to minimize mirror disk usage?
--delete-after is just an rsync level way of doing a 3-pass mirror:
. Send /pool debs;
. Send Packages.gz, Release and everything others, with
--delay-updates;
. Remove files on the client that don't exist on the server.
Justin
Reply to: