Re: Planning for a mirror using Google Cloud CDN
On 2016-12-01 17:37:47 +0100 (+0100), Bastian Blank wrote:
[...]
> If we use two-stage sync only within one regions, different regions may
> finish updating at different times. The exact difference depends on the
> network speed, but I assume the would be at most 15 minutes apart.
>
> With two-stage sync throughout the whole network we can get down to
> maybe one minute. Not much below, as we may want to be able to cache
> them at least a little while in the CDN.
[...]
I've never maintained official Debian mirrors so this might be a
terrible suggestion, but a project I work on operates a massive
CI/testing infrastructure spread across donations from many
different service providers and so we run our own package mirrors
for consistency and performance reasons. To solve the timing
variability between endpoints in our "CDN" design, we keep these
mirrors on an OpenAFS server cluster and `vos release` between
update pulses to get atomic cutover of mirror contents across our
entire distributed network of mirror servers (which are just httpds
in front of AFS client caches). This also presents us with an
opportunity to run consistency checks against the future state of
the mirror and avoid a cutover if it seems to be in a "broken"
state.
No idea if AFS would be an option in your case, but it's working
well for us at a pretty high client volume and update frequency.
--
Jeremy Stanley
Reply to: