Re: where to sync from?
So, ftp.br.d.o syncs with ftp.de.d.o. Is it being pushed? Would
ftp.br.d.o be willing to push ftp.cl.d.o ?
I would like to strive for an open debate on how mirrors sync,
particularly since we, being rather far away, distance- and
network-wise, are in a weak link situation.
Coming back to my previous message, it seems to me that uniting mirror
admins is a way to go, and that the Debian mirror and submission pages
could help in bringing mirrors together, am I right? Maybe it's not
technically possible?
Ricardo Yanez
On Wed, 2007-03-21 at 18:07 +0100, Josip Rodin wrote:
> On Wed, Mar 21, 2007 at 12:07:09PM -0400, Ricardo Yanez wrote:
> > Many years ago, before 2002, we were syncing with ftp.br.debian.org,
> > then one day, the entire debian/ tree was gone with the catastrophic
> > result that our entire tree was deleted as well. It took us weeks to
> > re-build the tree again (yeah, that slow! and debian in those days was a
> > 10th of what is today). Based on this painful experience I've been very
> > reluctant to use leaf mirrors to sync.
>
> ftp.br.d.o had its share of problems in the past, yes, but that was probably
> when it was hosted elsewhere. The alias changed hands a few times by now.
>
> About that particular problem - please use the option --max-delete for
> rsync. Set it to something like 50000, that should be safe enough.
>
> > I mean, ftp.cl.debian.org is a leaf mirror, low in the hierarchy, and
> > should stay there based on it's not-so-hot "stableness". I see
> > ftp.br.debian.org is also a leaf mirror, which I interpret as not being so
> > high in the hierarchy either. Maybe I'm wrong about my interpretation of
> > Leaf, Secondary and Primary mirrors, but these categories must somehow
> > reflect how committed the sponsor and admins are to the mirror, right?
>
> In this particular case, we have a bit of an inconsistency. By assigning the
> ftp.*.d.o alias, we at Debian want to recognize the quality of a mirror and
> relay this information to all the users, too.
>
> The mirrors in the western hemisphere are mostly dependent on the Northern
> American mirrors, mainly because ftp-master.debian.org is in the US and the
> layout of the Internet links in general works that way. It would be logical
> for us Debianites to work with mirror admins in both North and South America
> in order to provide the best service; for example, we should organize the
> first tier of push-mirrors and provide assistance with the second tier;
> allowing more mirrors to get pushed so that they can update more
> efficiently.
>
> Yet, we have this bizarre situation where the North American mirrors
> have been continuously either neglected or struggling due to external
> influences, despite the fact that they are actually the closest ones to
> ftp-master.debian.org.
>
> We have the ftp.debian.org which is constantly overloaded, but graciously
> hosted at a university which won't kick us out (at least not yet). It is
> maxing out its 100Mbit connection virtually all the time[1]. We have
> ftp.us.debian.org as a round-robin of three, currently - one is ftp.d.o,
> the other is another mirror at a university, and the third is
> mirrors.kernel.org, hosted by ISC. All of those are generally under a high
> load, because the US users aren't aided in order to spread the load. The
> Canadians either; we used to have a separate ftp.ca.debian.org, but once
> that site went offline, nobody worked to seek a replacement, and instead
> all that load was shifted onto ftp.debian.org. It's been like that for years
> now.
>
> The selection of mirrors in the round-robin is controlled by a few Debian
> admins who aren't particularly interested in improving it, and the selection
> has only shrunk in the last few years (once upon a time, we had six machines
> in it; now we only have three). I have tried to contribute to fixing this
> problem, but was told to butt out, mostly because as a European I didn't
> have enough information about the US matters; I then on a separate occasion
> organized a syncproxy.wna.d.o on the west coast, but that's as far as that
> went (that was needed to fix up the Australian mirrors). On a few occasions
> I contributed a few bits and pieces (such as asking elmo to up
> ftp-master.d.o tcp_wmem so that it does cross-continental syncs faster), but
> the main problem remained unsolved. I later went AWOL and nobody seemed to
> pick up the torch after me. Now I'm a bit more active again, and I see the
> status quo remains; and also I lost my (unprivileged) account on
> ftp-master.d.o so I can't even track the log files over there any more,
> and my requests to have it reinstated are ignored for months.
>
> The secondary mirrors in the US which aren't part of the round-robin alias
> are mostly not pushed, and I guess most people just don't use them. These
> mirrors end up servicing a local community, which was usually the reason
> they were set up; as well as the few people who actually go through the huge
> list and pick a mirror from it.
>
> So, the logical thing to do here would be to get your two mirrors, which
> are the official mirrors for two large countries, pushed from either
> ftp-master.debian.org or from one of the ftp.us.debian.org sites.
> But, I don't have any idea how to do that, because a) nobody cares to do
> these things b) the sites are mostly overloaded anyway so they would
> probably be slow to sync from, unless some special arrangements were made.
>
> I guess that now that I spent time writing this mail, I could have another
> go at trying to find out which debian-admin member now tends after these
> things, and see if anything can be changed.
>
> > In our case, the environment is somewhat hostile, perhaps unforgiven about
> > the needs of a server. For example, over some weekends, people saw the
> > mirror on and turned it off thinking someone must have forgotten it on,
> > because that's what people do with their windows desktops. Today the
> > server is locked up, which only reduces the frequency of these events. Or,
> > someone unplugs a switch to plug an electric kettle. Even more common,
> > power outages lasting days, particularly during weekends when some people
> > think switching off the entire building at the main fuse box helps
> > conserving energy and money. That's the kind of things we have to struggle
> > against, not to mention a Computing Department that seems to be playing
> > around with a filter.
>
> I trust you plan to get it into a proper server room in the future :)
> Many of us have experienced similar problems in the past; I sympathise :/
>
> --
> 2. That which causes joy or happiness.
>
> [1] http://saens.debian.org/mrtg/saens.eth0.html
>
>
Reply to: