Bug#698477: Do we really need mirror in AWS?
On 21/07/16 at 09:45 +0900, Charles Plessy wrote:
> Le Wed, Jul 20, 2016 at 03:48:23PM +0100, Marcin Kulisz a écrit :
> >
> > do you think that there is still need for mirror on AWS once we have CloudFront
> > CDN which is working quite nicely from within AWS?
>
> Hi Marcin,
>
> I think that http(s)://cloudfront.debian.net/ is exactly what we need.
>
> And I am not recommending to add it to the list of official geographic mirrors,
> because it is not a geographic service.
>
> Providers of geographic mirrors know that they will never bear the full cost of
> the whole Debian users downloading packages, given that - obvisouly - users at
> a far distance from their mirror will use a different one. But CloudFront or
> Azure (if open to the outside; I do not remember) are available worldwide. If
> presented together with the geographic mirrors, and in absence of the official
> Debian CDN that is to come but is not ready for prime time yet, then there is a
> risk that through blogs, forums, mail lists, magazine articles etc, one of
> these cloud-based mirrors start to become over-popular and attract a lot of
> outside traffic, just because it works well from any geographic location. In
> that situation we should be prepared to be told that the provided never
> intended to pay for so much non-cloud traffic, and shuts down the service or
> asks for financial contribution. For that reason, I think that we should
> refrain from presenting these mirrors in a similar context as the geographical
> official ones.
>
> Of course, if there are good plans to have cloudfront.debian.net served from
> "debian.org" instead, there would be no reason to refrain from doing so.
FWIW, I asked on #debian-admin about cloudfront.d.n and deb.debian.org (which
is the new CDN setup from DSA, using Fastly at the moment -- see video from
DC16 talk):
09:25 <@Mithrandir> lucas: it's not suitable for deb.d.o in its current configuration.
09:25 <@Mithrandir> as I said whenever somebody last asked about it.
09:25 < lucas> is this documented somewhere? if it's not suitable for deb.debian.org, it might not be
suitable for the official EC2 images Debian provides
09:25 < lucas> but it's used there
09:27 <@Mithrandir> > curl -s -i http://cloudfront.debian.net/debian-security/ | head -n1
09:27 <@Mithrandir> HTTP/1.1 404 Not Found
09:28 <@Mithrandir> > curl -s -i http://cloudfront.debian.net/debian-debug/ | head -n1
09:28 <@Mithrandir> HTTP/1.1 404 Not Found
09:28 <@Mithrandir> > curl -s -i http://cloudfront.debian.net/debian-ports/ | head -n1
09:28 <@Mithrandir> HTTP/1.1 404 Not Found
09:28 <@Mithrandir> so, well, it's missing 3/4 of the top level "directories" that's supported by deb.d.o
09:28 <@Mithrandir> it's not documented outside of the configuration for deb.d.o atm, no.
09:30 < lucas> ok
09:30 <@Mithrandir> (which is in git)
09:34 <@pabs> uh, why is debian-security supported by deb.d.o?
09:35 <@Mithrandir> why shouldn't it be?
09:35 <@pabs> I thought we discourage use of it via anything other than security.d.o
09:35 <@Mithrandir> it's a one-stop shop for all things .deb (as distributed by Debian)
09:36 <@Mithrandir> we discourage random mirrors, which is slightly different.
09:39 <@weasel> pabs: deb.d.o is not a mirror.
09:40 <@Mithrandir> I wouldn't have a problem with folks using -security through cloudfront.d.n either,
fwiw, but static mirrors are very different.
09:44 < lucas> is DSA interested in onboarding cloudfront.d.n as part of deb.d.o, actually?
09:44 <@weasel> I would welcome a second backend
09:46 <@Mithrandir> I'd be fine with it, as long as it's sanely configurable and it has the bits we want.
09:48 < lucas> ok, unless you tell me not to, I'll quote this IRC on debian-cloud@, so there's a trace of it
09:51 <@Mithrandir> I'm fine with my bits being quoted, but if people want DSA input, they should Cc
debian-admin@lists.d.o
09:52 <@weasel> and we should at some point extract an overview of the config from our fastly settings.
09:52 <@weasel> but I agree, -cloud is not the place for this discussion for deb.d.o purposes
09:56 <@Mithrandir> I kinda feel like cloudfront should then be under DSA control so we can update its
config if we add more bits to deb.d.o, but I'd be happy to have a discussion about how
best to solve that.
Lucas
Reply to: