[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: https://www.debian.org/mirror/list debian.inode.at 403 Forbidden

On Wed, 2021-03-17 at 10:43 -0400, Calum McConnell wrote:

> there are many different places where it is stated that you should
> select a local mirror rather than the CDN, and a large part of the
> community still holds that belief.

Do you have some examples of these places or people?

I expect that most of the docs were simply written before the CDN
existed and that Debian hasn't fully communicated with the community
about using the CDN rather than the mirrors.

> Is that confirmed?

Depends who you ask, certainly the DSA folks who set up use of the CDN,
shut down the mirror redirector (httpredir), switched security.d.o and
ftp.d.o to the CDN, made d-i default to the CDN and demoted the mirror
choice question in d-i definitely prefer that folks use the CDN.

Right now I personally think that on balance the CDN is the best option
for many people. People in specific circumstances probably should
instead use local mirrors or the mirrors on the Tor onion service.

There is one downside common to both CDNs and mirrors:

The debdeltas aren't on a CDN nor mirrored anywhere, so the bandwidth
used for updating on bandwidth-constrained devices is potentially
expensive international traffic instead of local traffic.

There some downsides of CDNs though:

There are parts of the world that our CDN provider (Fastly) isn't great
in, doesn't have coverage in or could get banned in.

There are networks where local traffic is gratis but external traffic
costs money (such as cloud providers) or counts to quota and the CDN
provider is unlikely to have nodes within all of these networks.

CDN providers are an opportunity for MITM attacks, the design of the
Debian repo format mitigates most of these attacks though. The same
attacks of course apply to each mirror but are limited to its users.

Each CDN node does not contain a full copy of the apt repository, so
there are fewer full copies of Debian out there.

The benefit provided by https is lower for the CDN than the mirror,
since the range of file sizes being transferred is lower to the CDN
domain (which is just Debian) than to mirrors (which have more files).

There are some downsides of the mirror network too:

Having to choose a specific mirror is not something many computer users
want to be doing, so requiring a choice rather than just making one
global worldwide mirror (aka CDN) the default means the audience for
Debian is restricted to geeks, power users and folks who know them or
can hire them.

The hosting trend worldwide is towards CDNs rather than mirrors, so the
amount of mirrors is likely to go down over time.

The popularity of Debian derivatives means that Debian gets mirrored
less and in some cases ISPs make the choice to drop Debian mirrors but
keep the derivatives.

Lots of mirrors are very suboptimal, they stop updating, or do updates
in the wrong way leading to errors on user systems or just silently
leave user systems without updates.

We have no mirror redirector nor apt mirror:// provider. Consequently
the user experience when a mirror is created or goes bad isn't great,
as the submitter of this thread found, they need to be vigilant about
finding new local mirrors, or need to manually update after errors.

Since mirrors come and go, the copy of the mirror list within d-i will
always be outdated to some extent. I'm not sure if d-i auto-updates the
mirror list when it gets network though?

The ftp.CC.debian.org country-specific mirror aliases have to be
manually updated by the DSA folks rather than being automatically
updated from the mirror monitoring service.

The new mirror additions have to be manually added rather than being
automatically added to the mirrors list after a successful monitoring

The mirrors that die or stop being updated have to be manually removed
from the mirrors list rather than automatically being removed.

There is no automatic mechanism for mirror operators to keep their
mirror online and updated, but indicate that it should stop being used
due to maintenance or end of life shutdown.

The mirror sync infrastructure is extra hardware that the Debian
project needs to buy (since we no longer have server vendors giving us
gratis hardware like HP/HPE used to) and that DSA need to find hosting
for and maintain and that the mirror team (which is just DSA right now)
need to setup and maintain syncing on.

The support for https within the mirror network is spotty and the
mirror monitoring infrastructure doesn't check for https support. OTOH
https only provides minimal obscurity for traffic, since all the file
sizes are known, a passive observer could correlate request size with
individual packages, especially for larger ones. For mirrors there is a
wider distribution of sizes due to mirrors of other distros on the same
domain names as the Debian mirrors, so it is harder to correlate.

PS: the mirror monitoring service:


PS: personally I just use the CDN everywhere, since I don't have to
make a choice, I don't have a bandwidth quota, my ISP's Debian mirror
was always terrible, my ISP dropped its Debian mirror and some of my
devices move between networks.



Attachment: signature.asc
Description: This is a digitally signed message part

Reply to: