[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: HTTPS metadata in Mirrors.masterlist?


sorry for digging up that old thread from 2014, but it's exactly what
I wanted to bring up, just with today's needs and possibilities:

* CVE-2016-1252 in APT showed that HTTPS might still bring additional
  security. After that issue, the amount of people asking for
  HTTPS-secured Debian mirrors noticeably raised, see e.g. the edits
  and comments from 2016 to answers to the 2013 question

* With Let's Encrypt it's much easier for mirror admins to get SSL
  certificates for ftp.$COUNTRY.debian.org and their local mirror
  hostname without having to fall back to using SNI or not providing
  SSL or at least not proper SSL certificates on all hostnames. (The
  latter still seems rather common, see below.)

Joerg Jaspert wrote:
> On 13484 March 1977, Colin Watson wrote:
> >> Would it be possible, then, to add "Archive-https: /debian/" to the
> >> "Site: mirrors.kernel.org" stanza in Mirrors.masterlist, and perhaps
> >> start maintaining Archive-https fields for other mirrors willing to
> >> participate?  That would at least get a minimal list started for this
> >> mode.
> The list should be the smallest problem, one more field doesn't matter
> too much.

Especially with Let's Encrypt, getting proper SSL certificates for all
DNS entries pointing to a mirror shouldn't be such a big issue
anymore. See e.g. https://ftp.ch.debian.org/ aka
https://debian.ethz.ch/ which have a single SSL certificate for hosts
under different second-level domains.

> The biggest problem I see is with what Kurt posted:
> > So the first question I have about this if we can get
> > ftp.TLD.debian.org certificates for this, and what happens when
> > that host is down and DNS gets pointed to a different host?
> > I have to guess that we should only do that on the hostname that
> > is not ftp.TLD.debian.org, while I think it now only shows that
> > name?
> I see no real problem in getting certificates for those domains - way
> more interesting is the handling of them. ftp.*.d.o gets pointed around
> to other mirrors when the usual "owner" of it is down for whatever
> reason. Depending on the country it may also end up on mirrors really
> far away (better that than no ftp.whatever.d.o). So some mirror
> somewhere may not just need one of those certs, but multiple[1]. And a
> single cert/key must be on loads of mirrors. And then comes handling of
> renewals too.

This is probably still an issue despite getting an updated certificate
via Let's Encrypt can be done much quicker than in the past.
Nevertheless there still will be a short downtime without valid
certificate when switching DNS entries.

A wildcard certificate for selected (e.g. DSA-maintained) mirrors
which then would be used as backup host for any primary mirror,
which already provides HTTPS access, would probably suffice.

Example: When ftp.ch.debian.org goes down for longer maintenance we
usually redirected that CNAME to one of the hosts around
kassia.debian.org (ftp.nl.debian.org or similar) where also one of the
European syncproxies is located.

So I can imagine the following setup to mitigate this:

* We track which mirrors (also) provide HTTPS in Mirrors.masterlist.

* If a primary mirror which doesn't provide HTTPS goes down, we
  redirect it to the same mirror as we did in the past.

* If one with HTTPS in central Europe goes down, we redirect it to
  that mirror on kassia.debian.org given that it will have a wildcard
  SSL certificate for ftp*.*.debian.org or similar. (We should look at
  ftp*.*.debian.org instead of just ftp.*.debian.org if possible
  because of mirrors like ftp2.de.debian.org.)

And since quite some mirrors already provide access via HTTPS, IMHO we
should start tracking them independent of having a proper solution to
temporary CNAME changes or not -- because they're in use already if
you want it or not.

So far I'm at least aware of the following working HTTPS mirrors
(picked out a few I read about in previous mailing list threads or
found out by quickly checking in a web browser):

* https://ftp.ch.debian.org/debian/ aka https://debian.ethz.ch/debian/
  (same mirror but two entries in the Mirrors.masterlist)
* https://ftp.de.debian.org/debian/
* https://ftp.cz.debian.org/debian/
* https://ftp.se.debian.org/debian/ (ftp.no.debian.org seems to point
  to the same host, but is not yet accessible via HTTPS due to not
  being listed in the certificate)
* https://mirrors.kernel.org/debian/
* https://mirror.sinavps.ch/debian/
* https://pkg.adfinis-sygroup.ch/debian/
* https://ftp.halifax.rwth-aachen.de/debian/ (not yet accessible as
  https://ftp2.de.debian.org/debian/ due to certificate only being
  valid for ftp.halifax.rwth-aachen.de)
* https://mirrors.wikimedia.org/debian/ (not yet accessible as
  https://ftp.us.debian.org/debian/ due to certificate only being
  valid for mirrors.wikimedia.org)
* https://mirror.as35701.net/debian/ (not yet accessible as
  https://ftp.be.debian.org/debian/ due to certificate only being
  valid for mirror.as35701.net)

A short scan over all mirrors in the Mirror.masterlist showed that
more than one third (176, around 38%) of them have at least port 443

  → GET https://anonscm.debian.org/viewvc/webwml/webwml/english/mirror/Mirrors.masterlist\?view\=co \
    | egrep '^Site:' \
    | awk '{print $2}' \
    | sort \
    | while read m ; do nmap -p443 $m; done \
    > https-mirrors.txt
  → ( echo -n 'scale=2;'`egrep -c '^443/tcp open  https' https-mirrors.txt`/; \
      egrep -c '^443/tcp.*https' https-mirrors.txt ) | bc
  → egrep -c '^443/tcp.*https' https-mirrors.txt
  → egrep -c '^443/tcp open  https' https-mirrors.txt

(Didn't check more certificates or subjectAltNames than those listed
above yet, though. Full list of hosts I found having port 443 open can
be found at the end of this mail.)

After having HTTPS-enabled mirrors listed in the Mirrors.masterlist,
the next step would be to make httpredir.debian.org HTTPS-aware.
Currently https://httpredir.debian.org/ shows me the following error

  httpredir.debian.org uses an invalid security certificate.
  The certificate is only valid for www.debian.org

(Yes, I'm aware that httpredir.debian.org points to quite some

Luckily the software behind httpredir.debian.org seems to be already
HTTPS-aware in some way. At least with Kali's HTTP redirector (which I
assume uses the same software), I get different mirror lists depending
on if I access the service by HTTP or HTTPS:

  → GET -SUsed http://http.kali.org/kali/dists/kali-rolling/Release | egrep '^L'
  Location: http://archive-3.kali.org/kali/dists/kali-rolling/Release
  Link: <http://http.kali.org/kali/dists/kali-rolling/Release.meta4>; rel=describedby; type="application/metalink4+xml"
  Link: <http://archive-3.kali.org/kali/dists/kali-rolling/Release>; rel=duplicate; pri=1; geo=de
  Link: <http://ftp.halifax.rwth-aachen.de/kali/dists/kali-rolling/Release>; rel=duplicate; pri=2; geo=de
  Link: <http://ftp.belnet.be/kali/kali/dists/kali-rolling/Release>; rel=duplicate; pri=3; geo=be
  Link: <http://archive-4.kali.org/kali/dists/kali-rolling/Release>; rel=duplicate; pri=4; geo=fr
  Link: <http://ftp.free.fr/pub/kali/dists/kali-rolling/Release>; rel=duplicate; pri=5; geo=fr
  Last-Modified: Thu, 06 Apr 2017 12:16:51 GMT
  → GET -SUsed https://http.kali.org/kali/dists/kali-rolling/Release | egrep '^L'
  Location: https://ftp.halifax.rwth-aachen.de/kali/dists/kali-rolling/Release
  Link: <http://http.kali.org/kali/dists/kali-rolling/Release.meta4>; rel=describedby; type="application/metalink4+xml"
  Link: <https://ftp.halifax.rwth-aachen.de/kali/dists/kali-rolling/Release>; rel=duplicate; pri=1; geo=de
  Link: <https://archive-3.kali.org/kali/dists/kali-rolling/Release>; rel=duplicate; pri=2; geo=de
  Link: <https://archive-4.kali.org/kali/dists/kali-rolling/Release>; rel=duplicate; pri=3; geo=fr
  Link: <https://ftp2.nluug.nl/os/Linux/distr/kali/dists/kali-rolling/Release>; rel=duplicate; pri=4; geo=nl
  Link: <https://ftp1.nluug.nl/os/Linux/distr/kali/dists/kali-rolling/Release>; rel=duplicate; pri=5; geo=nl

(And IIRC the Kali installer also offered me to choose if I want HTTP,
FTP or HTTPS mirrors, at least with debconf priority set to "low". No
more sure about Debian's installer on my last Stretch installation.)

Full list of mirrors I found having port 443 open (using the data from
above mentioned script):

→ fgrep -B5 '443/tcp open  https' https-mirrors.txt \
    | fgrep 'Nmap scan report for' \
    | cut -c22-        
archive.kernel.org (
archive-klecker.debian.org (
artfiles.org (
buaya.klas.or.id (
cdimage.debian.org (
cosmos.cites.illinois.edu (
deb.debian.org (
debian.anexia.at (
debian.bononia.it (
debian.c3sl.ufpr.br (
debian.charite.de (
debian.cs.binghamton.edu (
debian.dynamica.it (
debian.ethz.ch (
debian.inf.tu-dresden.de (
debian.iskon.hr (
debian.koyanet.lv (
debian.lth.se (
debian.ludost.net (
debian.mirror.ate.info (
debian.mirror.lhisp.com (
debian.mirrors.crysys.hu (
debian.netcologne.de (
debian.osuosl.org (
debian.redimadrid.es (
debian.redparra.com (
debian.revolsys.fr (
debian.simnet.is (
debian.superhosting.cz (
debian.tu-bs.de (
debian.ues.edu.sv (
debian.usu.edu (
debian.uvigo.es (
debian.volia.net (
debian.xtdv.net (
dennou-k.gfd-dennou.org (
dennou-q.gfd-dennou.org (
freedom.dicea.unifi.it (
free.hands.com (
ftp2.cn.debian.org (
ftp2.de.debian.org (
ftp.acc.umu.se (
ftp.antik.sk (
ftp.arnes.si (
ftp.be.debian.org (
ftp.belnet.be (
ftp.bg.debian.org (
ftp.br.debian.org (
ftp.caliu.cat (
ftp.cc.uoc.gr (
ftp.ch.debian.org (
ftp-chi.osuosl.org (
ftp.cn.debian.org (
ftp.crifo.org (
ftp.cz.debian.org (
ftp.debianclub.org (
ftp.debian.cz (
ftp.debian.nl (
ftp.de.debian.org (
ftp.dk.debian.org (
ftp.fau.de (
ftp.gwdg.de (
ftp.halifax.rwth-aachen.de (
ftp.icm.edu.pl (
ftp.iitm.ac.in (
ftp.is.debian.org (
ftp.jaist.ac.jp (
ftp.jp.debian.org (
ftp.lanet.kr (
ftp-master.debian.org (
ftp.metu.edu.tr (
ftp.mpi-sb.mpg.de (
ftp.nluug.nl (
ftp.no.debian.org (
ftp-nyc.osuosl.org (
ftp-osl.osuosl.org (
ftp.rnl.tecnico.ulisboa.pt (
ftp.se.debian.org (
ftp.sg.debian.org (
ftp.sh.cvut.cz (
ftp-stud.hs-esslingen.de (
ftp.tu-graz.ac.at (
ftp.udc.es (
ftp.uk.debian.org (
ftp.uni-mainz.de (
ftp.uni-sofia.bg (
ftp.u-strasbg.fr (
ftp.zcu.cz (
hanzubon.jp (
kambing.ui.edu (
merlin.fit.vutbr.cz (
mirror.0x.sg (
mirror.aarnet.edu.au (
mirror.amaze.com.au (
mirror.applebred.net (
mirror.as35701.net (
mirror.bytemark.co.uk (
mirror.cedia.org.ec (
mirror.checkdomain.de (
mirror.corbina.net (
mirror.crazynetwork.it (
mirror.csclub.uwaterloo.ca (
mirror.cse.iitk.ac.in (
mirror.cse.unsw.edu.au (
mirror.daniel-jost.net (
mirror.de.leaseweb.net (
mirror.dkm.cz (
mirror.hmc.edu (
mirror.i3d.net (
mirror.kku.ac.th (
mirror.liquidtelecom.com (
mirror.litnet.lt (
mirror.math.princeton.edu (
mirror.nexcess.net (
mirror.nforce.com (
mirror.nl.leaseweb.net (
mirror.one.com (
mirror.plusserver.com (
mirror.pmf.kg.ac.rs (
mirror.poliwangi.ac.id (
mirror.positive-internet.com (
mirror.pregi.net (
mirror.rit.edu (
mirrors.acm.jhu.edu (
mirrors.bloomu.edu (
mirrors.cat.pdx.edu (
mirrors.dcarsat.com.ar (
mirrors.dotsrc.org (
mirrorservice.org (
mirrors.evowise.com (
mirror.sinavps.ch (
mirrors.ircam.fr (
mirror.sjc02.svwh.net (
mirrors.kernel.org (
mirrors.linux.iu.edu (
mirrors.lug.mtu.edu (
mirrors.namecheap.com (
mirrors.netix.net (
mirrors.ocf.berkeley.edu (
mirrors.pdx.kernel.org (
mirrors.sfo.kernel.org (
mirrors.syringanetworks.net (
mirror.steadfast.net (
mirrors.tuna.tsinghua.edu.cn (
mirrors.ucr.ac.cr (
mirror.sucs.swan.ac.uk (
mirrors-usa.go-parts.com (
mirrors.ustc.edu.cn (
mirrors.wikimedia.org (
mirrors.xjtu.edu.cn (
mirrors.xmission.com (
mirrors.xservers.ro (
mirror.t-home.mk (
mirror.ueb.edu.ec (
mirror.units.it (
mirror.usertrust.info (
mirror.us.leaseweb.net (
mirror.uta.edu.ec (
mirror.vorboss.net (
mirror.yandex.ru (
oyu-net.jp (
packages.hs-regensburg.de (
pkg.adfinis-sygroup.ch (
pubmirror.plutex.de (
rsync.osuosl.org (
security-master.debian.org (
sft.if.usp.br (
shadow.ind.ntou.edu.tw (
suro.ubaya.ac.id (
syncproxy2.eu.debian.org (
syncproxy2.wna.debian.org (
syncproxy3.eu.debian.org (
syncproxy3.wna.debian.org (
syncproxy.au.debian.org (
syncproxy.cna.debian.org (
the.earth.li (

		Regards, Axel
 ,''`.  |  Axel Beckert <abe@debian.org>, http://people.debian.org/~abe/
: :' :  |  Debian Developer, ftp.ch.debian.org Admin
`. `'   |  4096R: 2517 B724 C5F6 CA99 5329  6E61 2FF9 CD59 6126 16B5
  `-    |  1024D: F067 EA27 26B9 C3FC 1486  202E C09E 1D89 9593 0EDE

Attachment: signature.asc
Description: Digital signature

Reply to: