Re: help wanted, standing up mirroring sync proxies on public cloud
Hi Bastian,
On Thu, Mar 17, 2022 at 09:58:12PM +0100, Bastian Blank wrote:
> Hi Julien
>
> On Thu, Mar 17, 2022 at 12:03:18PM +0100, Julien Cristau wrote:
> > DSA's looking into options to replace some of our archive mirroring
> > infrastructure. For context, so far we've been maintaining a few machines
> > around the globe, called syncproxies, that serve as "hubs" for archive
> > mirroring and push downstream mirrors.
>
> You are just talking about the authenticated rsync and push stuff right
> now? Because mirror-isc.d.o for example does more.
>
I figured we'd start there, yes. Moving static mirrors around seems a lot
easier.
> > Would it be possible to work with the cloud team to stand up appropriate
> > accounts and so on on one of the cloud infras Debian has a relationship
> > with?
>
> We only have a relationship with AWS, via SPI, that allow us to just do
> things within reason. We should ask them nevertheless just as good
> measure, because that's a ongoing commitment.
>
> > (One possibly complicating factor is there's some element of sensitivity
> > because these machines host embargoed binaries for the security
> > archive.)
>
> There are some mails about that from January 2018 in the mirrors@
> mailbox.
>
> Some questions:
>
> How much resources to you think you need?
> Resource in cloud environments are usually tightly coupled. You get X
> cpu, X*Y ram and X*Z network/disk throughput.
>
The machines we got most recently (smit and schmelzer) have 2x8c (2x
Xeon Silver 4110 @ 2.10GHz), 96G ram, 10x4T SAS disks (12Gbps, 7.2k rpm)
mirror-isc is older: 2x 12c (2x Xeon E5-2670 v3 @ 2.30GHz), 64G ram,
4x2T SAS disks (12Gbps, 7.2k rpm)
cpu-wise they're probably way overkill, and the ram is obviously mostly
used for cache.
> Do you intend to make the syncproxy setup a bit more failover friendly?
> So you can kill one and make another take up the work.
>
I'm not sure. Some of that is a bit constrained by things like
downstream firewalls. I'd be interested though if you have suggestions
of things we could do.
Cheers,
Julien
Reply to: