Re: Snapshot behind Fastly; roles and responsibilities
On Mon, 2024-11-18 at 10:35 +0100, Linus Nordberg wrote:
> Hi all,
>
> Snapshot is behind Fastly since Sunday Nov 17 2024. I think that's
> bad
> and would like to change that. It's bad in the short term since we
> expose user data to a third party. It's bad in the long term since
> the
> short term bad won't go away until we learn how to deal with web
> traffic.
That's a trade off between the advantages of a CDN and privacy.
For me as snapshot user that needs it to build reproducible things in
CI systems, the most important aspect is reliability and performance.
>
> I have not been able to solve the problem with more incoming HTTP
> traffic than what the snapshot setup comfortably can deal with.
> Partly
> because I'm not very knowledgeable in this field and partly because I
> have not been given enough access to the cache layer(s).
I also had a look at this topic (mostly based on code-review) and
identified a couple of problems:
1. apt behaves badly on 429 TooManyRequests. Addressed in [1]
2. Expensive redirects to farm (DB lookup!) are cached too short.
Addressed in [2], also affected by [3]
3. Varnish internal redirect to farm not working [4], unfortunately
reverted due to not working properly in prod setup
[1] https://salsa.debian.org/apt-team/apt/-/merge_requests/383
[2] https://salsa.debian.org/snapshot-team/snapshot/-/merge_requests/23
[3]
https://salsa.debian.org/dsa-team/mirror/dsa-puppet/-/commit/63f16e08199040871752135df533f0001fe537fb
[4] https://lists.debian.org/debian-snapshot/2024/11/msg00008.html
>
> DSA have legitimate concerns about exposing user data to people who
> do
> not need access to it. Would it help if my relation to Debian was
> formalised further than the current status of Debian Contributor?
I'm just a DM, but I definitely want to help improving the situation.
>
> More generally, I sometimes find it hard to understand the roles and
> responsibilities wrt the snapshot service. This results in me on the
> one
> hand being overly cautious with asking for some things and on the
> other
> hand sometimes pestering the wrong people, most probably also in the
> wrong way. It would be good to minimise unnecessary frustration and
> lost
> calendar time.
Same! It took me quite some time to get an understanding of the overall
architecture of s.d.o which all its layers. Also I don't know who is
responsible for the intermediate infrastructure (basically everything
between the s.d.o flask app and the DNS entry s.d.o).
I further can only guess where exactly the bottlenecks are. These
obviously depend on the usage patterns which I (for good reasons) do
not have insights into.
>
> There's a Snapshot service meeting today at 1700Z in #debian-snapshot
> for all who are interested in helping out.
I'll try to join.
Best regards,
Felix Moessbauer
--
Siemens AG, Technology
Linux Expert Center
Reply to: