Re: overly short max-age of archive -> file redirect
"MOESSBAUER, Felix" <felix.moessbauer@siemens.com> wrote
Tue, 5 Nov 2024 09:09:21 +0000:
> Further, the rate-limits should be precisely documented so clients /
> caching proxies can adapt to this. The limits also need to match the
> retry-after header in the 429 responses. Currently s.d.o responds with
> retry-after 5 (seconds), which is by far to short to overcome the
> limit.
Thanks for reporting this.
I've been planning on documenting the rate limiting somewhere once it
seems to behave reasonably well. It was added in a bit of a haste last
weekend.
One thing I would hesitate to put in a document somewhere for a human to
read once and then base their implementation on is numbers that are
likely to change. What do you think about the server setting
X-RateLimit-Remaining in responses?
The Retry-After has been fixed in [this patchset][] which I will try to
get merged in a day or two.
[this patchset]: https://salsa.debian.org/linus/dsa-puppet/-/compare/production...linus%2Fsnapshot-ratelimit?from_project_id=2990
> If rate-limiting would be implemented correctly, downstream caches
> could properly cache the results and clients like apt could behave
> nicely. I further recommend to use WAY higher request limits in
> combination with a moving average limit on the amount of transferred
> data. By that, the cheap "is my cache still valid" requests could pass,
> while the more heavy payload transfers are avoided. Also clients could
> hit s.d.o without reduced transfer rates, hence reducing the amount of
> open handles on the server.
What do you think would be a reasonable request limit?
We're currently limiting number of requests per time unit since that's
what killed one of the servers last week. I haven't looked into how to
limit the amount of transferred data yet.
Reply to: