Bug#1033632: qa.debian.org: sourceforge redirector for debian/watch files fails with a 500 error
Hi,
On 12/04/23 at 09:16 +0800, Paul Wise wrote:
> On Wed, 2023-03-29 at 08:05 +0200, Christian Marillat wrote:
>
> > For several days sf.php no longer works:
> >
> > ,----
> > > uscan warn: In watchfile debian/watch, reading webpage
> > > https://qa.debian.org/watch/sf.php/synfig/ failed: 500 Error
> > `----
>
> This issue is caused by the underlying SourceForge infrastructure
> (their files RSS feed) starting to apply rate limiting and returning
> HTTP 429 Too Many Requests errors, which the Debian QA redirector
> easily hits, depending on how much use the service has per day.
>
> We could have individual contributors rewrite every single one of their
> SourceForge debian/watch files to use the SourceForge files RSS feeds.
>
> Alternatively we could move the code for the SourceForge redirector
> into uscan so that individual uscan users get separate rate limit
> buckets, rather than having one large Debian rate limit bucket.
>
> Unfortunately these changes will not fix the problem of UDD getting
> errors all the time. To fix that, UDD would need to gain a distributed
> architecture with multiple IP addresses all contacting SourceForge.
> That may cause overloads of the SourceForge server resources though,
> which would probably lead to uscan getting blocked again.
>
> So maybe we need to discuss this with SourceForge again.
There's specific code in the UDD uscan wrapper[1] to handle github's
rate limiting. We could have something similar for either sf.net, or the
sf.net redirector. Before I work on that, it would be great if someone
could change the sf.net redirector to return 429 instead of 500 when
sf.net returns 429, so that this specific case is easier to identify.
[1] https://salsa.debian.org/qa/udd/-/blob/master/rimporters/upstream.rb#L161
Lucas
Reply to: