Bug#1033632: qa.debian.org: sourceforge redirector for debian/watch files fails with a 500 error

[Lucas Nussbaum <lucas@debian.org>]

On 13/04/23 at 08:29 +0800, Paul Wise wrote:
> On Wed, 2023-04-12 at 07:14 +0200, Lucas Nussbaum wrote:
> 
> > There's specific code in the UDD uscan wrapper[1] to handle github's
> > rate limiting. We could have something similar for either sf.net, or the
> > sf.net redirector. Before I work on that, it would be great if someone
> > could change the sf.net redirector to return 429 instead of 500 when
> > sf.net returns 429, so that this specific case is easier to identify.
> 
> This is now done, tested and deployed on the server:
> 
> https://salsa.debian.org/qa/qa/commit/395d923257e954663156fa315142415f50d1be6a
> 
> I elected to just pass on all SourceForge HTTP error codes,
> with the HTTP error text prefixed to clarify the error source.

Thanks!
I added code to handle sf.net's rate limiting in the UDD importer, and
triggered a refresh of all sf.net-hosted packages.

I wonder if we should close this bug. The redirector has not been fixed
(it will still hit rate limiting, but there's not much we can do about
that); but the main path by which maintainers probably access watch data
(UDD -> dashboards) has been fixed.

- Lucas

Some UDD notes for reference:
To watch the status of UDD trying to refresh all SF sources:
udd=> select status, count(*) from upstream where watch_file ~ 'sf.(net|php)' group by status;
            status            | count 
------------------------------+-------
 newer package available      |   120
 up to date                   |   469
 error                        |   976
 only older package available |    53
(4 rows)

udd=> select warnings is null, count(*) from upstream where watch_file ~ 'sf.(net|php)' group by 1;
 ?column? | count 
----------+-------
 f        |   986
 t        |   632
(2 rows)

To force a refresh of all sf.net sources:
update upstream set last_check = null where watch_file ~ 'sf.(php|net)' and warnings is not null;

- Lucas