new in UDD: duck importer (URL checker)
Hi,
I added a duck importer[0] and a duck dashboard[1] to UDD.
Previously the duck.debian.net service was providing regular checks
about URLs in Debian package, but it broke at some point around 2020.
# importer internals
The UDD importer[0] works as follows:
- URLs are collected out of all source packages using the 'duck' package
(a vendorized version[2], because I needed to add a dump mode, which
is also submitted as #1107642)
This is tracked in the duck_raw (raw output from duck) and duck_urls
(easier to work with, one row per (source/version, URL))
- On a regular basis, UDD checks all known URLs (with a retry policy
depending on whether it's possibly a transient failure).
duck is _not_ used for this part (the importer has its own check
logic), because the check happens URL per URL, not package per package.
This is tracked in the duck_url_status.
# dashboards
- per maintainer view: https://udd.debian.org/duck/?email1=pkg-perl-maintainers%40lists.alioth.debian.org#results
- The DMD 'duck' column works again: https://udd.debian.org/dmd/?email1=pkg-perl-maintainers%40lists.alioth.debian.org&email2=&email3=&packages=&ignpackages=&format=html#details
# statistics
* UDD knows about 207055 URLs
* 15207 URLs (7.34%) are failing
* 937 (2.37%) source packages failed to be processed by duck (I need to
look into that)
[0] https://salsa.debian.org/qa/udd/-/blob/master/rimporters/duck.rb?ref_type=heads
[1] https://udd.debian.org/duck/
[2] https://salsa.debian.org/qa/udd/-/tree/master/vendor/duck?ref_type=heads
Best,
Lucas
Reply to: