Bug#966649: UDD: 'upload_history' importer broken; needs porting to Python3
Package: qa.debian.org
User: qa.debian.org@packages.debian.org
Usertags: udd
Hi,
The upload_history importer works as follows:
1) /srv/udd.debian.org/email-archives/debian-devel-changes/ contains a copy
of the email archives, copied manually from master.debian.org. The
latest emails are received directly on ullmann, to /srv/udd.debian.org/email-archives/debian-devel-changes/debian-devel-changes.current
This part is about OK. It would be better if DSA provided a way to
access those archives from ullmann without having to copy them from time
to time.
2) When started, the importer first runs 'make' in /srv/udd.debian.org/upload-history/. This:
2.1) updates local copies of keyrings
2.2) using 'munge_ddc.py', converts email archives into summarized versions, stored as, e.g.:
/srv/udd.debian.org/upload-history/debian-devel-changes.201209.gz.out
3) then the importer reads *.out and import them into postgres.
'munge_ddc.py' has the following issues:
- it's not version-controlled
- it doesn't support xz email archives, so it's broken for recent
archives
- it's python2 (but the lzma module is python3-only)
Help would be welcomed to port it to python3 and resolve the other
issues. Also, the data files around the upload_history gatherer should
probably be reorganized with a cleaner separation between code (that
should be versioned in UDD) and data.
Lucas
Reply to: