Great!
It sounds to me like if we use the *mtime* of /srv/
udd.debian.org/email-archives/debian-devel-changes/debian-devel-changes.current (but not its contents), that would smoothly and solidly overcome the worries about unnecessary polling. If the file's mtime is the same as the last time the tool ran, then no need to run any import process (no matter if the import process involves HTTP or not). If we are only relying on mtime (and we're the only consumer), we can truncate it before looking at it. :)
For the XZ mbox files, it sounds to me like they're currently not reliable -- their existence on
ullmann.debian.org depends on disk space stuff.
Related question: My code requires a 4-5GB cache file if it stores all data from all years of debian-devel-changes. Is that workable? If not, it's easy to cut it down. Is 1GB appropriate to expect as cache storage space? If not, what about approx 400MB? (Context: 2020's data so far takes up 208MB in my cache format.)
I need a few days to find my Debian GPG key (I moved house as well as primary computer recently), which will allow me to reset my Debian SSH key, which will allow me to SSH to ullmann and check these things directly. I hope that explains why I'm not just ssh-ing to the host and finding out what "df -h" outputs. I'm mildly embarrassed to be in this confused GPG key situation, but that's the situation, so there you go.
Grateful,
Asheesh.