Re: dl10n-spider stops parsing mails when there is a month without mails
On Tue, 18 Sep 2018 10:55:47 +0200
Laura Arjona Reina wrote:
> It seems that the script that parses the l10n-language mailing list archives and
> creates the databases in l10n.debian.org/coordination/00data/status.XX stops
> (without errors) when there is a month in which no mails were sent to the list
> (thinking that we arrived the current month/year).
> ./dl10n-spider --year 2016 --month 01 --message 0 tr
> (it begins to parse and stops in 2016/06)
> ./dl10n-spider --year 2016 --month 07 --message 0 tr
> (it continues to parse and stops in 2017/05)
> ./dl10n-spider --year 2017 --month 07 --message 0 tr
> (it continues to parse and stops in 2017/11, etc.)
> If you look at https://lists.debian.org/debian-l10n-turkish/ the script stops
> when it arrives a year/month without mails, but can continue if we provide a
> starting point after that empty month.
> I've had a look at the code and I think that we should modify the code of the
> file
> https://salsa.debian.org/l10n-team/dl10n/blob/master/lib/Debian/L10n/Spider.pm ,
> sub spider($$$$$@) (Line 173 on). I'm afraid I cannot provide a patch, the code
> is difficult for me to understand and I'm not sure what should I change and how.
assumed your assumption is correct (nonexistent index causes crash/die),
you can prevent/skip the month by supplying year+month list
instead of letting the script generate it (line 225-);
you can extract the list from the index page (lists..../...-l10n-{lang}/)
--
victory
no need to CC me :-)
Reply to: