[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: dl10n-spider stops parsing mails when there is a month without mails

On Tue, 18 Sep 2018 10:55:47 +0200
Laura Arjona Reina wrote:

> It seems that the script that parses the l10n-language mailing list archives and 
> creates the databases in l10n.debian.org/coordination/00data/status.XX stops 
> (without errors) when there is a month in which no mails were sent to the list 
> (thinking that we arrived the current month/year).

> ./dl10n-spider --year 2016 --month 01 --message 0 tr
> (it begins to parse and stops in 2016/06)
> ./dl10n-spider --year 2016 --month 07 --message 0 tr
> (it continues to parse and stops in 2017/05)
> ./dl10n-spider --year 2017 --month 07 --message 0 tr
> (it continues to parse and stops in 2017/11, etc.)

> If you look at https://lists.debian.org/debian-l10n-turkish/ the script stops 
> when it arrives a year/month without mails, but can continue if we provide a 
> starting point after that empty month.

> I've had a look at the code and I think that we should modify the code of the 
> file 
> https://salsa.debian.org/l10n-team/dl10n/blob/master/lib/Debian/L10n/Spider.pm , 
> sub spider($$$$$@) (Line 173 on). I'm afraid I cannot provide a patch, the code 
> is difficult for me to understand and I'm not sure what should I change and how.

assumed your assumption is correct (nonexistent index causes crash/die),
you can prevent/skip the month by supplying year+month list 
instead of letting the script generate it (line 225-);
you can extract the list from the index page (lists..../...-l10n-{lang}/)

no need to CC me :-)

Reply to: