[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Using DDTP translations



Il giorno Thu, 1 Nov 2007 10:46:35 +0100
Steffen Moeller <steffen_moeller@gmx.de> ha scritto:

> 
> ...
> 
> A direct link to the German translations can be offered via 
> http://kleptog.org/cgi-bin/ddtss2-cgi/de/fetch?force=yes&package=%s
> with %s being the name of the binary package. The "de" needs to be
> substituted by the respective language.

I'm working on a draft script that automatically fetches the packages
description and, using templates (like the bugs page does), generates a nice
page with (possibly) icons, flags, ...

A problem I've just come across is the server load. I'm testing the script with
just two packages, not with all debian-med's, and it's just taking too long. If
we want to support _all_ the languages DDTP does, we'll take ages to
automatically do the thing. Here's the issue:

- DDTP supports 25 languages:
-- langs = ["ca", "cs", "da", "de", "eo", "es", "fi", "fr",
    "go", "hu", "it", "ja", "km_KH", "ko", "nl", "pt",
    "pt_BR", "pt_PT", "ru", "sk", "sv", "uk", "zh", "zh_CN",
    "zh_TW"]
- for each package I need to:
-- a) fetch http://ddtp.debian.net/ddt.cgi?package=<pkg> to get the last
description ID;
-- b) fetch and parse http://ddtp.debian.net/ddt.cgi?desc_id=<id>, to get
translated/untranslated status;
-- c) fetch
http://kleptog.org/cgi-bin/ddtss2-cgi/<lang>/fetch?force=yes&package=<pkg>

Total count for each package: 3 x 25 languages = 75 *SEVENTY-FIVE* urlopen()
calls. This for *EACH* package.

My test with two packages gave:

$ time ./get-bugs.py 
gnome-rdp [('ca', 0), ('cs', 0), ('da', 0), ('de', 0), ('eo', 0), ('es', 0),
('fi', 0), ('fr', 0), ('go', 0), ('hu', 0), ('it', 0), ('ja', 0), ('km_KH', 0), ('ko', 0), ('nl', 0), ('pl', 0), ('pt', 0), ('pt_BR', 0), ('pt_PT', 0), ('ru', 0), ('sk', 0), ('sv', 0), ('uk', 0), ('zh', 0), ('zh_CN', 0), ('zh_TW', 0)]
proda [('ca', 0), ('cs', 0), ('da', 0), ('de', 0), ('eo', 0), ('es', 0), ('fi',
0), ('fr', 0), ('go', 0), ('hu', 0), ('it', 0), ('ja', 0), ('km_KH', 0), ('ko',
0), ('nl', 0), ('pl', 0), ('pt', 0), ('pt_BR', 0), ('pt_PT', 0), ('ru', 0),
('sk', 0), ('sv', 0), ('uk', 0), ('zh', 0), ('zh_CN', 0), ('zh_TW', 0)]

real    4m31.510s
user    0m2.076s
sys     0m0.476s
$

four minutes for just two packages :(

(and no, I commented out bugs page generation, bugs retrieval and all the SOAP
stuff which takes ages to complete).

This means that, as for now, we have 90 packages x 75 urlopen()s each = ...
6750 calls. *SIX THOUSANDS SEVEN HUNDREDS FIFTY*. That's a *bit* too much IMHO.
We can:

1) reduce the number of supported languages. Probably we can support the same
languages on the website and in the packages. This means lowering the languages
count from 25 to 4 (en, it, fr, de) (the preferable one to me);
2) do not support automatic fetching of urls -- one must click an icon / link
to fetch the url and work on that (not desiderable IMHO);
2) find some other way to do this.


Kindly,
David

P.S.: I'll upload the rough script tomorrow morning maybe, I'm going to bed
now :)

-- 
 . ''`.  Debian maintainer | http://snipurl.com/qa_page/
 : :'  :  Linuxer #334216  |  http://www.hanskalabs.net/
 `. `'`    GPG: 1392B174   | http://www.debianizzati.org/
   `-   2BAB C625 4E66 E7B8 450A C3E1 E6AA 9017 1392 B174

Attachment: signature.asc
Description: PGP signature


Reply to: