[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: test of debconf po files and some questions...

Michael Bramer wrote:
> I make the first files from the ddtp db. 
> Please look at http://ddtp.debian.org/debconf/maintainer and you will
> see a list of all package maintainers with debconf templates. On the
> pages you will find a list of debconf po file links. 

I have written the attached program to aid in downloading from those
pages. When run in a package's directory, it downloads the page[1], parses
it, and tries to figure out what templates files to download. Current
modes of operation are:

- The default mode of operation: Download all tranlsations that are
  marked as complete, and also download incomplete translations if the
  translation does not exist at all locally. In theory this mode will
  lose no local data and will get as many translations as it can.
- Only download translations that are marked as new on the page. This is 
  of limited value, since I'm not sure over what time period something
  is considered "new" -- if it is just new today then this is useless
  unless you run the script every day, etc.
- Download everything, even incomplete translations, overwriting
  existing files that may heve more translations in them from non-DDTP
  sources. Mostly useful so far if you are just starting out on
  collecting translations for a package, or want to switch over entirely
  to DDTP translations for your package.
- Download better translations -- ones which the DDTP has more strings
  traslated than does the package in unstable.

It can also skip any translations that are not marked as updated.

None of these modes are perfect, but they seem to be the best I can do
with the available data. Some example runs:

de.po:      complete    downloading     (100% vs 66%)
ja.po:      exists      skipping        (66% vs 66%)
pt_BR.po:   exists      skipping        (66% vs 66%)
fr.po:      exists      skipping        (66% vs 66%)
it.po:      exists      skipping        (0% vs 66%)
da.po:      complete    downloading     (100% vs 66%)
pl.po:      exists      skipping        (0% vs 66%)
ru.po:      exists      skipping        (66% vs 66%)
fi.po:      exists      skipping        (66% vs 66%)
es.po:      exists      skipping        (66% vs 66%)
joey@dragon:~package/mooix>gettrans --all
de.po:      complete    downloading     (100% vs 0%)
joey@dragon:~package/base-config>gettrans --better --dry-run
de.po:      better      downloading     (93% vs 59%)
ja.po:      better      downloading     (65% vs 43%)
pt_BR.po:   worse       skipping        (87% vs 100%)
fr.po:      worse       skipping        (71% vs 100%)
it.po:      better      downloading     (62% vs 43%)
da.po:      better      downloading     (96% vs 90%)
pl.po:      better      downloading     (71% vs 62%)
hu.po:      better      downloading     (59% vs 40%)
nl.po:      better      downloading     (62% vs 43%)
ru.po:      worse       skipping        (68% vs 68%)
cs.po:      better      downloading     (75% vs 53%)
es.po:      better      downloading     (71% vs 53%)

It would be a lot better of course if it could merge existing with DDTP
translations. I suspect that in my base-config run above, I'd have lost
several translated strings while downloading the "better" translations
from the ddtp if I'd let it really download.

Security is another unaddressed issue.

It does look like it will work resonably well at least for complete
translations, but this program should be considered an experimental,
unsafe and unsecure prototype for now. Updated versions will be
available from http://cvs.kitenet.net/joey-cvs/bin/gettrans until
something better comes along.

One thing I do not understand is that, after downloading the complete da and
de translations for debconf, and verifying they look ok, and making sure I
have run debconf-updatepo, po-debconf is still dropping the descriotion of
debconf/priority for both languages. Dennis?

> Joey: If you remove the '.po' from the links, you will get the old
>   debconf templates files. Now I don't add the encoding to this files.
>   Make debconf-mergetemplate problems if I add the encoding in this old
>   files? (like Description-de.ISO-8859-1: xxx)

I think that _might_ work, but I've never tried it.

see shy jo

[1] It would be marginally easier for automation if similar pages were
    available per source package.
# First attempt at automatic debconf po file updates from the DDTP.
# Joey Hess <joeyh@debian.org>, GPL
# Run this in the directory of your package.

use warnings;
use strict;

sub usage {
	print STDERR <<"EOF";
Usage: $0 [nbadh]
  -n, --new	Only download newly updated translations.
  -b, --better	Overwrite existing translations if DDTP has more complete ones.
  -a, --all	Download even incomplete tranlations, overwriting existing.
  -d, --dry-run	Do not really download anything.
  -h, --help	Show this help text.

Default behavior is to download all complete tranlations, as well as any
incomplete translations that are not translated at all yet locally.
Files are downloaded to debian/po/<ll>.po

Note that some modes may cause loss of data if you have translations
from third parties. Do a dry run first!

use Getopt::Long;
my ($only_new, $all, $better, $dry_run);
	"new|n" => \$only_new,
	"all|a" => \$all,
	"better|b" => \$better,
	"dry-run|d" => \$dry_run,
	"help|h" => \&usage,
) || usage();

my $email='';
my $src='';
open (CONTROL, "<debian/control") || die "open debian/control: $!\n";
while (<CONTROL>) {
	if (/^Maintainer:\s*.*?<(.*)>/) {
	if (/^Source:\s*(.*)/) {
close CONTROL;

if (! length $email) {
	die "unable to find email address in debian/control\n";
if (! length $src) {
	die "unable to find source package name in debian/control\n";

open (LIST, "wget -O - --quiet http://ddtp.debian.org/debconf/maintainer/$email |") ||
	die "wget: $!\n";
while (<LIST>) {
	if (/^\s*(\+)?\s*(\*)?\s*<\s*a\s*href="([^"]+)"\s*>([^\/]+)\/templates-([^.]+)\.po\s*<\/a>\s*(\d+)\/(\d+)\/(\d+)\s*(\d+)\/(\d+)/) {
		my ($file, $pkg, $ll, $trans_in_unstable, $trans_from_ddtp,
		    $total_in_unstable, $trans_texts, $total_texts) =
			($3, $4, $5, $6, $7, $8, $9, $10);
		my $updated = (defined $1);
		my $complete = (defined $2);

		next if $pkg ne $src;
		next if $trans_texts == 0;
		print "$ll.po:". ' ' x (8 - length $ll);

		my $pct = "(".int($trans_from_ddtp / $total_in_unstable * 100).
		          "% vs ".int($trans_in_unstable / $total_in_unstable * 100).
		if ($only_new && ! $updated) {
			print "old\t\tskipping\t$pct\n";
		if (! $complete && ! $all) {
			if ($better && $trans_from_ddtp > $trans_in_unstable) {
				print "better\tdownloading\t$pct\n";
			elsif (-e "debian/po/$ll.po") {
				if (! $better) {
					print "exists\tskipping\t$pct\n";
				else {
					print "worse\tskipping\t$pct\n";
			else {
				print "new\tdownloading\t$pct\n";
		else {
			print "complete\tdownloading\t$pct\n";
		system ("wget --quiet -O debian/po/$ll.po http://ddtp.debian.org/$file";)
			unless $dry_run;
close LIST || die "wget close: $!\n";

Attachment: pgpTcj6_3e4hd.pgp
Description: PGP signature

Reply to: