ddts-rev version 0.9.5
Bonjour,
Voici le changelog.
Les astérisques indiquent les changements les plus significatifs.
version 0.9.5
* - add: mail save in $pkg.$new when not sent
- add: if $pkg.$mail exists, try to send it
- add: if $pkg.$new exists, send nothing
- add: sub to test and remove files to clean
- add: sub to remove temp files
- add: $tmp to set a dir for temporary files
- add: sub to extract comments and new version from review
- add: sub to extract translator version, name and report id from description
file
- add: sub to diff two strings
* - add: when parsing, if desc is ok, rename $pkg.$todo -> $pkg.$rev so nothing
as to be done by the reviewer
- change: clean tests
- change: statistics presentation according longest figure
* - change: sent mail extension, caution : if upgrading from previous version,
change all your *.mailSent into *.sent
- change: default 'yes' when sending mail
- bug fix
- add: \n in test_remove message
- add: missing messages when removing temp files
- add: mail next package after sending existing $pkg.$mail
- add: don't mail/clean hiden files
- change: test in 'clean' to find file without extention, as '.' search
conflict with package name containing version number
- change: use $text for building mail, as $mail conflits with the file
extension
ATTENTION : j'ai changé l'extention des courriels envoyés : de
« mailSent » (trop longue à mon goût) en « sent », utilisez le script
« sentext-change » joint après l'avoir configuré pour modifier vos
fichiers existants.
J'utilise beaucoup moins de fichiers temporaires, surtout en écriture,
un fin connaisseur de Perl pourrait-il me dire si ceci suffit à
expliquer l'accélération foudroyante constatée ? (Celle-ci m'inquiète
car je ne me l'explique pas autrement)
Je pense qu'il s'agit d'une version presque terminée du script,
peut-être reste-t-il des bogues que je m'empresserai de corriger si vous
m'en faites part. Si vous voulez d'autres fonctions, je peux aussi
essayer de les ajouter.
J'essaye d'avoir une page perso sur wanamou, pour éviter de toujours
poster le script sur la liste.
Nicolas
--
#! /usr/bin/perl
use strict;
### Config part
# where to put all the files
# Things you MUST change
my $home="/home/nico/Mail/rel/ddts";
my $pkg;
opendir (PKGLIST,$home) || die "Cannot read the content of $home: $!\n";
chdir $home || die "Can't chdir to $home: $!\n";
foreach (readdir(PKGLIST)) {
if ($_ =~ /\.mailSent$/) {
$pkg = $`;
print "$pkg\n";
rename "$home/$pkg.mailSent", "$home/$pkg.sent";
}
}
#! /usr/bin/perl
use strict;
### Config part
# where to put all the files
# Things you MUST change
my $home="/home/nico/Mail/rel/ddts";
my $tmp="/tmp";
my $mail_from='Nicolas Bertolissio <nico.bertol@wanadoo.fr>';
my $mail_begin="Bonjour,\n\n"
#."Tu trouveras en attachement un fichier diff entre la version que tu as\n"
#."soumise au ddts et une version corrigée par mes soins de la description\n"
#."de ce paquet. Si tu es d'accord avec les corrections, merci de renvoyer\n"
#."une version corrigée au ddts. Dans le cas contraire, n'hésite pas à me\n"
#."contacter.\n";
."Si tu n'es pas d'accord avec mes corrections, contacte-moi, sinon envoie\n"
."une version corrigée au ddts.\n";
my $mail_end="\na+\n\nNicolas\n";
# Things you can change
my $debug=1; # level of verbosity 0..3
my $desc_ok="DescOk.txt"; # filename for ok description list
my $comment=">> "; # comment string
my $linenumber="ligne %d :"; # new comment line, must contain %d
my $sendnew="no"; # if set to "yes", send the new description
my $selfsend="yes"; # if set to "yes", you will be sent a copy of outgoing mails
my $mail_title="[relecture] %s"; # must contain %s
my $mail_enc="8bits";
my $mail_charset="iso-8859-1";
my $todo="todo"; # extention of files waiting for a review (*)
my $rev="relu"; # extention of files you already reviewed
my $new="newmail"; # extention of newly created mail files (*)
my $mail="mail"; # extention of ready to be sent mail files
my $sent="sent"; # extention of mails sent (*)
# files in category marked with (*) are automatically generated.
# Don't edit them, change their name before it.
### End of the config part
=head1 NAME
ddts-rev - a tool to ease the work of reviewer of ddts
=head1 SYNOPSIS
ddts-rev [parse|mail|clean|stast]
=head1 DESCRIPTION
This program helps the reviewer of package description translation with the ddts.
Here is the basical review process if you use this tool:
=over 4
=item o You get a daily report from the ddts, containing changed translations.
Pass this mail to the standard input of this script, providing the I<parse> option.
It will create two files called I<pkg-name> and I<pkg-name.todo> for each translation
in the mail. The first one is the unchanged translation, for internal use.
The second is a template to ease your review.
=item o You do your review
For that, rename I<pkg-name.todo> to I<pkg-name.relu>. Then, change the description
found there, and add any comments you want in line starting with:
'>> '
=item o You're done with the review and want to send your work to translators back.
Call the script with the I<mail> option. It will make a diff of your version and
the translators one. If the two versions are equal, nothing will be done. Else,
the script will send a mail in mime format with three parts:
=over 4
=item - the comments you've put in the comment lines
=item - the version resulting of your review
If the variable $sendnew is set to "yes"
=item - the diff between your version and the translator one.
=back
Then, the script will show you the resulting mail, and prompt you if you want to
send it or not. Be carfull with sending mails. Please make sure you are really
done with the review before.
Lastly, the script will save the sent mail to I<pkg-name.mailSent> to make sure
the mail won't be sent several times if you run the script several times.
=item o clean your workspace
Calling the script with the I<clean> option removes all temp files like *~ ones,
or the ones created internally. If I<pkg-name.todo> and I<pkg-name.relu> both
exists, the script will remove the first. So, be carfull when using this option...
=item o stats gives statistics
Print some statistics about the descriptions review process
=back
=cut
my $version="0.9.5";
# Test if configuration as been made
if (! -d $home) {
mkdir $home || die "Can't create $home. Is the script configured?";
}
# Remove $path/$file
# Print the corresponding message
sub remove {
my($path, $file)=@_;
if (-e "$path/$file") {
print "Removing $file\n";
unlink "$path/$file" || die "Can't remove $file: $!";
}
}
# Extract translator address
# ddts report id
# original translation
# from $path/$file
sub pkg2tmptransid {
my $tmp="";
my ($path, $file)=@_;
open PKG, "$path/$file" || die "Can't read $file: $!";
chomp(my $translator=<PKG>);
$translator =~ s/Translator: //;
$debug && print "Translator: $translator\n";
chomp(my $reportid=<PKG>);
$reportid =~ s/ddts Id: //;
$debug && print "ddts Id: $reportid\n";
$tmp .= $_ while <PKG>;
close PKG || die "Can't close $file: $!";
return ($tmp, $translator, $reportid);
}
# Extract comments
# new version
# from $path/$file
sub rev2newcomment {
my $com="";
my $new="";
my $linecount=0;
my $lastline=0;
my ($path, $file)=@_;
open REV, "$path/$file" || die "Can't read $file: $!";
# discard translator adress and ddts report id
<REV>; <REV>;
while (<REV>) {
if (/^$comment/) {
if ($linecount!=$lastline) {
$lastline=$linecount;
$com.= "\n".sprintf($linenumber, $linecount)."\n";
}
$com.= $';
} else {
$new .= $_;
$linecount++;
}
}
close REV || die "Can't close $file: $!";
return ($new, $com);
}
sub parse_report {
my $pkg; # package name
my $translator; # translator address
my $desc; # english description
my $trans; # translated description
my $lang; # language translated into
my $nonamecount=0; # extention number for noname files
my $reportId; # Message-Id field
my $reportDate; # Date of ddts report
# Read the mail header
# Get the 'Message-Id:' field so we can use 'In-Reply-To:' when sending email
# Get teh 'Date:' field for use in unamed descriptions filename
while (<>) {
last if (/^$/);
if (/^message-id: /i) {
chomp($reportId = $');
$debug && print "Message-Id: $reportId\n";
}
if (/^date: /i) {
chomp($reportDate = $');
$debug && print "Date: $reportDate\n"
}
}
# Read all packages
while (<>) {
# Read the translator name
chomp($translator=$_);
$debug && print "Translator: $translator\n";
last if $translator eq "";
# Read the package name
chomp($pkg=<>);
$debug && print "Package: $pkg\n";
# Read the english description
if (<> =~ /^Description: *(.*)$/) {
$desc="$1\n";
$desc .= $_ while (($_ = <>) =~ /^ /);
}
$debug>2 && print "Description: $desc\n";
# Read the translated description
if (/^Description-(..)(_..)?: *(.*)$/) {
$lang="$1$2";
$trans="$3\n";
$trans .= $_ while (($_ = <>) =~ /^ /);
}
$debug>2 && print "Description-$lang: $trans\n";
if (/^$/) {
# Search for package name if not provided by ddts
if ($pkg eq "") {
my $shortdesc=$desc;
$shortdesc =~ s/\n.*//mg;
$debug && print "short description: $shortdesc\n";
open APT, "apt-cache search '$shortdesc'|" || die "Unable to run apt-cache: $!" ;
chomp ($pkg = <APT>);
close APT || die "Unable to run apt-cache: $!\n";
$pkg =~ s/^([^ ]*) .*$/$1/;
if ($pkg ne "") {
print "I guess '$shortdesc' referes to '$pkg'\n";
} else {
$pkg="noname-$reportDate-".$nonamecount++;
print "I can't guess '$shortdesc', output file named '$pkg'\n";
}
}
if ($pkg eq "" ||
$translator eq "" ||
$desc eq "" ||
$trans eq "" ||
$lang eq "") {
die "Parse error: empty line before the package is well defined.\n"
."Are you sure I'm parsing a repport from the ddts?\n"
.undef($pkg).": $pkg";
}
warn "Overwritting $home/$pkg.$todo\n" if -e "$home/$pkg.$todo";
# Outputs the package
open PKG, ">$home/$pkg" || die "Can't open $home/$pkg";
open TODO, ">$home/$pkg.$todo" || die "Can't open $home/$pkg.$todo";
my $str = "Translator: $translator\n"
."ddts Id: $reportId\n"
."Description: $desc"
."Description-$lang: $trans";
$debug>2 && print "$str\n";
print PKG "$str\n";
print TODO "$str\n";
close PKG || die "Can't write '$home/$pkg': $!\n";
close TODO || die "Can't write '$home/$pkg.$todo': $!\n";
# Move rev to rev.old
if (-e "$home/$pkg.$rev") {
print "Moving $pkg.$rev -> $pkg.$rev.old\n";
rename "$home/$pkg.$rev", "$home/$pkg.$rev.old";
# If description eq last review rename $pkg.$todo in $pkg.$rev
# so the reviewer won't spend time with it
my($tmp) = &pkg2tmptransid("$home", "$pkg");
my($new) = &rev2newcomment("$home", "$pkg.$rev.old");
if ($tmp eq $new) {
print "$pkg is ok\n";
&remove ($home, "$pkg.$rev.old");
print "Moving $pkg.$todo -> $pkg.$rev\n";
rename "$home/$pkg.$todo", "$home/$pkg.$rev";
}
}
&remove($home, "$pkg.$sent");
# clears the variables
$pkg=$translator=$desc=$trans=$lang = "";
}
# stop when encountering the signature
last if /^-- ?$/;
}
}
# Diff two strings
# $name is only provided for debugging purposes
sub diff {
my $diff="";
my ($name, $orig, $dest)=@_;
open ORIG, ">$tmp/$name" || die "Can't create temp file $name: $!";
print ORIG $orig;
close ORIG || die "Can't write temp file $name: $!";
open DEST,">$tmp/$name.new" || die "Can't create temp file $name.new: $!";
print DEST $dest;
close DEST || die "Can't write temp file $name.new: $!";
system "diff -u $tmp/$name $tmp/$name.new > $tmp/$name.diff" || die "Can't run diff: $!";
open DIFF, "$tmp/$name.diff" || die "Can't open temp file $name.diff: $!";
$diff .= $_ while <DIFF>;
close DIFF || die "Can't close temp file $name.diff: $!";
unlink "$tmp/$name" || die "Can't remove temp file $name:$!";
unlink "$tmp/$name.new" || die "Can't remove temp file $name.new:$!";
unlink "$tmp/$name.diff" || die "Can't remove temp file $name.diff:$!";
return $diff;
}
sub make_mails {
my $boundary;
my $BCount=0;
opendir (PKGLIST,$home) || die "Cannot read the content of $home: $!";
open OK, ">$home/$desc_ok" || die "Can't create $desc_ok: $!";
foreach (readdir(PKGLIST)) {
next if /\.$todo$/;
next if /\.$rev$/;
next if /\.$new$/;
next if /\.$mail$/;
next if /\.$sent$/;
next if /^\./;
my $pkg = $_;
if (-e "$home/$pkg.$mail") {
my $text = "";
open MAIL, "$home/$pkg.$mail" || die "Can't open $pkg.$mail: $!";
$text .= $_ while <MAIL>;
close MAIL || die "Can't close $pkg.$mail: $!";
print "Here is the mail:\n$text\nDo you want to send it [Y/n] ?\n";
if (!(<> =~ /^[nN]/)) {
rename "$home/$pkg.$mail", "$home/$pkg.$sent";
open SENDMAIL,"| /usr/lib/sendmail -t -oi -oem" || die "Can't run sendmail: $!";
print SENDMAIL $text;
close SENDMAIL || die "Can't run sendmail: $!";
print "Mail sent\n";
} else {
print "Mail not sent\n";
}
next;
}
if (-e "$home/$pkg.$rev" && -e "$home/$pkg") {
if (-e "$home/$pkg.$sent") {
print "$pkg.$sent exists. I won't send the same mail twice\n";
next;
}
if (-e "$home/$pkg.$new") {
print "$pkg.$new exists. rename it to $pkg.$mail to send it\n";
next;
}
$boundary="----------=_".scalar(time)."-$$-".$BCount++;
my($tmp, $translator, $reportid) = &pkg2tmptransid("$home", "$pkg");
# build the mail
my $text = "From: $mail_from\n"
."To: ".($debug>1?$mail_from:$translator)."\n";
$text .= "Cc: $mail_from\n" if ($selfsend eq "yes");
$text .= "Subject: ".sprintf ($mail_title, $pkg)."\n"
."In-Reply-To: $reportid\n"
."Mime-Version: 1.0\n"
."Content-Type: multipart/mixed; boundary=\"$boundary\"\n"
."Content-Disposition: inline\n"
."Content-Transfer-Encoding: $mail_enc\n"
."User-Agent: ddts review helper\n\n\n"
."--$boundary\n"
."Content-Type: text/plain; charset=$mail_charset\n"
."Content-Disposition: inline\n"
."Content-Transfer-Encoding: $mail_enc\n\n"
.$mail_begin;
my($new, $comment) = &rev2newcomment("$home", "$pkg.$rev");
$text .= $comment;
if ($sendnew eq "yes") {
$text .= "\n\n"
."--$boundary\n"
."Content-Type: text/plain; charset=$mail_charset\n"
."Content-Disposition: attachment; filename=\"$pkg.new\"\n\n"
.$new;
}
$text .= "\n\n"
."--$boundary\n"
."Content-Type: text/plain; charset=$mail_charset\n"
."Content-Disposition: attachment; filename=\"$pkg.diff\"\n\n";
if ($tmp eq $new) {
# empty diff
print OK $pkg."\n";
} else {
# puts the diff
$text .= &diff($pkg, $tmp, $new);
# Ends the mime stuff
$text .= "--$boundary--\n\n";
print "Here is the mail:\n$text\nDo you want to send it [Y/n] ?\n";
if (!(<> =~ /^[nN]/)) {
open SENT, ">$home/$pkg.$sent" || die "Can't create $pkg.$sent: $!";
print SENT $text;
close SENT || die "Can't write $pkg.$sent: $!";
open SENDMAIL,"| /usr/lib/sendmail -t -oi -oem" || die "Can't run sendmail: $!";
print SENDMAIL $text;
close SENDMAIL || die "Can't run sendmail: $!";
print "Mail sent\n";
} else {
open MAIL, ">$home/$pkg.$new" || die "Can't create $pkg.$new: $!";
print MAIL $text;
close MAIL || die "Can't write $pkg.$new: $!";
print "Mail saved as $pkg.$new\n";
}
}
}
}
close OK || die "Can't write $desc_ok: $!";
closedir PKGLIST || die "Cannot read the content of $home: $!";
}
# remove $pkg.$rm if $pkg.$e also exists in $path
# print corresponding message
sub test_remove {
my($path, $pkg, $e, $rm)=@_;
if (-e "$path/$pkg.$e" && -e "$path/$pkg.$rm") {
print "Removing $pkg.$rm\n";
unlink "$path/$pkg.$rm" || die "Can't remove $pkg.$rm: $!";
}
}
sub clean {
opendir (PKGLIST,$home) || die "Can't read the content of $home: $!";
foreach (readdir(PKGLIST)) {
&remove_tmp ("$home", "$_", "") if /~$/;
next if /\.$todo$/;
next if /\.$rev$/;
next if /\.$new$/;
next if /\.$mail$/;
next if /\.$sent$/;
next if /^\./;
my $pkg = $_;
&test_remove ("$home", "$pkg", "$rev", "$todo");
&test_remove ("$home", "$pkg", "$rev", "$rev.old");
&test_remove ("$home", "$pkg", "$mail", "$new");
&test_remove ("$home", "$pkg", "$sent", "$mail");
}
closedir PKGLIST || die "Can't close $home: $!";
}
sub statistics {
my $count_pkg=0;
my $count_todo=0;
my $count_relu=0;
my $count_new=0;
my $count_mail=0;
my $count_sent=0;
my $count_ok=0;
my $l=0; # length of longest figure for nice formatting
opendir (PKGLIST,$home) || die "Can't read the content of $home: $!";
foreach (readdir(PKGLIST)) {
next if /\.$todo$/;
next if /\.$rev$/;
next if /\.$new$/;
next if /\.$mail$/;
next if /\.$sent$/;
next if /^\./;
my $pkg = $_;
if ((-e "$home/$pkg.$rev" || -e "$home/$pkg.$todo") && -e "$home/$pkg") {
$count_pkg++;
$count_todo++ if (-e "$home/$pkg.$todo");
$count_new++ if (-e "$home/$pkg.$new");
$count_mail++ if (-e "$home/$pkg.$mail");
$count_relu++ if (-e "$home/$pkg.$rev");
$count_sent++ if (-e "$home/$pkg.$sent");
}
}
closedir PKGLIST || die "Can't close $home: $!";
if (-e "$home/$desc_ok") {
open OK, "$home/$desc_ok" || die "Can't open $desc_ok: $!";
$count_ok++ while (<OK>);
close OK || die "Can't close $desc_ok: $!";
}
$l = $_>$l?$_:$l foreach ($count_pkg, $count_todo, $count_relu, $count_new, $count_mail, $count_sent, $count_ok);
$l = length $l;
printf ("new mail: %${l}d\n", $count_new);
printf (" to send: %${l}d\n", $count_mail);
printf (" sent: %${l}d\n", $count_sent);
printf (" ok: %${l}d\n", $count_ok);
print (" ------"."-"x$l."\n");
printf ("reviewed: %${l}d\n", $count_relu);
printf (" todo: %${l}d\n", $count_todo);
print ("=========="."="x$l."\n");
printf (" total: %${l}d\n", $count_pkg);
}
my $cmd=shift;
if ($cmd eq "parse") {
&parse_report();
} elsif ($cmd eq "mail") {
&make_mails();
} elsif ($cmd eq "clean") {
&clean();
} elsif ($cmd eq "stats") {
&statistics();
} else {
my $me=$0;
$me=~s,^.*?/([^/]*)$,$1,;
die "Usage $me [parse|mail|clean]\n"
." parse: read a ddts from the standard input and change the files in $home\n"
." mail: create the pkg.newmail files which you should edit and send\n"
." clean: remove : \n"
." - *~\n"
." - the $todo when a $rev exists\n"
." - all tmp files (.diff, .new, .tmp)\n"
."\n$me version $version\n";
}
=head1 AUTHORS
Martin Quinson <Martin.Quinson@ens-lyon.fr>: Idea, version until 0.7
Nicolas Bertolissio <nico.bertol@wanadoo.fr>: From version 0.8 to present version
=cut
Reply to: