Bug#625650: www.debian.org: Some UTF-8 characters are broken on WNPP pages

[David Prévot <david@tilapin.org>]

Package: www.debian.org
Severity: wishlist
Tags: patch
User: www.debian.org@packages.debian.org
Usertags: scripts devel

Hi,

HTML::Entities is only Latin1-aware and not able to convert UTF-8
characters, so fails to display properly Kannada script for example:
the title of #625467 is broken in the WNPP pages [0].

  0: http://www.debian.org/devel/wnpp/being_packaged

The following patch needs uni2ascii installed on www-master.

Regards

David

-- System Information:
Debian Release: wheezy/sid
  APT prefers unstable
  APT policy: (600, 'unstable'), (500, 'oldstable'), (500, 'stable'), (150, 'experimental')
Architecture: amd64 (x86_64)

Kernel: Linux 2.6.38-2-amd64 (SMP w/1 CPU core)
Locale: LANG=fr_FR.UTF-8, LC_CTYPE=fr_FR.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

*** /tmp/wnpp.patch
Index: english/devel/wnpp/Makefile
===================================================================
RCS file: /cvsroot/webwml/webwml/english/devel/wnpp/Makefile,v
retrieving revision 1.17
diff -u -r1.17 Makefile
--- english/devel/wnpp/Makefile	17 Aug 2008 12:46:40 -0000	1.17
+++ english/devel/wnpp/Makefile	4 May 2011 19:52:58 -0000
@@ -23,6 +23,8 @@
 $(DATADIR)/wnpp.data: WMLOUTFILE=$(DATADIR)/wnpp.data
 $(DATADIR)/wnpp.data:
 	$(WML) -W7,-Squotes $(DATADIR)/wnpp.pl
+	mv $(DATADIR)/wnpp.data $(DATADIR)/wnpp-utf8.data
+	uni2ascii -a D $(DATADIR)/wnpp-utf8.data > $(DATADIR)/wnpp.data
 
 ifeq "$(LANGUAGE)" "en"
 .PHONY: $(DATADIR)/wnpp.data
Index: english/devel/wnpp/wnpp.pl
===================================================================
RCS file: /cvsroot/webwml/webwml/english/devel/wnpp/wnpp.pl,v
retrieving revision 1.47
diff -u -r1.47 wnpp.pl
--- english/devel/wnpp/wnpp.pl	25 Apr 2011 15:56:55 -0000	1.47
+++ english/devel/wnpp/wnpp.pl	4 May 2011 20:04:14 -0000
@@ -9,17 +9,15 @@
 
 use SOAP::Lite;
 use Date::Parse;
-use HTML::Entities;
-use Encode qw(decode);
 
 # The maintainers flat database
 my $maintainers_file = "$(ENGLISHDIR)/devel/wnpp/Maintainers";
 
 my %maintainer;
-open MAINTAINERS, '<:utf8', $maintainers_file or die "Can't find $maintainers_file file at $host: $!\n";
+open MAINTAINERS, '<', $maintainers_file or die "Can't find $maintainers_file file at $host: $!\n";
 while (<MAINTAINERS>) {
     if (/^(\S+)\s+(.*)$/) {
-	$maintainer{$1} = encode_entities($2);
+	$maintainer{$1} = $2;
     }
 }
 close MAINTAINERS;
@@ -46,7 +44,7 @@
      }
      $age{$bugid} = ($curdate - $status->{$bugid}->{date})/86400;
      $activity{$bugid} = ($curdate - $status->{$bugid}->{log_modified})/86400;
-     $subject = encode_entities($subject);    
+     $subject = $subject;    
      # Make order out of chaos    
      if ($subject =~ m/^(?:ITO|RFA):\s*(\S+)(?:\s+-+\s+)?(.*)$/) {
          $rfa{$bugid} = $1 . ($2?": ":"") . $2;