Bug#796170: lintian: [new check] warn on non-UTF8 text files

[Jakub Wilk <jwilk@debian.org>]

Hi Adam!

Here are some comments from a grumpy Lintian contributor:

* Adam Borowski <kilobyte@angband.pl>, 2015-08-19, 23:43:
> +Tag: text-file-uses-obsolete-encoding

We have already a few tags named *-uses-obsolete-national-encoding, so 
it would be nice if this tag used the same scheme.

Also, please make sure that we don't complain twice about the same file.

> +Severity: normal

I'd say wishlist here.

> + characters (often called "mojibake").  You should convert it to UTF8 using
> + iconv or a similar tool.

iconv should be good for plain text files in /usr/share/doc; but if 
applied blindly to code, or HTML documents or similar, it can cause more 
harm than good.

It would be good to emphasise that the conversion should be done 
upstream, not by "you", the Debian maintainer.

> +Info: The given file is text but uses non-UTF8 encoding.

s/UTF8/UTF-8/, here and elsewhere.

> +            # ---------------- encoding
> +            if (   $fname =~ m{^(?:usr/)?s?bin/}

Also /usr/games? Though I'm not sure what's the rationale for choosing 
these directories...

> +                or $fname =~ m{\.(?:pm|py|pl|txt)$}

... or these extensions.

--
Jakub Wilk