Bug#859294: hunspell-dict-ko: FTBFS: UnicodeDecodeError: 'ascii' codec can't decode byte 0xed in position 38: ordinal not in range(128)

[Changwoo Ryu <cwryu@debian.org>]

OK, another encoding= parameter is needed.

--- a/make-aff-dic.py
+++ b/make-aff-dic.py
@@ -200,7 +200,7 @@ class Dictionary:
     def output_aff(self, outfile):
         from string import Template
         import aff
-        template = Template(open('template.aff').read())
+        template = Template(open('template.aff', encoding='utf-8').read())

         # 주의: flag alias를 변경하므로 get_AF() 앞에 와야 한다.
         suffix_str = aff.get_suffix_defines(self.flag_aliases)

Actually I'm the upstream maintainer so it will be applied in the next
upstream revision.


2017-08-28 3:36 GMT+09:00 Adrian Bunk <bunk@debian.org>:
> Control: reopen -1
>
> On Wed, Apr 05, 2017 at 12:53:33PM +0900, Changwoo Ryu wrote:
>> OK I found a way of specifying the encoding of the I/O files.
>>
>>
>> diff --git a/make-aff-dic.py b/make-aff-dic.py
>> index 47dff10..98aba41 100644
>> --- a/make-aff-dic.py
>> +++ b/make-aff-dic.py
>> @@ -308,9 +308,9 @@ if __name__ == '__main__':
>>      dic = Dictionary()
>>      for filename in infilenames:
>>          if filename.endswith('.json'):
>> -            dic.load_json(open(filename))
>> +            dic.load_json(open(filename, encoding='utf-8'))
>>          else:
>>              print('ERROR: unknown file type: ' + filename)
>>              sys.exit(1)
>>      dic.process()
>> -    dic.output(open(afffilename, 'w'), open(dicfilename, 'w'))
>> +    dic.output(open(afffilename, 'w', encoding='utf-8'),
>> open(dicfilename, 'w', encoding='utf-8'))
>>
>>
>> 2017-04-05 12:44 GMT+09:00 Changwoo Ryu <cwryu@debian.org>:
>> > It only fails on a legacy non-Unicode locale, such as "C".
>> >
>> > So what is a solution? Specifying a UTF-8 locale and Build-Depending
>> > on locales-all do not seems to be a right way.
>
> Unfortunately it is still failing (this time in a different place?):
>
> https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/hunspell-dict-ko.html
>
> ...
> make[1]: Entering directory '/build/1st/hunspell-dict-ko-0.6.3'
> python3 make-aff-dic.py ko.aff ko.dic dict-ko-builtins.json
> dict-ko-galkwi-ccby.json dict-ko-galkwi-mplgpllgpl.json
> Progress: \ubcf5\uc218\ud615 \ud655\uc7a5...
> Progress: \ud50c\ub798\uadf8 \uacc4\uc0b0...
> Progress: \ubcf4\uc870\uc6a9\uc5b8 \ud655\uc7a5...
> Progress: dic \ucd9c\ub825...
> Progress: aff \ucd9c\ub825...
> Traceback (most recent call last):
>   File "make-aff-dic.py", line 333, in <module>
>     open(dicfilename, 'w', encoding='utf-8'))
>   File "make-aff-dic.py", line 181, in output
>     self.output_aff(afffile)
>   File "make-aff-dic.py", line 203, in output_aff
>     template = Template(open('template.aff').read())
>   File "/usr/lib/python3.5/encodings/ascii.py", line 26, in decode
>     return codecs.ascii_decode(input, self.errors)[0]
> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position
> 423: ordinal not in range(128)
> Makefile:28: recipe for target 'ko.aff' failed
> make[1]: *** [ko.aff] Error 1
> make[1]: Leaving directory '/build/1st/hunspell-dict-ko-0.6.3'
> dh_auto_build: make -j1 returned exit code 2
> debian/rules:5: recipe for target 'build' failed
> make: *** [build] Error 2
>
>
> cu
> Adrian
>
> --
>
>        "Is there not promise of rain?" Ling Tan asked suddenly out
>         of the darkness. There had been need of rain for many days.
>        "Only a promise," Lao Er said.
>                                        Pearl S. Buck - Dragon Seed
>