Bug#859294: hunspell-dict-ko: FTBFS: UnicodeDecodeError: 'ascii' codec can't decode byte 0xed in position 38: ordinal not in range(128)

[Adrian Bunk <bunk@debian.org>]

Control: reopen -1

On Wed, Apr 05, 2017 at 12:53:33PM +0900, Changwoo Ryu wrote:
> OK I found a way of specifying the encoding of the I/O files.
> 
> 
> diff --git a/make-aff-dic.py b/make-aff-dic.py
> index 47dff10..98aba41 100644
> --- a/make-aff-dic.py
> +++ b/make-aff-dic.py
> @@ -308,9 +308,9 @@ if __name__ == '__main__':
>      dic = Dictionary()
>      for filename in infilenames:
>          if filename.endswith('.json'):
> -            dic.load_json(open(filename))
> +            dic.load_json(open(filename, encoding='utf-8'))
>          else:
>              print('ERROR: unknown file type: ' + filename)
>              sys.exit(1)
>      dic.process()
> -    dic.output(open(afffilename, 'w'), open(dicfilename, 'w'))
> +    dic.output(open(afffilename, 'w', encoding='utf-8'),
> open(dicfilename, 'w', encoding='utf-8'))
> 
> 
> 2017-04-05 12:44 GMT+09:00 Changwoo Ryu <cwryu@debian.org>:
> > It only fails on a legacy non-Unicode locale, such as "C".
> >
> > So what is a solution? Specifying a UTF-8 locale and Build-Depending
> > on locales-all do not seems to be a right way.

Unfortunately it is still failing (this time in a different place?):

https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/hunspell-dict-ko.html

...
make[1]: Entering directory '/build/1st/hunspell-dict-ko-0.6.3'
python3 make-aff-dic.py ko.aff ko.dic dict-ko-builtins.json 
dict-ko-galkwi-ccby.json dict-ko-galkwi-mplgpllgpl.json 
Progress: \ubcf5\uc218\ud615 \ud655\uc7a5...
Progress: \ud50c\ub798\uadf8 \uacc4\uc0b0...
Progress: \ubcf4\uc870\uc6a9\uc5b8 \ud655\uc7a5...
Progress: dic \ucd9c\ub825...
Progress: aff \ucd9c\ub825...
Traceback (most recent call last):
  File "make-aff-dic.py", line 333, in <module>
    open(dicfilename, 'w', encoding='utf-8'))
  File "make-aff-dic.py", line 181, in output
    self.output_aff(afffile)
  File "make-aff-dic.py", line 203, in output_aff
    template = Template(open('template.aff').read())
  File "/usr/lib/python3.5/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 
423: ordinal not in range(128)
Makefile:28: recipe for target 'ko.aff' failed
make[1]: *** [ko.aff] Error 1
make[1]: Leaving directory '/build/1st/hunspell-dict-ko-0.6.3'
dh_auto_build: make -j1 returned exit code 2
debian/rules:5: recipe for target 'build' failed
make: *** [build] Error 2


cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed