UTF-8 problem in db/dak interaction

[Tollef Fog Heen <tfheen@varnish-software.com>]

Hi all,

(please Cc me on replies, I'm not subscribed)

with current git maste and a .changes file with an ø in it, I get the
following:

dak@trident:/srv/dak/queue/new$ dak process-new varnish-agent_1.16.0_source.changes 
Traceback (most recent call last):
  File "/srv/dak//bin/dak", line 243, in <module>
    main()
  File "/srv/dak//bin/dak", line 223, in main
    module.main()
  File "/srv/dak.git/dak/process_new.py", line 807, in main
    print "\n" + os.path.basename(upload.changes.changesname)
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/attributes.py", line 168, in __get__
    return self.impl.get(instance_state(instance),dict_)
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/attributes.py", line 453, in get
    value = self.callable_(state, passive)
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/strategies.py", line 541, in _load_for_state
    return q._load_on_ident(ident_key)
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2342, in _load_on_ident
    return q.one()
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2014, in one
    ret = list(self)
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2171, in instances
    fetch = cursor.fetchall()
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2991, in fetchall
    l = self.process_rows(self._fetchall_impl())
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2958, in _fetchall_impl
    return self.cursor.fetchall()
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 15: ordinal not in range(128)

The .changes is (sans signature):

Format: 1.8
Date: Wed, 29 Aug 2012 16:08:58 +0200
Source: varnish-agent
Binary: varnish-agent
Architecture: source
Version: 1.16.0
Distribution: wheezy
Urgency: low
Maintainer: Lasse Karstensen <lasse@varnish-software.com>
Changed-By: Kristian Lyngstøl <kristian@varnish-software.com>
Description: 
 varnish-agent - Agent used to control Varnish on behalf of the Varnish Admin Cons
Changes: 
 varnish-agent (1.16.0) wheezy; urgency=low
 .
   [ Kristian Lyngstøl ]
   * Fix heredocs detection to catch dashes too
   * Jenkins build #15 ID: 2012-11-15_12-50-29
Checksums-Sha1: 
 877042dee03c6fae278138efada0fded685718d3 1423 varnish-agent_1.16.0.dsc
 b1dbf0fb18cb7b2d85f1382a46071b40f79c8a84 26051 varnish-agent_1.16.0.tar.gz
Checksums-Sha256: 
 68e1239c402d636ad51246fecaca7309de3d159da13c7d11aece072aeb9938e3 1423 varnish-agent_1.16.0.dsc
 97c08bc7f141f13d0634f5541f6e506cbd544e05285f0651e2fe9e46a65d53a5 26051 varnish-agent_1.16.0.tar.gz
Files: 
 b08cf091a8e2863cee7c21f7a3f2d920 1423 web extra varnish-agent_1.16.0.dsc
 0eeaf8206208ad542955b9dc3ef9caea 26051 web extra varnish-agent_1.16.0.tar.gz

I wonder if this might be related to the db being SQL_ASCII:
 projectb  | dak      | SQL_ASCII | en_DK.UTF-8 | en_DK.UTF-8 | 

however, I see projectb on franck is SQL_ASCII too.

Anybody know what's up here, and how to fix it?

Cheers,
-- 
Tollef Fog Heen
Technical lead | Varnish Software AS
t: +47 21 98 92 64
We Make Websites Fly!