non-latin keyboard with unicode ?

[Drew Parsons <dparsons@emerall.com>]

There's been a bit of discussion lately on the Debian mailing list indicating 
the worthwhileness of Unicode (encoding UTF-8).  I've been trying to get it to
work properly myself.  

Most of my non-english correspondence is with Russians,
and Ilya Martenov helped me get properly configured for the 8-bit cyrillic 
encoding, koi8-r, by setting XkbLayout=ru in the InputDevice section of 
XF86Config-4, and setting LC-CTYPE=ru_RU.KOI8-R.  That's been working fine.

However, I've gotten sick of the odd European accented letter from someone from
Western Europe on the mailing lists being rendered with a cyrillic glyph, which
is of course what happens when you treat higher 8bit characters as Russian.

So what I really want is to use UTF-8 by default (and I'd get a kick out of 
being able to see what the regular korean spam actually looks like, even though
I can't read a word of it ;) )

I've got UTF-8 to display fine by adding UTF-8 to en_AU in /etc/locales.gen, and
setting LANG=en_AU.UTF-8. When I run mutt, I can now see all the languages of 
the world: russian, french, german, korean, etc. in the various emails I've 
received.  Nice.

But the last step is keyboard input.  With koi8-r, I had a spare button on the
keyboard (the MS-Windows button) set to toggle between latin and cyrillic modes.
Now with UTF-8 set, it no longer gives me cyrillic.  Typing at the command line
(this is using "xterm -u8 -fn -fn -misc-fixed-medium-r-normal--13-120-75-75-c-60-iso10646-1")
I see accented latin characters.  Typing here in vi in that same xterm I see
"ÊÃÕËÅÎ", which looks partly correct inside vi (it says " ~J ~C ~U.."
which is a latinised version of the correct cyrillic letters, but outside of vi
the letters turn out to be translated to accented european letters. I guess vi 
isn't setup to use multibyte characters properly yet.  jed (my favourite editor
for mutt) doesn't even handle 8bit characters successfully at the moment (slang
is broken or something). 

Anyway, I'd be happy if I could just see cyrillic letters at the command line 
(I know xterm is working: I can read UTF-8 files using less).  Getting the 
editors to work properly is a another task again.  So when I see the toggle 
button returning me latin letters, it tells me that XkbLayout=ru in the X config
file is getting itself hard-coded to 8bit koi8-r, rather than adapting to the 
cyrillic portion of Unicode when a UTF-8 locale is specified (this happens even
if I specify all LC_CTYPE and LANG = ru_RU.UTF-8, not just en_AU.UTF-8).

So have I missed something?  What does it take to get my keyboard to send
unicode cyrillic characters when using a UTF-8 locale?

Thanks,

Drew

-- 
PGP public key available at http://dparsons.webjump.com/drewskey.txt
Fingerprint: A110 EAE1 D7D2 8076 5FE0  EC0A B6CE 7041 6412 4E4A