[xkb] new japanese layout

[evan]

Short Description: Group switching between multiple layout does not work
correctly when one of them is the Japanese layout (jp)

System: Ubuntu Hoary on a Japanese Toshiba Dynabook V7/513LMDW Laptop

Packages concerned (I think): xlibs, libxkbfile1, or libxkbui1

When I set up my /etc/X11/xorg.conf config file to handle ONLY the Japanese
layout (I have a Japanese keyboard), it works perfectly:

Section "InputDevice"
    Identifier "Generic Keyboard"
    Driver "keyboard"
    Option "CoreKeyboard"
    Option "XkbRules" "xorg"
    Option "XkbModel" "pc106"
    Option "XkbLayout" "jp"
    Option "XkbOptions" "compose:ralt,grp:shift_caps_toggle"
EndSection

HOWEVER, I am french so I like to have also a french layout (yes, for the
accents), so I tried to replace the "XkbLayout" option from "jp" to "fr,jp".
Then when X starts, the french layout is loaded and works fine. But switching
the layout does not switch to the correct Japanese layout, but to one where only
"Katakana" can be input.

The correct behavior should be to switch to the "Roman letters version" of the
Japanese layout, which is the standard and the one that is widely used by the
Japanese.

I also tried to solve this by using the Gnome Keyboard Applet, but here also,
when I choose Japanese plus any other language, the Japanese layout is not
correct, while it sets up the correct layout when I choose only Japanese.

A workaround that I have found to handle this is to use the program "setxkbmap"
to switch layouts using a terminal:
The two following commands allow a perfect change of the layout, but it should
be handled correctly using group switching.
setxkbmap -rules xorg -model pc106 -layout fr -option ""
setxkbmap -rules xorg -model pc106 -layout jp -option ""

[Daniel Stone (daniels)]

Denis, any ideas?

[Denis Barbier (barbier)]

The jp layout has 2 variants: jp106 and latin.
When only jp is selected, this is equivalent to
  pc/pc(pc102)+pc/jp(latin)+pc/jp:2
(defined in rules/xorg), so latin variant is the 1st group and
jp106 the 2nd (since this one is the default). When jp is
combined with another layout, only jp106 is selected.

There are at least 2 solutions:
 a. In symbols/pc/jp, switch hidden and default attributes for
    jp106 and latin, and move name[Group1] to the latin variant.
 b. In symbols/pc/jp, remove the hidden attribute from latin,
    set another name[Group1] in this variant, and update
    rules/xorg.xml to define both variants so that they can be
    selected with the keyboard switcher applet.

With (a), katakana cannot be written, I do not know whether
this is a problem. With (b), both variants can be selected,
so it looks better, but requires patching rules/xorg.xml.

[evan (evan-monroig)]

(In reply to comment #2)

> With (a), katakana cannot be written, I do not know whether this is a problem.

I don't think that it is a problem, because what is commonly used to input
Japanese is UIM/Anthy or Kinput2/Canna (Both methods include the Katakana) as
described in the wiki http://www.ubuntulinux.org/wiki/JapaneseInputHowto or
http://www.ubuntulinux.org/wiki/JapaneseDocumentation.The katakana input is
pretty limited, and it seems that even the Japanese people don't use it.

Hope this helps.

[Denis Barbier (barbier)]

(In reply to comment #3)
You are right, but I now realize that (a) defines a keyboard
which is primarily designed for foreigners, so I feel quite
uncomfortable with it unless Japanese people explain that
they also like this change.

[evan (evan-monroig)]

(In reply to comment #4)
I don't have any news from Japanese linux users, but you are right about (a). I
think that the reason this hidden attribute was introduced in the first place
was to allow the Japanese to automatically have the two layouts simply by
choosing the "japanese" layout. But this approach loses its interest when having
multiple layouts. So in my opinion the best option would be (b).

Is there a Japanese Ubuntu Community?

[Alfonso (alf-phaser)]

(In reply to comment #5)
> Is there a Japanese Ubuntu Community?
After a few years living here in Japan I can say that there isn't. The main
distros used here are TurboLinux and Vine Linux, however I could be wrong, of
course. Ubuntu is relatively young and for that reason it's hard to tell. I also
think solution (a) is quite safe. I've never known of any Japanese that doesn't
use the "foreign" way (that is, romaji phonetic equivalence) to input Japanese
in a Linux distro or any other OS, either computer savvy or just desktop user
(email, browsing...).
I've noticed that with Spanish/Japanese layouts the Spanish one is a bit
different comparing it to Spanish only. The Hankaku/Zenkaku key, which is mapped
to the ordmasculine, ordfemenine (Shift) and backslash (Alt Gr) can only produce
the ordmasculine character. Is this also related? Does anyone know if the jp
symbols file is causing this behaviour?

[Daniel Stone (daniels)]

is this better now in breezy?

[evan (evan-monroig)]

(In reply to comment #7)
> is this better now in breezy?

I upgraded to breezy to check for this, and unfortunately the behaviour is the same.

[evan (evan-monroig)]

As I really need this to work I set up to modify the files as described in the
part (b) of comment #2 by Denis.

I have lived in Japan for 18 months, and never met anyone using the katakana
keyboard layout. All the Japanese people I know now use "smarter" conversion
engines like anthy or canna for linux, microsoft ime, etc. which analyse roman
letters as they are typed and automatically convert them to hiragana/katakana
and then to kanji when we press spacebar.

For this reason I think solution (b) from comment #2 is the best, as it leaves
the possibility of choosing between the two.

Now I am not a Japanese linux user, and I don't know anything at all about xkb
and I never participated in any development under linux, so I might overlook
important issues, but here is my proposition for two patches of the files
/etc/X11/xkb/symbols/jp and /etc/X11/xkb/rules/base.xml

I am using this on my computer, so given the small change that it is, it should
work for others as well.

Hope this will help ^^

Evan

ps: I am on an updated breezy with the following xkb package versions:
libxkbfile-dev 7.0.0-2
libxkbfile1 7.0.0-2
libxkbui1 7.0.0-2
xkbutils 7.0-3

[evan (evan-monroig)]

Created an attachment (id=3686)
patch for symbols/jp

patch for /etc/X11/xkb/symbols/jp

[evan (evan-monroig)]

Created an attachment (id=3687)
patch for rules/base.xml

patch for /etc/X11/xkb/rules/base.xml

[evan (evan-monroig)]

Oh. As expected the keys when I type with the japanese (or french) layout
produce the letters or symbols that they should produce, but:

I cannot use "shift+space" to activate uim for japanese input anymore.

However, scim (which has "control+space" to activate the alternate input) works...

So maybe these changes were not enough (I'm too much of a development newbee,
hehe). What did I miss?

[reh4c (gene-hoffler)]

Using Dapper Flt 5, I chose a 106-key Japanese keyboard from the installation options. Then, during the installation process, I chose a Japanese keyboard from the list during the setup steps. From what I remember, there was only 1 choice named "Japanese," so I selected it. Finally, after Dapper finishes installing, the default keyboard is 104 intl pc, instead of a Japanese keyboard (either 105 or 106 key). Then, I try to tell Dapper under System-Preferences-Keyboard that my Keyboard Model is "Japanese 106-key," but this doesn't appear to change anything. Keep in mind that I want to use English characters and international symbols on a Japanese 106-key compatible laptop keyboard that has standard Japanese characters and English letters. I want the default language to remain English. For example, when hit "shift" and "2", I should see a " displayed, not @. So, I guess there are 2 problems here.
1. Setting the default keyboard to 106-key Japanese during the installation and having this setting remain in effect when the installation completes
2. Selecting a 106-Japanese keyboard correctly aligns what's on the keys with what is displayed as typing output on the screen.

From what I remember, I did not have this problem in Breezy. If I changed the keyboard model to Japanese 106-key, the output from typing matched what appeared on the screen. Finally, don't you think the terminology used for Keyboard Preferences is confusing? I really don't understand the difference between keyboard model and keyboard layout. Is keyboard layout actually the language selector? Also, isn't a 106-key keyboard an actual layout vice a model type??? Thanks.

[reh4c (gene-hoffler)]

I seem to have found the real problem with using my 106-key Japanese keyboard. Although I change my keyboard model to 106-key under Keyboard model in System-Preferences-Keyboard, the change is not written to xorg.conf. Please see the attached file. Please fix this problem to solve my problem.

[Corsaire01 (stephane-tranchemer)]

hello all,

I did saw the problem too on my computer, I have a Ubuntu 6.06 install in french with a french keyboard.
When I plug my spare japanese USB keyboard and I switch keyboard layout with gnome-keyboard-properties I am able to only input katakanas and alphabet (when using the switching keys).

On this regard, I found the keyboard layout being very incomplete.
When in gnome-keyboard-properties you have a look at the available layouts, you can see a LOT of keyboards with all their signs written on (have a look at latina america, bosnia or belguim), but the japanese one only revels alphabet signs.
There are keys missing too like at the left and the right of the space key.

Here I attach a photo-mount of my keyboard so that you can compare with what the gnome-keyboard-properties app shows.

[Bryce Harrington (bryce)]

These patches seem to no longer apply properly to xkeyboard-config. If this issue is still relevant in Hardy, could someone provide an updated patch to the package and reopen?

[Timo Aaltonen (tjaalton)]

Or better yet, post also to the upstream bugtracker which is at bugs.freedesktop.org. Then we can monitor the progress.