Error in dead key management of latin keyboards

[Sharcoux]

Composition events are incorrect in Ubuntu 22.04 for latin keyboard

I opened an issue at Firefox [here](https://bugzilla.mozilla.org/show_bug.cgi?id=1802989) that gives all the details, but they diagnosed a bug in ubuntu-ibus

To reproduce the issue, you can go [here](https://developer.mozilla.org/en-US/docs/Web/API/Element/compositionstart_event#result) and type ^+u with a fr+latin9 keayboard configuration.

Here is the conclusion on the bug report:

Ubuntu-ibus commits dead key sequence with empty string first, then, sends only commit string without "composing" state. The latter behavior is a usual case for inputting non-ASCII characters like Emojis. In this case, browsers need to handle it without composition for compatibility between browsers. Therefore, we cannot represent the composed character insertion as a set of composition events. Anyway, dispatching 2 set of composition events for one dead key sequence may cause trouble in web apps. Therefore, as expected, the first composition should update the "preedit string" to the composed string and commit it simply.

The bug was reproduced with installing Ubuntu 22.04 with English locale and adding "French (alt., Latin-9 only)" keyboard layout. And also reproduced with the other Western keyboard layouts which have dead keys.

[Gunnar Hjalmarsson (gunnarhj)]

Thanks for your report. I can't reproduce it, though.

Whether the layout is "French (alt., Latin-9 only)" or "Swedish" I can type the dead_circumflex symbol followed by u, and get û. For me that works also with Firefox.

Which physical keys are you using to type dead_circumflex? To access that symbol on "French (alt., Latin-9 only)" I need to hold <AltGr>+<Shift> while pressing the key to the very right at the third row (to the left of the Return key).

[Gunnar Hjalmarsson (gunnarhj)]

I also tested on Xubuntu 22.10, which I happen to have access to. The reason is that I there (unlike on GNOME) can disable ibus completely.

Using the Mozilla tool you linked to, and with ibus disabled, I get:

compositionstart:
compositionend:

in the event log (even if the û character is shown as expected).

Then I enabled ibus, and got this in the event log:

compositionstart:
compositionupdate: ^
compositionupdate: û
compositionend: û

So ibus is indeed involved somehow, but not (for me) in a way that results in an issue.

[Sharcoux (darksharcoux)]

Here is a layout looking almost like mine: https://en.wikipedia.org/wiki/AZERTY#/media/File:KB_France.svg

I press the key which is to the right to `p` and to the left of `$ ` and do not need to hold AltGr nor Shift.

If you run `gkbd-keyboard-display -l latin-9`, you should see my layout. It doesn't look like what you describe.

Don't hesitate if I can help more.

[Gunnar Hjalmarsson (gunnarhj)]

Are you using some layout which is not provided by xkb-data?

The one I used is named "French (alt., Latin-9 only)" and is represented by the dconf value fr+oss_latin9. The dconf value fr+latin9 is also available, but the name of that layout is "French (legacy, alt.)".

I'm not sure about the significance of which particular layout is used, though. But I'm wondering in what way your observation results in unexpected behavior for the user. Yes, there are differences under the hood. But how is that important in practice?

[Sharcoux (darksharcoux)]

I'm a bit confused about your questions related with xkb-data but I hope that this will help: here is the result of `setxkbmap -print`:

xkb_keymap {
 xkb_keycodes { include "evdev+aliases(azerty)" };
 xkb_types { include "complete" };
 xkb_compat { include "complete" };
 xkb_symbols { include "pc+fr(latin9)+fr(oss):2+inet(evdev)" };
 xkb_geometry { include "pc(pc105)" };
};

And `setxkbmap -query`:

rules: evdev
model: pc105
layout: fr,fr
variant: latin9,oss

About the consequences for users, well, the source of all this is the breaking of compositionupdate and compositionend events for those users on Firefox. My app relies on compositionend events, but, as those events were not created by myself, I guess that my app is not the only program relying on those events. If those events are not accurate, it will break any program relying on it.

[Gunnar Hjalmarsson (gunnarhj)]

On 2022-12-13 19:40, Sharcoux wrote:
> layout: fr,fr
> variant: latin9,oss

Then it's the "French (legacy, alt.)" layout you use. And yes, with that layout you get dead_circumflex by simply pressing the key to the right of the P key. (But I think the layout used to make use of some dead key doesn't really matter.)

The good news is that I have figured out a way to reproduce the behavior you state as "current result" in your upstream Firefox bug. I simply removed all my IBus sources from the list of input sources and relogged in. Then I get:

compositionstart:
compositionupdate: ^
compositionupdate:
compositionend:

So it would be interesting to know if you can do the opposite. Assuming you are on a standard Ubuntu 22.04 with GNOME, can you please

* install some ibus input method — suggestion: the ibus-libpinyin package

* relogin

* go to Settings -> Keyboard and add "Intelligent Pinyin" to your input sources

* relogin again

I did those steps myself, and now I'm back with this result:

compositionstart:
compositionupdate: ^
compositionupdate: û
compositionend: û

[Sharcoux (darksharcoux)]

Hum... I'm sorry, I might be doing something wrong.

I did: `sudo apt install ibus-libpinyin`
I restarted the computer
I went to Settings => Keyboard, and in "Input sources" I pressed the "+", but there I could only chose between English or French variations. If I press "Others", I just see more English or French alternatives.

What am I missing?

[Gunnar Hjalmarsson (gunnarhj)]

That UI is a three step thing. From the first view you should click "More..." (three vertically aligned dots), in the second view you should scroll to the bottom and there click "Other". Then you ought to see a very long list, including the one I mentioned.

[Sharcoux (darksharcoux)]

Ok, I can confirm that it works... It seems weird to me that it works, but it just works.

[Gunnar Hjalmarsson (gunnarhj)]

Good. Then let me try to explain the nature of the problem you encountered.

It's not really an ibus issue. But GNOME integrates ibus into its desktop, and this is about the way they do so.

ibus is started and configured by default in GNOME desktops. But for some reason they have an on/off mechanism for GTK, so for a user who does not have any ibus input methods enabled (and does not make use of the on-screen keyboard) ibus is turned "off" for GTK applications. By adding "Intelligent Pinyin" to your list of input sources you turned ibus "on" and it started to work as expected.

Can't tell why you don't see the issue with Chrome. Maybe Chrome somehow ignores the GNOME on/off thing.

At the release of Ubuntu 22.04 there was a related issue, which I reported upstream:

https://gitlab.gnome.org/GNOME/gnome-settings-daemon/-/issues/682

I mention that because Carlos Garnacho's reply includes a description of how the mechanism is thought to work.

I can't help wondering how this relates to your app. In a previous comment you let me know that your "app relies on compositionend events". But if I understand it correctly, "compsitionend" is only used that way if ibus is running. Is your app an ibus app, but it fails to tell the system? Or is it a non-ibus app which still is relying on an ibus specific mechanism with respect to dead keys composing?

Have you tried to run your app on a desktop where ibus is not present at all?

[Sharcoux (darksharcoux)]

My app is a React app. It contains an advanced text editor. To develop it we needed to catch the input events made on a contentEditable div to update an internal model. So we need to be aware about any change made to the editor. As the compositionend event is wrong, it generates a difference between our internal model and the current html of the page.

I'm not sure about what you mean by an "ibus app". And how composition events are "ibus specific" mechanism? They are event documented in the HTML5 W3C standard : https://w3c.github.io/uievents/#interface-compositionevent

Is there something I should do in my app that would enable the correct mechanism?

I'm not sure neither about your last question. What would be a desktop where ibus is not present at all?

[Gunnar Hjalmarsson (gunnarhj)]

On 2022-12-20 21:39, Sharcoux wrote:
> I'm not sure about what you mean by an "ibus app". And how
> composition events are "ibus specific" mechanism? They are event
> documented in the HTML5 W3C standard :
> https://w3c.github.io/uievents/#interface-compositionevent

I'm certainly not an expert on the topic. At this point I just observe. And when testing, using your test case ^+u, I only see the expected sequence of composition* events if ibus is up and running. Btw, you saw it yourself when it started to work for you on a GNOME desktop only when you had enabled an ibus input method.

I should add that also Chrome requires ibus to generate the expected sequence of composition* events; on a desktop where ibus is not present (or completely disabled) you won't see the expected sequence.

Maybe GTK itself isn't standards compliant in this respect.

> Is there something I should do in my app that would enable the
> correct mechanism?

I don't know. In the upstream issue I mentioned, there are a couple of workarounds which might help. But that would just be a replacement for enabling an ibus input method. And in case of Wayland they would be poor workarounds with adverse side effects.

> I'm not sure neither about your last question. What would be a
> desktop where ibus is not present at all?

Most non-Ubuntu desktops, actually. :) One example is Xubuntu, which I used when testing. Another option is to simply uninstall ibus on your Ubuntu desktop.

[Launchpad Janitor (janitor)]

[Expired for gnome-settings-daemon (Ubuntu) because there has been no activity for 60 days.]