Switch to Fcitx 5 for Chinese

[Gunnar Hjalmarsson]

In Debian 11 Fcitx 5 will be the default IM framework for Chinese on non-GNOME desktops. I can think it's time to make the equivalent changes in Ubuntu 21.10 as well.

I'd appreciate input on the topic from the Ubuntu Kylin team as well as other Chinese speaking users.

[Gunnar Hjalmarsson (gunnarhj)]

Assuming that this may require apparmor changes, I added an apparmor (Ubuntu) bug task.

[Gunnar Hjalmarsson (gunnarhj)]

It struck me that Lubuntu, which ships Fcitx by default, has a special interest in this change, so added a Lubuntu bug task.

[Gunnar Hjalmarsson (gunnarhj)]

I submitted a merge proposal with suggested changes to language-selector (pkg_depends). Please review and give feedback.

[Gunnar Hjalmarsson (gunnarhj)]

As regards apparmor it's possible that no change is needed. fcitx5 creates the folder

~/.config/fcitx/dbus

(i.e. not ~/.config/fcitx5/dbus) with a long file name whose contents is "unix:path=/run/user/1000/bus". Not sure how to safely confirm that, though.

[Gunnar Hjalmarsson (gunnarhj)]

On 2021-05-16 22:23, Gunnar Hjalmarsson wrote:
> As regards apparmor it's possible that no change is needed.

Well, I simply tested with the Chromium snap. fcitx5 does not work in Chromium, while fcitx4 does. So something needs to be done.

[Seth Arnold (seth-arnold)]

On Tue, May 18, 2021 at 07:39:48PM -0000, Gunnar Hjalmarsson wrote:
> On 2021-05-16 22:23, Gunnar Hjalmarsson wrote:
> > As regards apparmor it's possible that no change is needed.
>
> Well, I simply tested with the Chromium snap. fcitx5 does not work in
> Chromium, while fcitx4 does. So something needs to be done.

Excellent, can you paste the DENIED lines from your test into the bug
report?

Thanks

[Gunnar Hjalmarsson (gunnarhj)]

Sure, Seth, attached please find the output from

cat /var/log/audit/audit.log | grep DENIED

[handsome_feng (feng-kylin)]

Hi, Gunnar,

After talking to our partners such as Sogou, they want to postpone this migration until 22.04 so that they have enough time to work on the adaptation, is this possible?

Thanks,
handsome_feng

[Gunnar Hjalmarsson (gunnarhj)]

@handsome_feng: I suppose it is possible. Of course it would have been desirable to do it in 21.10 to get more time to identify possible issues before next LTS, but OTOH I know that Sogou is considered important on Kylin.

Thinking... Maybe we can have it both ways. :) What I mean is that we could do it now for other flavors, but tell language-selector to not pull fcitx5 packages on Kylin for now. im-config will keep supporting both fcitx 4 and 5, and then we would still have reasons to identify and fix fcitx5 issues with the apparmor profiles in the 21.10 development cycle.

Would that plan make sense to you?

[handsome_feng (feng-kylin)]

> Would that plan make sense to you?

This is a good way for us, we agree with it, thanks a lot!

[Gunnar Hjalmarsson (gunnarhj)]

@handsome_feng: Great! I have a follow-up question then:

In the desktop seed of Ubuntu Kylin I see these packages:

 * (im-config)
 * (fcitx)
 * (fcitx-frontend-gtk2)
 * (fcitx-frontend-gtk3)
 * (fcitx-frontend-qt5)
 * (fcitx-ui-classic)
 * (fcitx-table)
 * (fcitx-module-cloudpinyin)
 * (fcitx-googlepinyin)

Given that, will it work with the proposed version of pkg_depends, or is there a reason to keep letting also pkg_depends pull a bunch of fcitx 4 packages for Kylin in 21.10?

[handsome_feng (feng-kylin)]

Hi,

With the fcitx items in the current seed, I think pkg_depends don't need to pull in the fcitx 4 packages for Ubuntu Kylin 21.10, also it's okay to pull in the fcitx 5 packages, since the im-config will set the default input method to fcitx 4 for Ubuntu Kylin 21.10?

[Gunnar Hjalmarsson (gunnarhj)]

On 2021-05-20 15:39, handsome_feng wrote:
> With the fcitx items in the current seed, I think pkg_depends don't
> need to pull in the fcitx 4 packages for Ubuntu Kylin 21.10,

Ok, good.

> also it's okay to pull in the fcitx 5 packages, since the im-config
> will set the default input method to fcitx 4 for Ubuntu Kylin 21.10?

Hmm.. Yes, AFAIK that would work. fcitx 4 and fcitx 5 can co-exist, and if both are installed im-config will pick fcitx 4 by default (can be changed by the user via Language Support).

Does this mean that the proposed changes to pkg_depends is fine from an Ubuntu Kylin POV, and that no further changes are needed?

[handsome_feng (feng-kylin)]

> Does this mean that the proposed changes to pkg_depends is fine from
> an Ubuntu Kylin POV, and that no further changes are needed?

Yes, Thanks!

[Gunnar Hjalmarsson (gunnarhj)]

Thanks handsome_feng!

The Kylin discussion makes me confident that it's fine for Lubuntu as well. The Lubuntu situation is different, though, since im-config is disabled by default unless a Chinese locale is in effect. This discourse topic comes to mind:

https://discourse.ubuntu.com/t/default-im-config-configuration/17136

I'm still interested in talking with some Lubuntu developer on that. But the outcome of such a discussion would affect other packages but language-selector, so I just pushed and uploaded the proposed change.

[Launchpad Janitor (janitor)]

This bug was fixed in the package language-selector - 0.212

---------------
language-selector (0.212) impish; urgency=medium

  * data/pkg_depends:
    - Don't pull Fcitx for non-Chinese languages. This is consistent
      with the IM_CONFIG_PREFERRED_RULE variable in
      /etc/default/im-config.
    - Pull IBus IMs for Chinese also on Ubuntu Budgie
    - Switch to Fcitx 5 for Chinese (LP: #1928360)

 -- Gunnar Hjalmarsson <email address hidden> Fri, 21 May 2021 06:41:05 +0200

[Gunnar Hjalmarsson (gunnarhj)]

@Seth: I suppose that the DENIED file I attached in comment #7 does not help much.

Fcitx 5 is a separate process, i.e. /usr/bin/fcitx5 as opposed to /usr/bin/fcitx. When in Chromium it behaves as if fcitx5 does not exist, even if the process runs and I can use fcitx5 in other applications. I can't even switch to a fcitx5 input method engine when in Chromium, let alone use such an input engine for typing. Maybe that explains why nothing fcitx5 related is reflected as DENIED lines.

I'm thinking that maybe there is a need to create the fcitx5 and fcitx5-strict abstractions using the already existing fcitx and fcitx-strict abstractions as models.

[Seth Arnold (seth-arnold)]

Gunnar, indeed, it had much less in it than I expected; I don't know much about the snap packaging for Chromium, but it looked to me like it was trying to do bluetooth things and that's all that was denied.

I'm no fcitx expert but I didn't think it looked related.

Thanks

[Gunnar Hjalmarsson (gunnarhj)]

I presented the snap problem for James Henstridge, and he gave me a very useful hint: The Chromium snap makes use of the bionic library stack (the gnome-3-28-1804 snap) including the im module installed by fcitx-frontend-gtk3, i.e. the fcitx 4 gtk im module.

That made me think of <https://bugs.debian.org/977203>. That bug is still open, and as a result im-config currently sets these variables for fcitx 5:

GTK_IM_MODULE=fcitx5
XMODIFIERS=@im=fcitx
QT_IM_MODULE=fcitx5

It means that it explicitly looks for the fcitx 5 gtk im module, while Chromium only has access to the fcitx 4 one.

So as a test I tweaked im-config to set:

GTK_IM_MODULE=fcitx
XMODIFIERS=@im=fcitx
QT_IM_MODULE=fcitx

and that made a difference. With that environment I could successfully input beautiful Chinese characters in Chromium using fcitx 5. :)

Consequently there seems to be no need to add any extra stuff to apparmor to handle fcitx 5. Instead I see two options:

1. Change im-config's variable assignment, or

2. somehow add the fcitx 5 im modules for gtk and qt to the snap library stack.

I'm going to consult with Debian's input method team about which route is preferable.

@James: Do you see a problem from your POV with adding the fcitx 5 im modules to the snaps?

[Seth Arnold (seth-arnold)]

On Sat, Jun 05, 2021 at 01:27:19AM -0000, Gunnar Hjalmarsson wrote:
> It means that it explicitly looks for the fcitx 5 gtk im module, while
> Chromium only has access to the fcitx 4 one.

Oh! Excellent debugging to find the root cause.

Thanks

[James Henstridge (jamesh)]

I hadn't realised that they renamed the IM module with the version bump: that means snaps built against older platforms won't load the old (but working) Fcitx 4 module when run on a host system configured with Fcitx 5.

We can certainly look at adding the new IM module to the platform snaps, but that will also switch to the new IPC protocol. From a quick look through the source it will probably be allowed by existing AppArmor rules in the "desktop" interface, but that will need testing.

It's probably going to be easiest to see if we can get things working with a core20 based platform snap though, since we have fcitx5 packages there (albeit for a git prerelease version).

For the core18 based platform snaps, we would need to backport the fcitx5 packages themselves. Alternatively, I wonder if we could get away with symlinking the old module name to the new one?

[Gunnar Hjalmarsson (gunnarhj)]

On 2021-06-05 06:32, James Henstridge wrote:
> It's probably going to be easiest to see if we can get things
> working with a core20 based platform snap though, since we have
> fcitx5 packages there (albeit for a git prerelease version).

Is there such a snap you could point me to so I can test?

> For the core18 based platform snaps, we would need to backport the
> fcitx5 packages themselves. Alternatively, I wonder if we could get
> away with symlinking the old module name to the new one?

The latter would be equivalent to assign "fcitx" instead of "fcitx5" to the IM_MODULE variables if I understand it correctly.

The Fcitx 5 folks upstream recommend "fcitx" for IM_MODULE. The reason why Debian/Ubuntu currently set "fcitx5" is to deal with systems during a transition period where both fcitx 4 and fcitx 5 are installed. In such cases "fcitx" would result in the fcitx 4 im modules being loaded also when the user runs fcitx 5, and we have assumed that that might be problematic.

OTOH, my successful test with Chromium indicates that it works. Now I'm not the right person to evaluate it, since I'm not an input method user, and that's what I want to talk with the Chinese developers in Debian's input method team about.

@handsome_feng: Maybe the Kylin team can help to test this, considering that Kylin will be shipped with both fcitx 4 and fcitx 5 in 21.10. If you override im-config and enforce GTK_IM_MODULE and QT_IM_MODULE to be assigned "fcitx" also when using fcitx 5, do you see any adverse side effects?

[Launchpad Janitor (janitor)]

This bug was fixed in the package im-config - 0.47-1

---------------
im-config (0.47-1) experimental; urgency=medium

  * Team upload
  * Bug script tweaks

  [ Shengjing Zhu ]
  * Change IM_MODULE env for fcitx5 to "fcitx" (closes: #977203,
    LP: #1928360)

 -- Gunnar Hjalmarsson <email address hidden> Mon, 07 Jun 2021 09:38:38 +0200

[Gunnar Hjalmarsson (gunnarhj)]

So it was changed in im-config. In Debian the change was uploaded to experimental instead of unstable/bullseye for now, since the principal fcitx package maintainer Shengjing Zhu senses some degree of uncertainty. He wants to give the users some time to provide feedback before changing it in bullseye (Debian 11).

@handsome_feng: Since Kylin will ship both fcitx 4 and fcitx 5 in 21.10, I'm thinking that Kylin would be a perfect platform for testing this input configuration. So it would be good if you could include some testing also of fcitx 5 and be attentive to possible issues in light of the fact that it loads the fcitx 4 im modules instead of the fcitx 5 ones.

@James: Leaving the just mentioned uncertainty aside, we can conclude that there seems to be no need to make any changes to the core18 based platform snaps. An upstream change was made in fcitx5 5.0.5 with the explicit purpose to deal with e.g. snaps where only fcitx 4 im modules are present; see <https://bugs.debian.org/977203#34>.

But it would be good if we somehow could test to confirm that fcitx 5 works with a core20 based platform snap without modifying the apparmor abstractions for fcitx.

[James Henstridge (jamesh)]

> Is there such a snap you could point me to so I can test?

The candidate channel of the gedit snap is built with core20 and the gnome-3-38-2004 platform snap. But as I said, I don't think that platform snap currently includes the fcitx5 input module.

[Gunnar Hjalmarsson (gunnarhj)]

On 2021-06-08 08:54, James Henstridge wrote:
>> Is there such a snap you could point me to so I can test?
>
> The candidate channel of the gedit snap is built with core20 and the
> gnome-3-38-2004 platform snap. But as I said, I don't think that
> platform snap currently includes the fcitx5 input module.

Right.

$ ls /snap/gnome-3-38-2004/current/usr/lib/x86_64-linux-gnu/gtk-3.0/3.0.0/immodules | grep fcitx
im-fcitx.so

So then let's consider that a separate matter to be addressed later.

[handsome_feng (feng-kylin)]

> @handsome_feng: Since Kylin will ship both fcitx 4 and fcitx 5 in 21.10, I'm thinking that Kylin
> would be a perfect platform for testing this input configuration. So it would be good if you
> could include some testing also of fcitx 5 and be attentive to possible issues in light of the
> fact that it loads the fcitx 4 im modules instead of the fcitx 5 ones.

Okay, we will test it in the near future, and if there are any problems, we will feedback here.