Japanese hiragana input fails after first keypress in anki

[fuzzyBSc]

This was working fine before I upgraded to 21.04. Since then I am seeing what appears to be a bad interaction between the Japanese input method and anki or anki's underlying widget set. Hiragana input works by typing the roman characters that make up a word. After typing a few characters you'll get suggestions for the correct Japanese text.

For example, I type aimasu, which will appear on screen as あいます, with the suggestion that I probably meant 会います which I can select from a drop-down.

Since upgrading to 21.04 when I type the first character into the anki "Add card" dialog I see the first hiragana あ appear correctly, the "i" character appears to be ignored and maybe is the place things fail, followed by the roman characters for "masu". In other words, aimasu appears on the screen as あmasu as opposed to the correct あいます. The drop-down I would normally expect to see appears only for the あ character and then gets stuck on the screen until I switch back to an English input method and back again to Japanese. I can't enter the full Japanese text, although my web browser and terminal window are able to accept Japanese text without any issues.

My input method shows in the UI as "Japanese (Mozc)" with input mode "hiragana".

ProblemType: Bug
DistroRelease: Ubuntu 21.04
Package: anki 2.1.15+dfsg-3
ProcVersionSignature: Ubuntu 5.11.0-18.19-generic 5.11.17
Uname: Linux 5.11.0-18-generic x86_64
NonfreeKernelModules: nvidia_modeset nvidia
ApportVersion: 2.20.11-0ubuntu65.1
Architecture: amd64
CasperMD5CheckResult: unknown
CurrentDesktop: ubuntu:GNOME
Date: Thu Jun 10 18:59:19 2021
InstallationDate: Installed on 2016-08-13 (1761 days ago)
InstallationMedia: Ubuntu 16.04.1 LTS "Xenial Xerus" - Release amd64 (20160719)
PackageArchitecture: all
SourcePackage: anki
UpgradeStatus: Upgraded to hirsute on 2021-06-06 (4 days ago)

[fuzzyBSc (benjamincarlyle-soundadvice)]

c.f. https://github.com/ankitects/anki/issues/1239

[fuzzyBSc (benjamincarlyle-soundadvice)]

I think I'm narrowing down on the source of the issue. I'm seeing crashes in ibus-engine-mozc as part of package ibus-mozc
After building from source and installing debug symbols the key parts of the stack trace appears to be as follows

#10 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::assign<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, void>(__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, __gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >)Python Exception <class 'gdb.error'> value has been optimised out:
Python Exception <class 'gdb.error'> value has been optimised out:
 (__last=, __first=, this=0x7ffc45391578)
    at /usr/include/c++/10/bits/basic_string.h:1471
#11 mozc::ibus::(anonymous namespace)::GetSurroundingText(_IBusEngine*, mozc::ibus::SelectionMonitorInterface*, mozc::ibus::(anonymous namespace)::SurroundingTextInfo*) [clone .part.0] [clone .constprop.0]
    (engine=<optimised out>, selection_monitor=<optimised out>, info=0x7ffc45391530) at ../../unix/ibus/mozc_engine.cc:207
#12 0x000056510ef5e7c8 in mozc::ibus::(anonymous namespace)::GetSurroundingText
    (info=0x7ffc45391530, selection_monitor=<optimised out>, engine=0x56510fc2c410) at ../../unix/ibus/mozc_engine.cc:170
#13 mozc::ibus::MozcEngine::ProcessKeyEvent(_IBusEngine*, unsigned int, unsigned int, unsigned int)
    (this=<optimised out>, engine=<optimised out>, keyval=<optimised out>, keycode=<optimised out>, modifiers=<optimised out>)
    at ../../unix/ibus/mozc_engine.cc:377

[fuzzyBSc (benjamincarlyle-soundadvice)]

mozc_engine.cc:207 appears to be source of an exception in a C++ string, so surrounding_text.end - selection_end_it < 0...
(gdb) list
202 const size_t selection_length = abs(info->relative_selected_length);
203 const auto &selection_start_it = surrounding_text.begin() + selection_start;
204 const auto &selection_end_it = selection_start_it + selection_length;
205 info->preceding_text.assign(surrounding_text.begin(), selection_start_it);
206 info->selection_text.assign(selection_start_it, selection_end_it);
207 info->following_text.assign(selection_end_it, surrounding_text.end());
208 return true;
209 }
210
211 std::unique_ptr<client::ClientInterface> CreateAndConfigureClient() {

[fuzzyBSc (benjamincarlyle-soundadvice)]

As far as I can tell line 207 above fails due to the text being the empty string "", but the cursor_pos and anchor_pos are both 1, so line 207 is trying to assign the string from position 1 to position 0 within the empty string. That makes some sense given how the input method works. At the time of the crash, a single character has been entered, but it hasn't been committed to the dialog as yet. That said I still don't understand whether it is Anki, QT, or mozc that is fundamentally at fault. My guess is mozc despite this working in other contexts I have tried.

[fuzzyBSc (benjamincarlyle-soundadvice)]

c.f. https://github.com/google/mozc/issues/510

[fuzzyBSc (benjamincarlyle-soundadvice)]

This issue has now been fixed in mozc under issue 510 above.

[Gunnar Hjalmarsson (gunnarhj)]

Thanks for your report and your research!

However, the upstream commit which is said to fix it is huge, and does not apply to the version of the source in Debian/Ubuntu.

mozc is packaged at Debian. It would be great if you could file a bug to Debian too, so the maintainer can consider the best way to deal with it.

[fuzzyBSc (benjamincarlyle-soundadvice)]

Thanks Gunnar,

I have filed https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=991228

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Christian Pernegger (fallenguru)]

FWIW, building new debs for jammy from lunar's source package is straight-forward, and I haven't had any issues so far. I'd still much prefer an official version, obviously.

Would it be possible to get a newer version of the mozc packages built for jammy? Throw it into backports, or an official PPA, if you must, but Japanese input not working in Qt applications (Google says it isn't just Anki) isn't fun.

[Gunnar Hjalmarsson (gunnarhj)]

@Christian: Thanks for the tip. I agree that a backport is a good idea, so I have proposed it in bug #2030259.

[Gunnar Hjalmarsson (gunnarhj)]

This was fixed in kinetic via a new upstream release.