onboard doesn't learn new words as quickly as in the previous versions

[kenn]

I installed Ubuntu 16.04 32 bit recently. I have been using Ubuntu for 5 years. I use my comp by means of onboard. I noticed that onboard in 16.04 does not learn as quick as in that Ubuntu 14.04. I just tested it by typing a command in terminal which it doesn't suggest because it's not in dictionary, I typed it five times but it doesn't remember the command I typed. In Ubuntu 14.04 it used to learn new words pretty quickly. Is there any setting that I can set to make it learn fast?

[Francesco Fumanti (frafu)]

There are settings in the Onboard Preferences to not have it learn new words. You can find it in the Advanced tab of the Typing Assistance section of the Preferences dialog.

There is no setting to make Onboard learn faster. If your Onboard version does learn words, but only after some delay, there might be a problem somewhere and I would suggest to try to see if the problem persists with the new version in our Stable PPA:
https://launchpad.net/~onboard/+archive/ubuntu/stable

[kenn (whatnext)]

I forgot to mention that I already checked that setting in preferences menu. Can it be related to accessibility or dbus daemon?

[marmuta (marmuta)]

If you see any suggestions at all, then AT-SPI is available/accessibility and dbus daemon are working.

I would look at the debug output first:
$ onboard -dinfo

Type something into another terminal, then press enter or make it lose focus. You should see something like:
Onboard.WPEngine learn_text: tokens=[['<bot:txt>', 'hello']]

Note that gnome-terminal has the most reliable AT-SPI support. Others may fail in various ways, learning could be prevented too.

Also, perhaps check your language models. Onboard ought to complain if there are any errors, but just in case run:
$ /usr/share/onboard/tools/checkmodels
They all should print "No errors"

[kenn (whatnext)]

@marmuta I just tried your instruction, in debug terminal no `Onboard.WPEngine learn_text: tokens=[['<bot:txt>', 'hello']]` prints, it doesn't print anything with `learn_text: tokens` when I enter a word in another terminal, gedit, pluma etc. or any other accessibility compliant text editors.

[kenn (whatnext)]

I purged `onboard` and `onboard-data` then added `sudo add-apt-repository ppa:onboard/stable` and did `sudo apt-get update && sudo apt-get update` but it didn`t work. I checked out language model path at `/home/kenn/.local/share/onboard/models/en_US.lm` then I removed `/home/kenn/.local/share/onboard` thinking that it might be a permission issue but after removing that folder it doesn`t recreate it in spite of reinstalling `onboard` and `onboard-data` . It might be a python path issue? Becaused I added `export PYTHONPATH=$PYTHONPATH:/usr/lib/python2.7/site-packages/` to .bashrc last week.

[marmuta (marmuta)]

PYTHONPATH changes would more likely result in tracebacks, rather than subtle changes to word suggestions, in particular since this mixes Python 2 with Python 3 packages. Not saying it's impossible, but I doubt that's the reason.

We'll have to check everything systematically. Please run this
$ gsettings list-recursively org.onboard | grep -v snippets >onboard-settings.txt
and attach onboard-settings.txt.

Then run onboard again with more debug output,
$ onboard -d atspi 2>&1 | tee onboard-debug.txt
focus a different terminal, type a word there, focus gedit, type a word there, then exit onboard (Ctrl+C is fine) and attach onboard-debug.txt in another post.

[kenn (whatnext)]

I added the debug info. I noticed that it completely stopped learning for a few weeks, it only suggests learned words from a few weeks ago.

[marmuta (marmuta)]

Hmm, everything appears to work as expected but when it's time to learn, there is no attempt being made. It's as if learning was turned off or paused, but looking at your settings, that clearly isn't the case.

I can't reproduce this on Zesty, will do some tests on Ubuntu 16.04.

It has to happen somewhere on the path from WordSuggestions.commit_changes() to WPEngine.learn_text(), but I can't tell where or why at the moment.

If you feel adventurous, add some print("hello 1") statements and find out where commit_changes() and below exits without learning.
$ sudo gedit /usr/lib/python3/dist-packages/Onboard/WordSuggestions.py /usr/lib/python3/dist-packages/Onboard/WPEngine.py

I'll add some more debug output anyway.

[marmuta (marmuta)]

Done testing trunk on 16.04 and it did learn as it should. No obvious regression there with Onboard 1.4. All unit tests pass too. It was the 64 bit version, though. If we get stuck I'll try the 32 bit version too.

Meanwhile, we have new debug output in trunk, for the relevant stuff between WordSuggestions.commit_changes() and WPEngine.learn_text().
http://bazaar.launchpad.net/~onboard/onboard/trunk/revision/2260

Francesco, would you please create a snapshot? Thanks in advance.

kenn, when revision 2260 is ready in
https://launchpad.net/%7Eonboard/+archive/snapshots
please install it and run onboard again with debug output:

$ onboard -d atspi 2>&1 | tee onboard-debug2.txt

Same procedure, focus a different terminal, type a word there, focus gedit, type a word there, then exit onboard (Ctrl+C is fine) and attach onboard-debug2.txt.

[Francesco Fumanti (frafu)]

Revision 2261 containing the additional debug output is now available in our Snapshots PPA for xenial, yakkety and zesty:
https://launchpad.net/~onboard/+archive/ubuntu/snapshots

[kenn (whatnext)]

Thank you guys, I am grateful to you. I added the ppa and did an upgrade. I used my own locale, some info printed in Turkish in the debug output, If you have a difficulty at understanding it, I can switch my locale to US.

[kenn (whatnext)]

Here it is ~/.local/share/onboard/models/tr_TR.lm

[marmuta (marmuta)]

> 13:33:13.881 DEBUG Onboard.WordSuggestions commit_changes(): has_changes=False
and
> 13:33:57.891 DEBUG Onboard.WordSuggestions commit_changes(): has_changes=True
> 13:33:57.891 DEBUG Onboard.WordSuggestions LearnStrategyLRU.commit_changes():
> changes=TextChanges(["TextSpan(14, 0, '', 0, None)"])

It's not recording text changes, apparently. That's a different piece of code in TextContext.py and that part only prints error messages to the log. That part is quite old and not known to be unreliable, I wonder what's going on. I'll have to add yet more debug output, I'm afraid.

Your tr_TR.lm is fine and even if it wasn't, recording text changes happens independently of the language models.

[kenn (whatnext)]

just an intuitive suggestion; may it be related to `system settings > language support > language tab > keyboard input method` ? In the dropdown list I have only XIM and None options. IIRC in 14.04 I had ibus and xinput?

[marmuta (marmuta)]

I won't exclude anything at this point, but this seems unlikely, as Onboard doesn't depend on any particular input method.

There is not that much left that can fail. Recording text changes is basically incoming AT-SPI events, AT-SPI method calls and a bit of glue code in Onboard. It's mostly one function, TextContext._record_text_change(). The AT-SPI text-change events may be OK, you can see incoming text in these lines
Onboard.WPEngine context=['<bot:txt>', 'hello', 'hel']

but perhaps one of the AT-SPI calls fails for some reason. I'll add debug output later today.

[marmuta (marmuta)]

OK, one more snapshot please, Francesco, thanks in advance.

kenn, when revision 2268 is ready please go through the same procedure again:

$ onboard -d atspi 2>&1 | tee onboard-debug3.txt

Focus a different terminal, type a word there, focus gedit, type a word there, switch away from gedit, exit onboard and attach onboard-debuge.txt.

One more thing, did you customize your terminal prompt? At some point I thought this could have been a problem, but from the log it probably isn't. If you changed it, it'd be good to test it here anyway.
$ echo "|$PS1|"

[Francesco Fumanti (frafu)]

Trunk revision 2270 is now available in our Snapshots PPA.

[kenn (whatnext)]

No, I haven't customized the prompt.

     echo "|$PS1|"
     |${debian_chroot:+($debian_chroot)}\u@\h:\w\$ |

[kenn (whatnext)]

As you said "I won't exclude anything at this point", I am also getting curious, I don't think it has not something to do with it but I should mention: I am using a chroot environment, so to speak I boot into a chroot environment in my external disk by means of this script https://github.com/lemonsqueeze/boot_chroot I have multiple distros that I created by means of debootstrap http://www.thegeekstuff.com/2010/01/debootstrap-minimal-debian-ubuntu-installation/

More info :
http://askubuntu.com/questions/405735/installing-multiple-linux-distros-in-a-single-partition/409229#409229

[marmuta (marmuta)]

Your special setup is probably part of the puzzle. It might just be that you get newer versions of packages with problems that haven't made it to other distributions yet. What's your version of libatspi2?

To the log: so there is an AT-SPI event missing after all: "object:text-changed:insert". You only receive "object:text-changed:delete". That is, text deletions, e.g. when pressing backspace, are recorded, but text insertions are not. Interesting, that's new.

I found
> AtkText::text-changed has been deprecated since version 2.9.4 and
> should not be used in newly-written code.
> Use “text-insert” or “text-remove” instead.

but the documentation and source of libatspi2 don't mention these and still list object:text-changed as valid event.

Listening to object:text-insert works, but apparently then object:text-remove does nothing. Weirder yet, object:text-insert receives events for both inserting and removing text and internally maps to text-changed events. This seems like a dead end.

[marmuta (marmuta)]

Let's see what happens if we only listen to object:text-changed events, rather than the insert and delete variants. I remember AtspiEventListener being picky when registering multiple events, seemingly ignoring some for unclear reasons. Maybe your version of AT-SPI has become even more picky.

Francesco, sorry I have to bother you again. One more snapshot, please.

kenn, same procedure please, when revision 2271 is ready:

$ onboard -d atspi 2>&1 | tee onboard-debug4.txt

Focus a different terminal, type a word there, focus gedit, type a word there, switch away from gedit, exit onboard and attach onboard-debuge.txt.

[kenn (whatnext)]

I am sorry for inconvenience I give rise to.
I use standard Ubuntu 16.04 repositories.
cat /etc/apt/sources.list
deb http://security.ubuntu.com/ubuntu/ xenial-security main universe restricted multiverse
deb http://archive.ubuntu.com/ubuntu xenial-updates main universe restricted multiverse
deb http://archive.ubuntu.com/ubuntu xenial main universe restricted multiverse
deb http://archive.ubuntu.com/ubuntu xenial-backports universe restricted main multiverse

dpkg -s libatspi2.0-0
Package: libatspi2.0-0
Status: install ok installed
Priority: optional
Section: libs
Installed-Size: 212
Maintainer: Ubuntu Developers <email address hidden>
Architecture: i386
Multi-Arch: same
Source: at-spi2-core
Version: 2.18.3-4ubuntu1
Depends: libc6 (>= 2.7), libdbus-1-3 (>= 1.9.14), libglib2.0-0 (>= 2.37.3), libx11-6
Recommends: at-spi2-core (= 2.18.3-4ubuntu1)
Description: Assistive Technology Service Provider Interface - shared library
 This package contains the shared library for applications that wish to use
 the at-spi interface.
Homepage: https://wiki.gnome.org/Accessibility
Original-Maintainer: Debian Accessibility Team <email address hidden>

And I found below processes in running applications:

dbus-daemon --config-file=/etc/at-spi2/accessibility.conf --nofork --print-address 3
at-spi-bus-launcher --launch-immediately
at-spi2-registryd --use-gnome-session

I found `Accessibility Profile Manager Indicator` in startup applications and I tried with both disabled and enabled but nothing changed.

[kenn (whatnext)]

I just noticed that a few more words added to `tr_TR.lm` but it's intermittent and stopped again.

[kenn (whatnext)]

I checked out the modification time of "tr_TR.lm" , it shows about 20 minutes ago.

[Francesco Fumanti (frafu)]

I just uploaded revision 2271 of trunk to the Snapshots PPA; it should be available in several minutes.

[kenn (whatnext)]

For the first time it suggested the word "hello" , I upload the debug log with "hello". I just tried another word "thanks" which it didn't recognize previously but it has just registered it.

[kenn (whatnext)]

Very strange, it seems to be running very well now. I upload another debug output which I typed "greetings", it remembered it after first typing. Have we come across a version of Heisenbug https://en.wikipedia.org/wiki/Heisenbug ?

[marmuta (marmuta)]

> Onboard.WPEngine learn_text: tokens=[['<bot:term>', 'hello', 'bello', 'hello', 'hello']]
> Onboard.WPEngine learn_text: tokens=[['<bot:term>', 'greetings', 'greetings', 'greetings', 'greetings']]
Yep, seems to work. It just means we found a fix, eventually:
http://bazaar.launchpad.net/~onboard/onboard/trunk/revision/2271

It's that picky AtspiEventListener. I can't tell why registering the insert event did nothing on your system, but as I mentioned, I've seen similar behavior with other events before, during development.

> 2.18.3-4ubuntu1
My libatspi2.0-0 is newer, 2.22.0-5ubuntu3. So the new, untested packages theory is out, too.

Anyway, keep an eye on it. If it misbehaves again let us know.

Thanks a lot for your patience.

[kenn (whatnext)]

Thank you marmuta , frafu and all those who contributed to this project. Thanks to you I can use a computer easily.