Selection buffer tracking causes high CPU usage

[David Schneider]

The selection buffer tracking feature (ctrl-T) can cause high CPU usage and non-responsiveness in both Kupfer and the application owning the selection buffer if the buffer is large and/or repeatedly updated.

Easiest way to reproduce is to open up a blank spreadsheet in OpenOffice/LibreOffice and start highlighting whole columns. You'll find that the spreadsheet application will be unresponsive and use lots of CPU as it serializes the data, and that Kupfer will be unresponsive and use lots of CPU retrieving it.

Disconnecting the callback (line 41, gobject_connect_weakly) in kupfer/plugin/core/selection.py gets rid of the issue (by breaking the feature).

Ideally, since the source is only accessible by hotkey, the selection should only be queried when requested.

[Sergei Ivanov (svivanov)]

Created attachment 45239
example spreadsheet

Selecting a large number of columns in Calc consumes enormous amount of memory.

Attached file 'test.ods' has 40 rows and about 240 columns, all cells are just numbers (it is artificial but similar to a file that I am working on). All columns except the first one and the last one are hidden, so only two are visible.

When this file is loaded into Calc, it uses about 100 Mb of memory, mainly from shared libraries. Selecting the two visible columns (by dragging with a mouse, so that the hidden ones get selected too) has a weird effect: the program stops responding for a while and consumes more than 800 Mb of memory before it can work again.

I understand that inserting 10,000 cells to the clipboard requires some space, but 70 kilobytes per cell(!) is a little bit too high.

Un-selecting the columns brings the program back into its normal 100 Mb size.

Selecting the columns when they are un-hidden has the same effect (I just made made them hidden for the test case, because it is easier to select them).

I first observed this bug on a 32-bit Linux system with openoffice.org-3.2. Then reproduced it on x86-64 with libreoffice-3.3.2 installed from Ubuntu PPA packages. The program version screen says:

LibreOffice 3.3.2
OOO330m19 (Build:202)
tag libreoffice-3.3.2.2, Ubuntu package 1:3.3.2-1ubuntu2~maverick1

The rest of the system is Kubuntu 10.10 with KDE 4.6.1. Installing or un-installing the 'libreoffice-kde' add-on package does not make any difference.

[Kohei-yoshida-w (kohei-yoshida-w)]

I can't replicate that on my end. Selecting columns A:HZ has no effect on the memory usage of the process. Not even a bit.

I'm testing this on the openSUSE distro version of 3.3.2. And I'm in Gnome.

LibreOffice 3.3.2
OOO330m19 (Build:202)
tag libreoffice-3.3.2.2

[Sergei Ivanov (svivanov)]

(In reply to comment #1)
> I can't replicate that on my end. Selecting columns A:HZ has no effect on the
> memory usage of the process. Not even a bit.
>
> I'm testing this on the openSUSE distro version of 3.3.2. And I'm in Gnome.

I found that it is a KDE-specific issue.

KDE has a tool called klipper. It is a "clipboard manager" that keeps clipboard history and can launch a web browser for you if you select something resembling a web address. It also managed to add that 700 Mb to libreoffice (and 500 Mb more to itself, but I did not notice). Killing this tool fixed the issue for me.

I don't know how these things are supposed to work. Should I file a bug against klipper or is it libreoffice that does not interoperate properly?

[Kohei-yoshida-w (kohei-yoshida-w)]

Honestly I don't know whose fault it is.

I'll give this to Lubos. He is our KDE expert here.

[L-lunak (l-lunak)]

I cannot reproduce either, with or without Klipper (the same effect should be presumably achieved by running e.g. GEdit and Ctrl+V there). It indeed looks like LO bug if it consumes that much memory for the selection, but for that a way to reproduce would be needed first.

[Sergei Ivanov (svivanov)]

(In reply to comment #4)
> I cannot reproduce either, with or without Klipper (the same effect should be
> presumably achieved by running e.g. GEdit and Ctrl+V there). It indeed looks
> like LO bug if it consumes that much memory for the selection, but for that a
> way to reproduce would be needed first.

I reproduced it with editors: kwrite (KDE) and mousepad (non-KDE). To reproduce, I launch the editor with no file, run 'libreoffice test.ods', select the columns with the mouse, then middle-click in the editor window.

The effect is different in two editors.

kwrite: soffice.bin memory grows up to 800 Mb, nothing happens in the editor.

mousepad: soffice.bin memory grows up to 600 Mb, a huge text is pasted into the editor.

I saved the resulting text file from mousepad. Its size is 240 Mb (245376144 bytes). It consists of 1048576 lines. The first 40 lines are what they should be (digits and tabs), the rest of lines consists of tabs only.

[X (u78qir8a9-deactivatedaccount)]

This is a way to demonstrate that a very large text object is made available to the selection clipboard. Using pygtk, I just typed this into the python console, I selected one column in libreoffice calc (in an empty spreadsheet)

$ python
import gtk
sel = gtk.clipboard_get(gtk.gdk.SELECTION_PRIMARY)
text = sel.wait_for_text()

# now examine the returned text
>>> len(text)
1048576
>>> text.count("\n")
1048576
>>>

What the clipboard managers do is that they run the above -- "wait_for_text" or equivalent -- every time the selection changes.

[X (u78qir8a9-deactivatedaccount)]

Created attachment 45714
A minimal program for fetching the selection clipboard as soon as it changes

This is my try at making a program that can reproduce it. This is a "selectionbully.py" program -- as soon as the selection clipboard has possibly changed, it fetches the text. Start this script in a terminal, and then go selecting columns in libreoffice calc.. the script will then force calc to realize the data for those selections. It should illustrate the memory-using effect. Both LO and the script should both start using a lot of memory.

[X (u78qir8a9-deactivatedaccount)]

only thing solved in master so far is saner handling of the big text object -- an empty column is still on my clipboard as 1 million newlines in a text string. Kupfer will now be able to compute a description text of that object without killing itself. :-)

[X (u78qir8a9-deactivatedaccount)]

Libreoffice bug: https://bugs.freedesktop.org/show_bug.cgi?id=35964

[Tobias Wolf (towolf)]

I was bitten by this. Using the Matlab editor any selection of text with the keyboard becomes excruciatingly slow. Sometime it would stall for 30 seconds and more after it had extended the selection by a few lines. I even pestered Mathworks with a bug report.

Killing kupfer makes this go away.

Why is Kupfer monitoring the clipboard at all times? I never use this feature and the clipboard plugin is disabled.

[Bfo-bugmail (bfo-bugmail)]

Is this still an issue on KDE Linux with the latest stable (4.1.x) version? I couldn't reproduce it on Windows.

[Spam35 (spam35)]

Since this is a very old bug (from 2011) setting to NEEDINFO until we have it re-confirmed on the latest LO release 4.1.x. In that case please set the bug back to NEW.

[Z-jbf-faure (z-jbf-faure)]

Not reproducible for me with current stable or active branches (4.0.6, 4.1.5.0.0+ and 4.2.0.0.beta1) under Ubuntu 13.10 x86-64 (Ok, not KDE).
No answer since 3 months after 2 years without confirmation that the bug is still there. So closing this bug report as WorksForMe.

Please, feel free to reopen if you can reproduce it and are reasonably sure that the problem is on the LibreOffice side.

Best regards. JBF