Memory leak in ThumbnailGenerator

[Florian Boucault]

A possible memory leak is exhibited by the attached QML program 'leako.qml'. Run it as is on arale with at least 100 photos in /home/phablet/Pictures/com.ubuntu.camera

$ phablet-shell
$ qmlscene --desktop_file_hint=/usr/share/applications/address-book-app.desktop leako.qml

Running top will show that RES memory usage is rising indefinitely.

[Florian Boucault (fboucault)]

When removing the use of the thumbnailer and only loading the images no memory increase is observed.
I attach the Valgrind massif's log as proof.

To run valgrind with massif on the device:
$ valgrind --tool=massif --time-unit=ms --detailed-freq=1 /usr/lib/arm-linux-gnueabihf/qt5/bin/qmlscene --desktop_file_hint=/usr/share/applications/address-book-app.desktop leako.qml

[Florian Boucault (fboucault)]

Here is a screenshot of the visulisation of the log mentioned just before:
http://people.canonical.com/~kaleo/thumbnailer_leak/no_thumbnailer.png

[Florian Boucault (fboucault)]

When using the thumbnailer as is done in the sample program, the memory usage increases forever.
I attach the Valgrind massif's log as proof.

Screenshot of a visualisation of that log:
http://people.canonical.com/~kaleo/thumbnailer_leak/thumbnailer.png

From that log it appears that the culprit memory allocation is QImageData::create(...)

Using Valgrind's callgrind we can observe that these are all directly caused by the instantiation of QImage caused in turn by ThumbnailerImageResponse::dbusCallFinished()

[Florian Boucault (fboucault)]

I attach the log of Valgrind callgrind that can be run like so on the device:

$ valgrind --tool=callgrind /usr/lib/arm-linux-gnueabihf/qt5/bin/qmlscene --desktop_file_hint=/usr/share/applications/address-book-app.desktop leako.qml

Here is a screenshot of its visualisation with kcachegrind:
http://people.canonical.com/~kaleo/thumbnailer_leak/callgraph.png

[Michi Henning (michihenning)]

This doesn't look like a bug in the thumbnailer per se to me. Is it possible that Qt is doing something wrong?

Certainly, we have tested extensively under valgrind, and all our tests come up clean.

[Albert Astals Cid (aacid)]

THere's leaks everywhere.

You're doing one thing wrong that is create the texturefactory even if Qt doesn't ask for it, which means it'll never free it.

I am working in patches to fix it.

[Albert Astals Cid (aacid)]

There's three different patches needed, the branch linked for the thumbnailer and also
https://codereview.qt-project.org/#/c/123420/ and https://codereview.qt-project.org/#/c/123432/ (both still on review upstream)

That reduces a lot the memory usage here and can't find any leak more using valgrind

[Michi Henning (michihenning)]

Thanks for picking us up on that one and stepping in to help!

[PS Jenkins bot (ps-jenkins)]

Fix committed into lp:thumbnailer/devel at revision 254, scheduled for release in thumbnailer, milestone Unknown

[Albert Astals Cid (aacid)]

Timo: The upstream Qt patches have been approved and merged, can we get them distro patched?

[Timo Jyrinki (timo-jyrinki)]

Yes. One qtdeclarative update is currently in the pipeline, this'd be for the next one.

[Timo Jyrinki (timo-jyrinki)]

https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/landing-026 now has the 5.4.1-1ubuntu10 - please test regarding this bugfix and eg Unity 8 in general.

[Albert Astals Cid (aacid)]

Tried the silo, can't find any problem with it and the memory leaks seem to be gone as far as i can see.

[Timo Jyrinki (timo-jyrinki)]

Ignore this comment.

Sorry for the noise, testing new Launchpad filter additions to bug e-mails.

[Launchpad Janitor (janitor)]

This bug was fixed in the package qtdeclarative-opensource-src - 5.4.2-1ubuntu6

---------------
qtdeclarative-opensource-src (5.4.2-1ubuntu6) wily; urgency=medium

  * debian/patches/Fix-memory-leak-when-QQuickPixmapReply-Event-is-dele.patch
    debian/patches/Fix-memory-leak-when-using-async-image-providers.patch:
    - Fix memory leaks (LP: #1484914)

 -- Timo Jyrinki <email address hidden> Tue, 01 Sep 2015 12:02:43 +0000

[Timo Jyrinki (timo-jyrinki)]

Also in vivid overlay since yesterday:

qtdeclarative-opensource-src (5.4.1-1ubuntu10) vivid; urgency=medium

  * debian/patches/Fix-memory-leak-when-QQuickPixmapReply-Event-is-dele.patch
    debian/patches/Fix-memory-leak-when-using-async-image-providers.patch:
    - Fix memory leaks (LP: #1484914)

 -- Timo Jyrinki <email address hidden> Fri, 28 Aug 2015 10:04:35 +0000

[Pat McGowan (pat-mcgowan)]

still waiting for thumbnailer fix

[Pat McGowan (pat-mcgowan)]

there is a chance this fix improved some overall performance so should consider for early release

[Michi Henning (michihenning)]

We have two more branches to go into the next release. We'll putting things into a silo on Tuesday.

[Michi Henning (michihenning)]

Sitting in silo 15 now. We haven't run the test plan yet, but I don't anticipate any issues.