thumbnailer crash when reading metadata

[Timo Jyrinki]

Test Case:
0. Make sure you have a clean install and you never started gallery app
1. Take a few photos with the camera and close it
2. open gallery

Actual Result:
It crashes.

Expected Result:
It doesn't crash.

Downgrading to the following packages fixes the problem:
qtubuntu-camera 0.3.3+14.10.20141001-0ubuntu1
thumbnailer 1.3+14.10.20141020-0ubuntu1

PS: you don't really need a clean install just delete gallery' QML cache in ~/.cache/QML/Apps

Analysis of what landed:
< Mirv> jibel: ok. the diff was this interesting http://launchpadlibrarian.net/191646180/thumbnailer_1.3%2B14.10.20141020-0ubuntu1_1.3%2B15.04.20141106~rtm-0ubuntu1.diff.gz together with qtubuntu-camera from rsalveti / jhodapp / satoris
< Mirv> the qtubuntu-camera was huge: http://launchpadlibrarian.net/192809040/qtubuntu-camera_0.3.3%2B14.10.20141001-0ubuntu1_0.3.3%2B15.04.20141208.is.0.3.3%2B15.04.20141204~rtm-0ubuntu1.diff.gz (publishing history seems to confirm it's correct: https://launchpad.net/ubuntu-rtm/+source/qtubuntu-camera/+publishinghistory )
< Mirv> I've updated the bug #1376500 which the landing fixed, which was also not updated to indicate it would have been fixed in the first place
< Mirv> the trello comment on that landing does not seem to indicate the "silo-diff" (from brendand) was used to actually check the rtm diff, as it only mentions the silo fixing two bugs
< Mirv> jibel: crash was also mentioned in the trello: https://trello.com/c/zpBMvnsD/443-ubuntu-rtm-landing-012-qtubuntu-camera-thumbnailer-jhodapp-satoris

Reproduced on :
current build number: 183
device name: krillin
channel: ubuntu-touch/ubuntu-rtm/14.09-proposed
last update: 2014-12-18 13:29:15
version version: 183
version ubuntu: 20141217.2
version device: 20141209-cae2b5f
version custom: 20141216-484-22-169

com.ubuntu.gallery 2.9.1.1113
thumbnailer 1.3+15.04.20141106~rtm-0ubuntu1
qtubuntu-camera 0.3.3+15.04.20141208.is.0.3.3+15.04.20141204~rtm-0ubuntu1

[Jussi Pakkanen (jpakkane)]

This is the same bug with more information: https://bugs.launchpad.net/gallery-app/+bug/1403643

One of these should be marked a dupe of the other but it's not clear to me which one.

[Jean-Baptiste Lallement (jibel)]

From comment #2 of Jussi Pakanen in duplicate bug 1403643
"""
The reason this happens is that whoever is creating the images produces invalid image files. I have attached one to this bug and trying to open it with eog prints the following error (which is the same as what thumbnailer prints in the background):

Failed to load image '/home/phablet/Pictures/com.ubuntu.camera/image20141218_143650580.jpg': Error interpreting JPEG image file (Quantization table 0x00 was not defined)

When this happens, thumbnailer passes the original image to Qt. Presumably this causes it to then crash. If so, this would indicate a bug in Qt's image processing code.
"""

[Florian Boucault (fboucault)]

A few notes after a conjoint investigation between Bill and myself:

1. Downgrading only the thumbnailer to version 1.3+14.10.20141020-0ubuntu1 fixes the issue.
2. We reproduced the crash many times but never had a corrupt JPEG nor a corrupt embedded thumbnail (they were read perfectly fine by eog and the thumbnailer extracted without issue by exiv2 ex -e p)
3. If the crash does not happen during a given run of the gallery then it does not happen again after that until the following folder is deleted: ~/.cache/com.ubuntu.gallery/ Once deleted the crash happens during the subsequent reruns of the gallery.
4. The crash is happening in a call to libthumbnailer. I was able to retrace the .crash file generated by apport by first adding a line to it (Package: gallery-app) and then running apport-retrace --gdb PATH_TO_CRASH_FILE. I could extract the stack traces of all the threads: http://pastebin.ubuntu.com/9561659/ and here is the thread responsible for the crash:

#0 __libc_do_syscall () at ../ports/sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:44
#1 0xb58f8e5e in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#2 0xb58f9b4e in __GI_abort () at abort.c:89
#3 0xb58f40c4 in __assert_fail_base (fmt=0x1 <error: Cannot access memory at address 0x1>, assertion=0xad6c5178 "ifilename[0] == '/'",
    assertion@entry=0x0, file=0xad6c5130 "/build/buildd/thumbnailer-1.3+15.04.20141106~rtm/src/imagescaler.cpp", file@entry=0xac13f370 "\001",
    line=101, line@entry=3046891692,
    function=function@entry=0xad6c5088 "bool ImageScaler::scale(const string&, const string&, ThumbnailSize, const string&, const string&) const")
    at assert.c:92
#4 0xb58f415a in __GI___assert_fail (assertion=0x0, file=0xac13f370 "\001", line=3046891692,
    function=0xad6c5088 "bool ImageScaler::scale(const string&, const string&, ThumbnailSize, const string&, const string&) const") at assert.c:101
#5 0xad6be25e in ?? () from /usr/lib/arm-linux-gnueabihf/libthumbnailer.so.0
#6 0xad6c3c54 in ?? () from /usr/lib/arm-linux-gnueabihf/libthumbnailer.so.0
#7 0xad6c3e6a in Thumbnailer::get_thumbnail(std::string const&, ThumbnailSize, ThumbnailPolicy) ()
   from /usr/lib/arm-linux-gnueabihf/libthumbnailer.so.0
#8 0xad6c400e in Thumbnailer::get_thumbnail(std::string const&, ThumbnailSize) () from /usr/lib/arm-linux-gnueabihf/libthumbnailer.so.0
#9 0xad6ecd86 in ThumbnailGenerator::requestImage(QString const&, QSize*, QSize const&) ()
   from /usr/lib/arm-linux-gnueabihf/qt5/qml/Ubuntu/Thumbnailer.0.1/libthumbnailer-qml.so

[Florian Boucault (fboucault)]

Because of 1. from previous comment I mark the task on qtubuntu-camera invalid

[Bill Filler (bfiller)]

Another important note, thumbnails are still being generated on disk on .cache/com.ubuntu.gallery and .cache/thumbnailer. These files should not be getting created anymore with the thumbnailer in question as it's supposed to be using the embedded thumbnailer from the meta-data.

So this is another issue with the proposed fix.

[Jussi Pakkanen (jpakkane)]

I have linked a branch with some memory safety fixes. Since I can't reproduce this on non-broken images would it be possible for you to try it?

If the new version still crashes for you, could you do the following:

- check out the branch and build it in debug mode
- then in your build dir do "tools/cachetool /path/to/image"

If the latter crashes, please post the full backtrace.

> Another important note, thumbnails are still being generated on disk on .cache/com.ubuntu.gallery and .cache/thumbnailer.

This is by design and will remain so. Thumbnailer only extracts the thumbnail from the source image once and caches it in the file specified. This behaviour is identical across all thumbnail generator backends.

[Ricardo Salveti (rsalveti)]

Was able to confirm the crash when using vivid on krillin.

[Ricardo Salveti (rsalveti)]

Backtrace:
#0 __libc_do_syscall () at ../ports/sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:44
44 ../ports/sysdeps/unix/sysv/linux/arm/libc-do-syscall.S: No such file or directory.
(gdb) bt full
#0 __libc_do_syscall () at ../ports/sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:44
No locals.
#1 0xb58d1e5e in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
        _a1 = 0
        _a3tmp = 6
        _a1tmp = 0
        _a3 = 6
        _nametmp = 268
        _a2tmp = 4924
        _a2 = 4924
        _name = 268
        _sys_result = <optimized out>
        pd = 0xac085370
        pid = 0
        selftid = 4924
#2 0xb58d2b4e in __GI_abort () at abort.c:89
        save_stage = 2
        act = {__sigaction_handler = {sa_handler = 0x0, sa_sigaction = 0x0}, sa_mask = {__val = {3069791548, 3046740840, 1130327653, 4294967295, 0, 3046601780, 4294967295, 4294967295, 3045793976, 2, 4, 2886220744, 2886228016, 0, 0, 3046727680, 3046630632, 0, 3046727680,
              1, 2906575152, 101, 2886223720, 3046065667, 3046730064, 0, 3046601816, 2808782848, 4294967295, 3046730064, 2906575220, 3046601816}}, sa_flags = -1388392144, sa_restorer = 0x65}
        sigs = {__val = {32, 0 <repeats 31 times>}}
#3 0xb58cd0c4 in __assert_fail_base (fmt=0x1 <error: Cannot access memory at address 0x1>, assertion=0xad3ed174 "ifilename[0] == '/'", assertion@entry=0x0, file=0xad3ed130 "/build/buildd/thumbnailer-1.3+15.04.20141202/src/imagescaler.cpp", file@entry=0xac085370 "\001",
    line=101, line@entry=3046731948,
    function=function@entry=0xad3ed088 <ImageScaler::scale(std::string const&, std::string const&, ThumbnailSize, std::string const&, std::string const&) const::__PRETTY_FUNCTION__> "bool ImageScaler::scale(const string&, const string&, ThumbnailSize, const string&, const string&) const") at assert.c:92
        str = 0xcd20a0 "\001"
        total = 4096
#4 0xb58cd15a in __GI___assert_fail (assertion=0x0, file=0xac085370 "\001", line=3046731948,
    function=0xad3ed088 <ImageScaler::scale(std::string const&, std::string const&, ThumbnailSize, std::string const&, std::string const&) const::__PRETTY_FUNCTION__> "bool ImageScaler::scale(const string&, const string&, ThumbnailSize, const string&, const string&) const") at assert.c:101
No locals.
#5 0xad3e625e in ImageScaler::scale (this=this@entry=0xa0eb88, ifilename=<error reading variable: Cannot access memory at address 0x65>, ofilename="/home/phablet/.cache/com.ubuntu.gallery/thumbnails/large/f4beee1bd41fc89bcd9ebc83e6140e11.png",
    wanted=wanted@entry=TN_SIZE_LARGE, original_location="/home/phablet/Pictures/com.ubuntu.camera/image20150106_164049006.jpg", rotation_source_file="/home/phablet/Pictures/com.ubuntu.camera/image20150106_164049006.jpg")
    at /build/buildd/thumbnailer-1.3+15.04.20141202/src/imagescaler.cpp:101
        rnd = {{_M_file = 0xcd2920, _M_mt = {static word_size = <optimized out>, static state_size = 624, static shift_size = <optimized out>, static mask_bits = <optimized out>, static xor_mask = <optimized out>, static tempering_u = <optimized out>,
              static tempering_d = <optimized out>, static tempering_s = <optimized out>, static tempering_b = <op...

[Pat McGowan (pat-mcgowan)]

Dropping priority based on status of broken in vivid not landed in rtm, but we want the optimization.

[Jussi Pakkanen (jpakkane)]

I have been able to reproduce this finally with the following steps:

- flash krillin (--bootstrap) with vivid-proposed
- put the broken image in ~/Pictures
- start Gallery
- if not crash, close Gallery, rm -rf .cache/com.ubuntu.gallery/ and go back to previous step

After a few tries it crashes. If you install the new packages from silo 19 (as mentioned in the linked branch MR) I can't make the crash happen any more.

One thing that I did encounter was that when I click on the image in gallery, it comes up fully black. The thumbnail looks correct, though. However as far as I understand it Gallery should not use thumbnailer for displaying full sized images. If that is the case then this is an issue somewhere else.

Florian, Bill: since you two seem to be the hardest hit by this bug could you test silo 19? If it works for you then post your results here so we can get this one landed. Thanks.

[Florian Boucault (fboucault)]

I'm unable to reproduce the issue with packages from silo 19

[Jim Hodapp (jhodapp)]

Good news, let's land this then.

Jim

On Wed, Jan 7, 2015 at 11:14 AM, Florian Boucault <
<email address hidden>> wrote:

> I'm unable to reproduce the issue with packages from silo 19
>
> --
> You received this bug notification because you are subscribed to a
> duplicate bug report (1403643).
> https://bugs.launchpad.net/bugs/1403906
>
> Title:
> thumbnailer crash when reading metadata
>
> Status in the base for Ubuntu mobile products:
> Confirmed
> Status in thumbnailer package in Ubuntu:
> New
> Status in qtubuntu-camera package in Ubuntu RTM:
> Invalid
> Status in thumbnailer package in Ubuntu RTM:
> Confirmed
>
> Bug description:
> Test Case:
> 0. Make sure you have a clean install and you never started gallery app
> 1. Take a few photos with the camera and close it
> 2. open gallery
>
> Actual Result:
> It crashes.
>
> Expected Result:
> It doesn't crash.
>
> Downgrading to the following packages fixes the problem:
> qtubuntu-camera 0.3.3+14.10.20141001-0ubuntu1
> thumbnailer 1.3+14.10.20141020-0ubuntu1
>
> PS: you don't really need a clean install just delete gallery' QML
> cache in ~/.cache/QML/Apps
>
> Analysis of what landed:
> < Mirv> jibel: ok. the diff was this interesting
> http://launchpadlibrarian.net/191646180/thumbnailer_1.3%2B14.10.20141020-0ubuntu1_1.3%2B15.04.20141106~rtm-0ubuntu1.diff.gz
> together with qtubuntu-camera from rsalveti / jhodapp / satoris
> < Mirv> the qtubuntu-camera was huge:
> http://launchpadlibrarian.net/192809040/qtubuntu-camera_0.3.3%2B14.10.20141001-0ubuntu1_0.3.3%2B15.04.20141208.is.0.3.3%2B15.04.20141204~rtm-0ubuntu1.diff.gz
> (publishing history seems to confirm it's correct:
> https://launchpad.net/ubuntu-rtm/+source/qtubuntu-camera/+publishinghistory
> )
> < Mirv> I've updated the bug #1376500 which the landing fixed, which was
> also not updated to indicate it would have been fixed in the first place
> < Mirv> the trello comment on that landing does not seem to indicate the
> "silo-diff" (from brendand) was used to actually check the rtm diff, as it
> only mentions the silo fixing two bugs
> < Mirv> jibel: crash was also mentioned in the trello:
> https://trello.com/c/zpBMvnsD/443-ubuntu-rtm-landing-012-qtubuntu-camera-thumbnailer-jhodapp-satoris
>
> Reproduced on :
> current build number: 183
> device name: krillin
> channel: ubuntu-touch/ubuntu-rtm/14.09-proposed
> last update: 2014-12-18 13:29:15
> version version: 183
> version ubuntu: 20141217.2
> version device: 20141209-cae2b5f
> version custom: 20141216-484-22-169
>
> com.ubuntu.gallery 2.9.1.1113
> thumbnailer 1.3+15.04.20141106~rtm-0ubuntu1
> qtubuntu-camera 0.3.3+15.04.20141208.is.0.3.3+15.04.20141204~rtm-0ubuntu1
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/canonical-devices-system-image/+bug/1403906/+subscriptions
>

[Launchpad Janitor (janitor)]

This bug was fixed in the package thumbnailer - 1.3+15.04.20150106-0ubuntu1

---------------
thumbnailer (1.3+15.04.20150106-0ubuntu1) vivid; urgency=low

  [ Jussi Pakkanen ]
  * Fix a few potential memory corruptions. (LP: #1403906)
 -- Ubuntu daily release <email address hidden> Tue, 06 Jan 2015 18:50:46 +0000

[Launchpad Janitor (janitor)]

This bug was fixed in the package thumbnailer - 1.3+15.04.20150106~rtm-0ubuntu1

---------------
thumbnailer (1.3+15.04.20150106~rtm-0ubuntu1) 14.09; urgency=low

  [ Jussi Pakkanen ]
  * Fix a few potential memory corruptions. (LP: #1403906)
 -- Ubuntu daily release <email address hidden> Tue, 06 Jan 2015 18:50:46 +0000