Fails to generate thumbnail for small videos recorded with camera-app

Bug #1391286 reported by Arthur Mello
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
gst-plugins-bad1.0 (Ubuntu)
In Progress
High
Unassigned
thumbnailer (Ubuntu)
Won't Fix
High
Michi Henning

Bug Description

Videos with less than one sec recorded by camera-app (like when you click on the record button and immediately stops the recording process) have problems while generating thumbnail. This seems to be a problem to gallery-app since if we have one file like this on the collection the thumbnails took a long time to generate.

I added an unit test that seems to fail on this branch: lp:~artmello/thumbnailer/thumbnailer-no_thumbnail_video

Attached a video sample that seems to fail

Tags: rtm14
Revision history for this message
Arthur Mello (artmello) wrote :
Bill Filler (bfiller)
Changed in thumbnailer:
importance: Undecided → High
Changed in thumbnailer (Ubuntu):
importance: Undecided → High
tags: added: rtm14
Revision history for this message
Bill Filler (bfiller) wrote :

The troubling part is that when a thumbnail like this is encountered, it seems to invalidate all of the other thumbnails that have already been loaded. For example, open the Photos tab of the gallery app and wait for the thumbnails to load, scroll up and down. Everything gets loaded. Then copy in the attached file into the ~/Videos directory. This failing thumbnail seems to make the other thumbnail reloaded quite often and things get slower.

Changed in thumbnailer (Ubuntu):
assignee: nobody → Jussi Pakkanen (jpakkane)
Changed in thumbnailer:
assignee: nobody → Jussi Pakkanen (jpakkane)
Revision history for this message
Arthur Mello (artmello) wrote :

It seems that when gallery requests the thumbnail for this file, even if it was generated before, all the process of getting new thumbnails for gallery collection, or getting pointers for the previous generated ones, get a lot of time to complete. That seems to be the reason why all the thumbnails enter in a "reloading" state. Its worse if the problematic file is the first one on the collection, since if it is not a couple of requests seems to be answered before the one that crashes.

Revision history for this message
Jussi Pakkanen (jpakkane) wrote :

What happens here is that thumbnailer calls the gst_app_sink_pull_preroll function which fails. The documentation (http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-base-libs/html/gst-plugins-base-libs-appsink.html#gst-app-sink-pull-preroll) says that this returns null if EOS has been reached. Printf debugging says that this is indeed the case, it returns null and gst_app_sink_is_eos reports true. After EOS no more samples can be obtained from the stream.

The previous gstreamer method we call is gst_element_seek_simple. It returns true indicating success and the seek point we specify is valid (134000000 which is less than the duration 469000000).

Unless someone can show that thumbnailer is using gstreamer incorrectly, it seems to be the case that gstreamer somehow internally causes EOS to be emitted. This would make this a bug in gstreamer rather than thumbnailer.

For comparison I also tested GST_SEEK_FLAG_ACCURATE instead of GST_SEEK_FLAG_KEY_UNIT but the behaviour is the same.

Changed in gst-plugins-base1.0 (Ubuntu):
assignee: nobody → Jim Hodapp (jhodapp)
importance: Undecided → High
Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in gst-plugins-base1.0 (Ubuntu):
status: New → Confirmed
Changed in thumbnailer (Ubuntu):
status: New → Confirmed
Revision history for this message
James Henstridge (jamesh) wrote :

This seems to be related to the androidmedia elements from gstreamer1.0-hybris, so changing the source package link: I can't reproduce it with the standard elements.

In the hope that this might be a quirk of the pipeline used by vs-thumb, I've ported the utility over to using the standard playbin element to auto-plug the full pipeline (the same as mediahub):

https://code.launchpad.net/~jamesh/thumbnailer/thumbnail-playbin/+merge/254025

The results are the same for these extra short videos. Jim: if you've got any ideas, I'd love to hear them.

affects: gst-plugins-base1.0 (Ubuntu) → gst-plugins-bad1.0 (Ubuntu)
Revision history for this message
Jim Hodapp (jhodapp) wrote :

@James I don't have any ideas offhand right now without digging further, but I do know that there indeed is some optimization that is needed for gstamc-hybris to help with situations like this as well as to greatly increase the speed of thumbnailing in general. I've got an item on my team's backlog to investigate this.

Changed in thumbnailer:
status: New → Confirmed
Changed in thumbnailer (Ubuntu):
assignee: Jussi Pakkanen (jpakkane) → nobody
assignee: nobody → Michi Henning (michihenning)
Changed in thumbnailer:
assignee: Jussi Pakkanen (jpakkane) → nobody
assignee: nobody → Michi Henning (michihenning)
Revision history for this message
Alfonso Sanchez-Beato (alfonsosanchezbeato) wrote :

It looks like gstreamer 1.5.2 (about to land in wily) solves the issue for the video attached to the bug. I have also rechecked that 1.4.5 is not able to create a thumbnail for it.

Changed in gst-plugins-bad1.0 (Ubuntu):
status: Confirmed → In Progress
assignee: Jim Hodapp (jhodapp) → nobody
Revision history for this message
Michi Henning (michihenning) wrote :

Nice one, thanks! I'll try it once it lands (or do you have a silo I can pull it from?)

Note that, with the new thumbnailer, the failure no longer delays everything that's queued up behind it. In general, the new thumbnailer achieves much better isolation and concurrency.

Revision history for this message
Alfonso Sanchez-Beato (alfonsosanchezbeato) wrote :

@michihenning it just landed in image 75, devel-proposed in arale (and in latest images for the other phones I guess). I am also under the general impression that gstreamer 1.5.2 greatly reduces the time needed for creating a thumbnail, it would be good if you can confirm this :-).

Revision history for this message
Michi Henning (michihenning) wrote :

No, 1.5.2 has the same problem. It isn't really gstreamer that's at fault here, but the hardware codecs underneath it.

James has been doing a lot of work isolating the problem. Current plan is to switch to software codecs which, going by James's benchmark results, should improve speed massively.

Revision history for this message
Jim Hodapp (jhodapp) wrote :

Switching to the software codecs is the right answer as the setup time of a hardware codec is currently much slower than with an equivalent software codec. I'd like to do some profiling of the hardware codec setup process some time in the near future to see if it's just how it will always be because it's not something we can change, or if there is something we can optimize to make setup a lot faster.

Changed in canonical-devices-system-image:
assignee: nobody → Alejandro J. Cura (alecu)
importance: Undecided → High
milestone: none → backlog
status: New → In Progress
no longer affects: thumbnailer
Revision history for this message
Michi Henning (michihenning) wrote :

Pragmatically, there is probably nothing we can do about this, short of finding someone who is willing to dig into the codec and figure out why it fails (which, in turn, might involve fixing some problem in the Android media elements.

For what it's worth, the thumbnailer will recover gracefully from this kind of failure, in the sense that, once it has tried to produce a thumbnail for such a video and has failed, it will permanently cache the failure and not try again until the modification time of the video changes. In other words, once the thumbnailer has encountered a failure for such a file, for future requests for a thumbnail of the same file, it will deliver a failure notification instantly.

Given the potential cost of fixing the problem compared to how often the problem actually occurs, I'd be inclined to mark this as "won't fix" because it doesn't happen very often that someone takes a video that lasts for less than a second.

Changed in thumbnailer (Ubuntu):
status: Confirmed → Won't Fix
no longer affects: canonical-devices-system-image
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.