Audio/video metadata extraction does not work

[none]

Binary package hint: tracker

Tracker was recently added to Feisty and I noticed that it does not index audio/video metadata. I talked to Jamie McCracken (main Tracker developer) and extraction works fine for him on Edgy, so it appears that the problem is confined to Feisty. Compiling Tracker from source results in the same problem.

Attempting to manually extract the metadata for any video or audio file results in the following:

$ tracker-extract silverman.mpg video/mpeg

GThread-ERROR **: file gthread-posix.c: line 367 (): error 'Cannot allocate memory' during 'pthread_create'
aborting...
Aborted

This is an up-to-date version of Feisty on amd64.

[Baptiste Mille-Mathias (bmillemathias)]

For me it works fine on feisty

baptiste@babel > tracker-extract cooking_mama_012307_qtlow.mov video/mov
Audio:Samplerate=44100;
Audio:Bitrate=128000;
Video:Duration=67;
Audio:Channels=2;
Video:Codec=Sorensen video v.3;
Video:Width=320;
Video:FrameRate=30;
Video:Height=240;
Audio:Codec=MPEG-1 layer 3;

baptiste@babel > tracker-extract /share/Music/Cat\ Power\ -\ The\ Greatest/01\ -\ The\ Greatest.mp3 audio/mpeg
Audio:Samplerate=44100;
Audio:Bitrate=192000;
Audio:Channels=2;
Audio:Album=The Greatest;
Audio:AlbumTrackCount=12;
Audio:Artist=Cat Power;
Audio:Duration=202;
Audio:Title=The Greatest;
Audio:TrackNo=1;
Audio:ReleaseDate=2005;
Audio:Codec=MPEG-1 layer 3;

I'm on x86 architecture.

[Jamie McCracken (jamiemcc-blueyonder)]

amd64 bug in glib or kernel?

This seems tobe limited to amd64 so maybe someone could investigate this in areas outside of tracker?

[didier (did447-deactivatedaccount)]

IIRC tracker is using totem for video/audio so maybe it's a totem-xine vs totem-gstreamer issue?

[none (ubuntu-bugs-nullinfinity-deactivatedaccount)]

I don't think it's a xine vs gstreamer issue. I tried compiling tracker with either xine or gstreamer for extraction and tracker-extract crashed in both cases.

[none (ubuntu-bugs-nullinfinity-deactivatedaccount)]

This bug is still there for me after the latest Feisty updates, but the error message has changed. I now get

$ tracker-extract silverman.mpg video/mpeg

** (<unknown>:14876): WARNING **: Error: A FILE protocol source plugin is required to play this stream, but not installed. (gstplaybasebin.c(1581): gen_source_element (): /playbin:
No URI handler for file)

Some people in #gstreamer were kind enough to look at debugging info for my gstreamer installation but didn't see anything out of the ordinary. That debugging information can be found at http://nullinfinity.org/tmp/dbg.log.gz .

[none (ubuntu-bugs-nullinfinity-deactivatedaccount)]

I just compiled Tracker from SVN and I *still* get the above "No URI handler for file" error when I compile Tracker with the gstreamer backend. If I instead compile Tracker with the xine backend, I get a segmentation fault. Doing a GDB backtrace gives me

(gdb) run silverman.mpg video/mpeg
Starting program: /usr/local/bin/tracker-extract silverman.mpg video/mpeg
[Thread debugging using libthread_db enabled]
[New Thread 47863582330528 (LWP 628)]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 47863582330528 (LWP 628)]
0x00002b8819ae3548 in pthread_join () from /lib/libpthread.so.0
(gdb)

Looks like sort of threading issue in Tracker.

[Pär Lindfors (paran)]

I can confirm this on amd64. Output below for mpeg, mp3 and ogg files.

paran@bacon:~$ uname -a
Linux bacon 2.6.20-15-generic #2 SMP Sun Apr 15 06:17:24 UTC 2007 x86_64 GNU/Linux

paran@bacon:~$ file Katter.mpg
Katter.mpg: MPEG sequence, v1, system multiplex
paran@bacon:~$ tracker-extract Katter.mpg video/mpeg

(<unknown>:5678): GStreamer-CRITICAL **: gst_element_get_bus: assertion `GST_IS_ELEMENT (element)' failed

(<unknown>:5678): GStreamer-CRITICAL **: gst_bus_add_signal_watch_full: assertion `GST_IS_BUS (bus)' failed

(<unknown>:5678): GLib-GObject-WARNING **: invalid (NULL) pointer instance

(<unknown>:5678): GLib-GObject-CRITICAL **: g_signal_connect_data: assertion `G_TYPE_CHECK_INSTANCE (instance)' failed

(<unknown>:5678): GStreamer-CRITICAL **: gst_object_unref: assertion `object != NULL' failed

(<unknown>:5678): GLib-GObject-CRITICAL **: g_object_set: assertion `G_IS_OBJECT (object)' failed

(<unknown>:5678): GLib-GObject-CRITICAL **: g_object_set: assertion `G_IS_OBJECT (object)' failed

(<unknown>:5678): GLib-GObject-CRITICAL **: g_object_set: assertion `G_IS_OBJECT (object)' failed

(<unknown>:5678): GStreamer-CRITICAL **: gst_element_set_state: assertion `GST_IS_ELEMENT (element)' failed

** (<unknown>:5678): CRITICAL **: poll_for_state_change: assertion `extractor && extractor->playbin' failed

(<unknown>:5678): GStreamer-CRITICAL **: gst_element_set_state: assertion `GST_IS_ELEMENT (element)' failed

(<unknown>:5678): GStreamer-CRITICAL **: gst_object_unref: assertion `object != NULL' failed
paran@bacon:~$ file leksaksbilen.mp3
leksaksbilen.mp3: Audio file with ID3 version 23.0 tag, MP3 encoding
paran@bacon:~$ tracker-extract leksaksbilen.mp3 audio/mpeg

(<unknown>:5707): GStreamer-CRITICAL **: gst_element_get_bus: assertion `GST_IS_ELEMENT (element)' failed

(<unknown>:5707): GStreamer-CRITICAL **: gst_bus_add_signal_watch_full: assertion `GST_IS_BUS (bus)' failed

(<unknown>:5707): GLib-GObject-WARNING **: invalid (NULL) pointer instance

(<unknown>:5707): GLib-GObject-CRITICAL **: g_signal_connect_data: assertion `G_TYPE_CHECK_INSTANCE (instance)' failed

(<unknown>:5707): GStreamer-CRITICAL **: gst_object_unref: assertion `object != NULL' failed

(<unknown>:5707): GLib-GObject-CRITICAL **: g_object_set: assertion `G_IS_OBJECT (object)' failed

(<unknown>:5707): GLib-GObject-CRITICAL **: g_object_set: assertion `G_IS_OBJECT (object)' failed

(<unknown>:5707): GLib-GObject-CRITICAL **: g_object_set: assertion `G_IS_OBJECT (object)' failed

(<unknown>:5707): GStreamer-CRITICAL **: gst_element_set_state: assertion `GST_IS_ELEMENT (element)' failed

** (<unknown>:5707): CRITICAL **: poll_for_state_change: assertion `extractor && extractor->playbin' failed

(<unknown>:5707): GStreamer-CRITICAL **: gst_element_set_state: assertion `GST_IS_ELEMENT (element)' failed

(<unknown>:5707): GStreamer-CRITICAL **: gst_object_unref: assertion `object != NULL' failed
paran@bacon:~$ file /usr/share/sounds/KDE_Beep_Double.ogg
/usr/share/sounds/KDE_Beep_Double.ogg: Ogg data, Vorbis audio, mono, 44100 Hz, ~64000 bps, created by: Xiph.Org libVorbis I (1.0.1)
paran@bacon:~$ tracker-e...

[Jamie McCracken (jamiemcc-blueyonder)]

you need to use the full path in tracker-extract

eg tracker-extract /home/jamie/music/test.mp3 audio/mp3

pls retest as above before confirming the problem

[Pär Lindfors (paran)]

Are you sure? Looking at the first comment to this bug it works fine without full path on i386. I can also get good output from other file types without full path:

paran@bacon:~$ tracker-extract vga1.png image/png
Image:Comments=Created with The GIMP;
Image:Height=315;
Image:Width=1078;

However I do get a different crash when specifying full path for audio/video files. Ogg files still produce no output at all, with or without full path.

paran@bacon:~$ tracker-extract /home/paran/Katter.mpg video/mpeg

GThread-ERROR **: file gthread-posix.c: line 367 (): error 'Cannot allocate memory' during 'pthread_create'
aborting...
Aborted
paran@bacon:~$ tracker-extract /home/paran/leksaksbilen.mp3 audio/mpeg

GThread-ERROR **: file gthread-posix.c: line 367 (): error 'Cannot allocate memory' during 'pthread_create'
aborting...
Aborted
paran@bacon:~$ file finalman-hus.ogg
finalman-hus.ogg: Ogg data, Vorbis audio, stereo, 44100 Hz, ~112001 bps, created by: Xiph.Org libVorbis I (1.0.1)
paran@bacon:~$ tracker-extract /home/paran/finalman-hus.ogg application/ogg
paran@bacon:~$ tracker-extract finalman-hus.ogg application/ogg

There is no lack of free memory to allocate:
paran@bacon:~$ free
             total used free shared buffers cached
Mem: 4046192 4002520 43672 0 72888 2187080
-/+ buffers/cache: 1742552 2303640
Swap: 0 0 0

[Emilio Pozuelo Monfort (pochu)]

Looks like this is confirmed on amd64...

Jamie, do you still think this is a glib or kernel bug?

[alextud (alextud)]

I found why its not working, after I comment this line :
 if (setrlimit (RLIMIT_AS, &rl) != 0) g_printerr ("Error trying to set resource limit for tracker-extract\n");
 it's work fine.
I think is related with setting virtual memory, and on x86_64 is not working correctly. Maybe we can bypass setting memory limit .

[Jamie McCracken (jamiemcc-blueyonder)]

Alextud,

Can you experiment with changing RLIMIT_AS to RLIMIT_DATA?

The purpose of the call is to limit memory use of helper apps to 128MB maximum - it just might need another param (see man setrlimit)

[alextud (alextud)]

Yes, after changing to RLIMIT_DATA, program is running well.
But this results don't appear in tracker-search-tool :(( , also if I
don't start trackerd from terminal tracker-stats, tracker-search stop
with this kind of message :

** (tracker-stats:10773): WARNING **: tracker-stats: an error has
occured: Process /usr/bin/trackerd exited with status 0

tracker-search maria
tracker-search: internal tracker error: Process /usr/bin/trackerd
exited with status 0

On 06/07/07, Jamie McCracken <email address hidden> wrote:
> Alextud,
>
> Can you experiment with changing RLIMIT_AS to RLIMIT_DATA?
>
> The purpose of the call is to limit memory use of helper apps to 128MB
> maximum - it just might need another param (see man setrlimit)
>
> --
> Audio/video metadata extraction does not work
> https://bugs.launchpad.net/bugs/81217
> You received this bug notification because you are a direct subscriber
> of the bug.
>

[Jochen Kemnade (jochenkemnade)]

you are probably already aware of this, but i wanted to make sure you haven't overlooked the comment in line 175 of tracker-extract.c (that is before another call to setrlimit), which says:
"/* Set memory usage to max limit (128MB) - some archs like AMD64 machines do not fully support this or are buggy */"
that's why there is a conditional compilation of the following lines depending on "defined(__i386__)".
maybe you could do the same for the other call (around line 484) to make it work for 0.6.0 on x86_64?
i've not really familiar with the code and i just took a quick look. if that sounds like a bad idea, just ignore me. ;-)
hth

[alextud (alextud)]

For me this bug only occur for setrlimit (RLIMIT_AS, &rl) line 484,
function main (). If I disable conditional compilation it also work.
Sorry about not saying in previous post, that I test it only on line
484, when I filled this bug.

By the way, tracker-search-tool now show results correctly :)), good job.

[none (ubuntu-bugs-nullinfinity-deactivatedaccount)]

With current SVN of Tracker, this bug has been fixed.

[Sebastien Bacher (seb128)]

 tracker (0.6.0-1) unstable; urgency=low
 .
   * New upstream release.
     - Uses XDG directories to store the settings, cache, database and log
       files. Closes: #414860
     - Fixes the crash of the deskbar applet at session start. Closes: #434078
     - Does not fall back to polling when the inotify limit is exceeded.
       Closes: #412089
     - Correctly uses xdg-open to open the search results with the preferred
       application. This requires the xdg-utils package to be installed, which
       is a Recommends of tracker-search-tool. Closes: 415705
   * Removed debian/man/ as the man pages are now shipped upstream.
     Update tracker.manpages, tracker-search-tool.manpages and
     tracker-utils.manpages accordingly.
   * Removed patches that were applied upstream or are now obsolete
     - debian/patches/20-trackerd_manpage.patch
     - debian/patches/30-deskbar_plugin_max_results.patch
     - debian/patches/40-inotify_syscalls.patch
     - debian/patches/50-expand_pathname.patch
     - debian/patches/60-check_pointer.patch
     - debian/patches/70-check_magic_file_results.patch
   * debian/copyright
     - The htmless sources are not shipped anymore and the qdbm sources are now
       in a separate subdirectory. Update the copyright file accordingly.
   * debian/libtracker-gtk-dev.install, debian/libtracker-gtk0.install
     - Added. libtracker-gtk is a new library that provides handy GTK+ widgets
       for applications that use tracker.
   * debian/control
     - Drop libmagic-dev from Build-Depends, add libunac1-dev and libqdbm-dev.
     - Add djvulibre-bin and gnumeric to Suggests. They are used as external
       filters for indexing djvu, csv and spreadsheet documents.
     - Add binary packages libtracker-gtk0 and libtracker-gtk-dev.
     - Update the long description of tracker-utils and include the new tool
       "tracker-status".
     - Add binary package tracker-dbg which provides the debugging symbols of
       trackerd and its utilities. Closes: #412140
   * debian/rules
     - Make sure we link against the system qdbm library.
     - Enable the support for libunac.
     - Explicitly enable the compilation of the GUI, the preferences applet
       and libtracker-gtk.