indexers keep processor usage at 100%

Bug #137352 reported by insane_alien on 2007-09-04
6
Affects Status Importance Assigned to Milestone
tracker (Ubuntu)
High
Unassigned

Bug Description

In normal usage processor is always at 100% utilization this is due to three processes.

tracker-extract
totem-video-indexer
and to a lesser extent trackerd

this is on a thinkpad R50e upped to 1GB of RAM. the usage appears under ni(NICE?) in top and doesn't appear in gnome-system monitor.

this happens from a default bootup and occurs after 45 seconds. this is the delay time set in the indexing preferences. all other settings are set to low priority.

Related branches

This is probably fixed in svn - we issue a 10 second alarm call which should crash tracker-extract if its taking too long (this might not work if tracker-extract is sleeping as sleep interferes with alarm calls)

But to be sure can you identify the video file thats causing the problem?

you can do this by running trackerd -v 2

this will print out the files as it indexes them so hopefully you can see which one it gets stuck on

once identified can you run totem-video-indexer uri and see if that hangs.

insane_alien (stephen-lucchesi) wrote :

it appears to be several videos being downloaded through bittorrent.

i get a whole load of these errors:

[code](trackerd:6201): GLib-CRITICAL **: g_hash_table_lookup: assertion `hash_table != NULL' failed
[/code]

i'm not sure what URI is. when i run totem-video-indexer i just get "Expects exactly one URI"

Its the full path name of the video file

If its downloading very slowly then it will trigger tracker into trying to index them continually

in latest svn we pause when we detect disk IO outside of tracker so it might help a lot with your case.

insane_alien (stephen-lucchesi) wrote :

any idea when the SVN version will hit the repos?

totem-video-indexer doesn't seem to hang on the torrent. could it be that it just reindexes when a chunk is added?

thanks for the help.

yes it will reindex any file that both changes *and* closes the file write handle. I guess bittorrent really needs to keep the file handle open so it does not trigger us (feel free to file a bug with them)

I hope to have a new release any day now - dunno if it will make tribe 6

Changed in tracker:
importance: Undecided → High
status: New → Fix Committed
John Dong (jdong) wrote :

Yeah, I get this problem too on latest gutsy as of now. KTorrent while downloading a torrent will cause Tracker to peg CPU at 100% for 5-second blips. I don't think the client's closing the file handle though....

Martin Pitt (pitti) wrote :

tracker (0.6.3-0ubuntu1) gutsy; urgency=low

  [ Emilio Pozuelo Monfort ]
  * New upstream release (LP: #130794, #131983, #132320, #137352, #138331,
    #139173, #132505, #131559, #131735, #132710, #133246, #137873, #138778.
  * debian/patches/01-version_fix.patch,
    debian/patches/02-getenv.patch,
    - Removed, fixed upstream.
  * debian/patches/03-system_ioprio.patch: not needed anymore, as tracker
    now tries ioprio system syscalls if available.
  * debian/patches/01_from_upstream_fix_stemming.patch:
    - Added, fixes language selection.

  [ Martin Pitt ]
  * debian/control: Promote o3read to a dependency. That way, updates will get
    it, too, and we avoid making it a dependency of ubuntu-desktop. With the
    external dependency we can avoid installing the internal code copy.

 -- Martin Pitt <email address hidden> Fri, 28 Sep 2007 17:45:16 +0200

Changed in tracker:
status: Fix Committed → Fix Released

If this persists then Its a bug in the torrent client - it affects tracker and beagle (and most likely strigi too) as it opens files in rw mode when it should open them in r (readonly)

if opened in rw mode, inotify will trigger beagle, tracker and other inotify monitors into thinking that the file has finished changing and hence cause constant indexing

Fix 1) make sure the directory for torrents downloads is a no watch directory or a crawl directory in tracker ( use a crawl directory if you still want it indexed)

Fix 2) report bug to the torrent client and get them to fix it

afraid there is nothing we can do at tracker end

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers