trackerd eating CPU and memory

[Ryan Grieve]

Binary package hint: tracker

System:

Lenovo IBM Thinkpad r60e
1.7GHz Intel Core Duo
1024MB DDR
Intel 945GM w/ 950GMA
Intel ProWireless 3945
Ubuntu Gutsy Gibbon (as of 06/08/07 updates)

Problem:

From the moment the GDM screen appears after boot trackerd starts to devour system resources bringing keystrokes, mouse movements and X refreshes to an almost complete standstill.

Switching out a virtual terminal allows me to login and run 'top'. Revealing that trackerd is taking from 40% - 90% CPU and fairly consistently taking ~85% of available memory. Killing the app is fruitless as it appears to be persistent.

Temporary Solution:

I have uninstalled tracker for now.

[Jamie McCracken (jamiemcc-blueyonder)]

Did you try setting a throttle in tracker-preferences?

Also pls get us a log by running trackerd -v 2 and sending us the log file in ~/.local/share/tracker/tracker.log

[Jamie McCracken (jamiemcc-blueyonder)]

also can you isolate the problem at all?

EG is it email indexing or file indexing that causes the problem?

you can disable email indexing in tracker-preferences and then relaunch tracker with:

trackerd --reindex

Also can you send us your tracker.cfg file in ~/.config/tracker/tracker

the default 60 second index delay should prevent tracker from indexing when GDM first appears

[Jamie McCracken (jamiemcc-blueyonder)]

we need much more info in order to diagnose this problem

[Jamie McCracken (jamiemcc-blueyonder)]

Checking gutsy kernel, ioprio has changed interface from feisty which means no disk scheduling is done by trackerd (hence it causes slow downs to anything accessing the disk)

will fix shortly...

The memory issue Im not sure about

[Martin Pool (mbp)]

I'm suffering this too, trackerd had just been hammering my laptop for nearly 2 hours. I have previously set the throttle to 10. The amount of io is such that the machine is only barely usable - switching windows takes a few seconds, etc.

tracker-preferences is not installed in my current gutsy machine.

Running just trackerd -v2: it sleeps for 45s, then reports that it's watching 8111 directories, and then it rests. So maybe the problem was in building the initial index?

[Martin Pool (mbp)]

I'm not going to post the whole log because it contains personal information.

Basically it's listing every file under my home directory like this:

08 Aug 2007, 15:12:27:960 - Indexing /home/mbp/work/pybaz--devo--0/{arch}/++<email address hidden>/pybaz--devo--0--patch-124/pybaz/backends/knotted.py with service Development and mime text/x-python (new)

It's getting through several per second but I have a lot of files.

I'm sorry but I'm going to have to kill this off or my laptop will go flat before I can do anything else with it.

[Thom Pischke (thom-pischke)]

I'm seeing this too to some extent. In my case trackerd can often be found using 20-50% of the CPU, though it's not as extreme as the cases above. Just uninstalled it for now, since I'm trying out Google Desktop anyway.

[Jamie McCracken (jamiemcc-blueyonder)]

IOPrio is being correctly set - ionice reports best effort 7 (which is low priority)

Problem confined to Gutsy kernel. Indexing same set of files on feisty is considerably faster and has no discernible effect on slowing down other running apps.

This is therefore a critical problem with gutsy kernel - will take this up with kernel lads

[Thom Pischke (thom-pischke)]

Actually I'm booting the feisty kernel due to broadcom driver issues in the gutsy kernel, and I'm seeing trackerd performance problems even in the feisty kernel.

[Jamie McCracken (jamiemcc-blueyonder)]

I cant replicate at all on feisty - its much faster and no slow down is noticeable (thats the whole fiesty btw with tracker 0.6.0 compiled on it and no gutsy stuff)

if feisty kernel under gutsy is also causing problems then it could be something in userspace or a gutsy setting - hmmm

Im pretty sure its a disk access or write issue, perhaps not a big enough write buffer for the disk?

[Robert Collins (lifeless)]

perhaps trackerd should use O_DIRECT if it doesn't already, which as trackerd is reading files just once to index really sends the right signal to the mm - that this data shouldn't affect the system wide cache.

[Mike Richards (mrmikerich)]

I've just seen this on a fairly fresh install of Tribe 5. My laptop had some problem coming out of suspend so I had to restart gdm (/etc/init.d/gdm restart), and when I logged in immediately afterwards I could see that trackerd was eating an entire CPU of this dual-core machine. I only let it run for a few minutes before killing it manually.

I don't know if it is normal for trackerd to run for several minutes like this, but I suspect if this was a single-CPU machine, the rest of the machine would have been completely unusable due to trackerd hogging the CPU.

The "fix" of removing trackerd from the system seems to work though :-)

[Jamie McCracken (jamiemcc-blueyonder)]

trackerd is schedules - it will happily use 99% of cpu if no other process is using it - this is normal and desired behaviour.

it runs at nice +19 so if you do something like play an opengl game trackerd goes to about 1% cpu usage

indexing is cpu intensive but if you prefer to throttle it down anyhow we have a throttle setting in tracker-preferences (it really is not necessary to do this unless your machine is getting hot or annoying fans are howling)

[Peter Clifton (pcjc2)]

It eats battery and appears to reduce IO performance quite a bit (increasing latency).

I went with Mike's fix some while ago, and stopped trackerd loading for now.

[Jamie McCracken (jamiemcc-blueyonder)]

Peter,

yes thats right - and we are working on fixes for those

My point is the CPU at 99% is a red herring as you rightly say the slow down in the system appears to be caused by IO wait which we hope to have a fix for soon

The battery eating is also being worked on (it will pause indexing when on battery)

we expect to have fixes for these in time for tribe 6

[Thom Pischke (thom-pischke)]

Just for the record, I dislike anything that pegs the CPU at 100% for more than a few seconds. Not only does it make me spend time trying to figure what's going on, it forces the fan to come on and stay on, which is annoying and wasteful. I installed and am testing Google Desktop for Linux. One thing I liked about it right away is that it indexed my computer in less than one day, and it NEVER pegged the CPU. At least not for more than a few seconds at a time. This seems to me much more 'polite' than just grabbing the whole CPU for who knows how long, regardless of how 'nice' it is. Laptops aren't servers.

[Darik Horn (dajhorn)]

I had this bug on several laptop computers upgraded to Gutsy. Looking at the output of `trackerd -v 2`, it seems that any home directory with more than a few hundred files will cause the tracker process to pin system utilization to 100% and make the computer unusable longer than most people will wait.

[Darik Horn (dajhorn)]

Why was this bug hidden from the public? -- I'm relating it to #132320 so that it doesn't fall out of the Launchpad database so early.

The behavior of trackerd makes the Gutsy slow and unresponsive on many computers. Blaming it on recent kernel changes is a weak excuse.

[Jamie McCracken (jamiemcc-blueyonder)]

Its not valid as its not a cpu issue but a disk iowait issue - see bug 131983

[Roland Ronquist (roland-ronquist)]

no matter if it is beneficial or not having this trackerd track down every single
bit in every file fast or not, it is definitely not good for anybody if your systems
feels like you have upgraded to an 1 MHz 6502 CPU instead of that multi
GHz monster that is actually juggling all your computrons.

On top of that, not everybody has their systems designed for extended
period extreme CPU-loads. That can become a cooling disaster regardless
if you are on batteries or not.

if we want to stay competitive with other systems, we must provide an
user experience where the system is responsive for interactive use,
and then it is very much counter productive having daemons munching
up every last cycle.

on my computer the trackerd blasts on at full cpu load for hours.
having a work tool that is not ready or usable for any work until it
is time for the user to go home is not clever :-(

btw has anybody heard about the efforts of reducing the over all power
consumption of the computers on this planet? that effort has obviously
not been part of the design considerations of this greedy little power-tool.

[aot2002 (jasonbronson)]

mine doing this now too today
i was on idle then came back to find CPU going nuts

sudo tail /root/.local/share/tracker/tracker.log
[sudo] password for jason:
19 Sep 2007, 06:54:38:551 - ERROR: while reading file /usr/share/tracker/sqlite-stored-procs.sql on line 170
jason@jason-laptop:~$ ls -l /usr/share/tracker/sqlite-stored-procs.sql
-rw-r--r-- 1 root root 15938 2007-09-28 17:12 /usr/share/tracker/sqlite-stored-procs.sql

trackerd -v 2

Tracker version 0.6.3 Copyright (c) 2005-2007 by Jamie McCracken (<email address hidden>)

This program is free software and comes without any warranty.
It is licensed under version 2 or later of the General Public License which can be viewed at http://www.gnu.org/licenses/gpl.txt

Initialising tracker...

** (trackerd:19585): WARNING **: Tracker daemon is already running - exiting

I had to remove TRACKER program
apt-get remove tracker

[Jamie McCracken (jamiemcc-blueyonder)]

please do killall -9 trackerd and then run trackerd

you have the old 0.6.2 version running and that needs to be stopped before the new one (0.6.3) can run

[naughtykid (naughtykid001)]

Hi, I'm having the same symptoms actually.

My system freeze after the trackerd took out the CPU resources up to 60%.

However, my system appeared normal after I took out one piece of RAM. So as my suggestion, please take a check on your RAM as well? Thank you.

[arthurv (arthur-amarra)]

Hi, i'd like to report trackerd consuming 100% of 1 of my CPUs (Core 2 Duo), tracker 6.4, using sqlite 3.5.3, on a gentoo system.

I've disabled it for now, sorry for the lack of more info - tell me what to do and I'll try provide more details.