tracker-search-tool reports no result

[Ka-Hing Cheung]

Binary package hint: tracker

tracker doesn't seem to work at all, search in tracker-search-tool produces no result. I've upgraded to gusty for more than a week already, and it looks like it already indexed quite a bit of data:

~/.cache/tracker $ ls -lh
total 115M
-rw------- 1 khc khc 45M 2007-09-06 19:56 email-content.db
-rw------- 1 khc khc 12K 2007-09-06 10:19 email-index.db
-rw------- 1 khc khc 52M 2007-09-06 19:56 email-meta.db
-rw------- 1 khc khc 7.0M 2007-09-06 19:55 file-content.db
-rw------- 1 khc khc 12K 2007-09-06 10:19 file-index.db
-rw------- 1 khc khc 11M 2007-09-06 19:55 file-meta.db

I looked around and it doesn't seem like tracker has any log, or any way to get debug output at all. Running tracker-search-tool in terminal produces no error or any message.

[Ka-Hing Cheung (kahing)]

ahh I found ~/.local/share/tracker/tracker.log, but all it has is this:

06 Sep 2007, 19:55:38:935 - ERROR: no matching account found for email
06 Sep 2007, 19:55:38:935 - ERROR: no matching account found for email
06 Sep 2007, 19:55:38:935 - ERROR: no matching account found for email
06 Sep 2007, 19:55:38:935 - ERROR: no matching account found for email
06 Sep 2007, 19:55:38:971 - ERROR: no matching account found for email
06 Sep 2007, 19:55:38:972 - ERROR: no matching account found for email
06 Sep 2007, 19:55:38:972 - ERROR: no matching account found for email
06 Sep 2007, 19:55:38:972 - ERROR: no matching account found for email06 Sep 2007, 19:55:38:973 - ERROR: no matching account found for email
06 Sep 2007, 19:55:38:973 - ERROR: no matching account found for email
06 Sep 2007, 19:55:38:973 - ERROR: no matching account found for email
06 Sep 2007, 19:55:38:973 - ERROR: no matching account found for email
06 Sep 2007, 19:55:38:998 - ERROR: no matching account found for email
06 Sep 2007, 19:55:38:998 - ERROR: no matching account found for email
06 Sep 2007, 19:55:38:998 - ERROR: no matching account found for email
06 Sep 2007, 19:55:38:999 - ERROR: no matching account found for email

[Ka-Hing Cheung (kahing)]

Some more investigating. Setting verbosity to 2, now I get a bunch of this for the last 10 minutes (still going!):

06 Sep 2007, 20:19:10:163 - deleting email email://1105674638.11952.1@struggle/javabsp;uid=8606
06 Sep 2007, 20:19:12:581 - deleting email email://1105674638.11952.1@struggle/javabsp;uid=8608
06 Sep 2007, 20:19:14:909 - deleting email email://1105674638.11952.1@struggle/javabsp;uid=8609

Using tracker-files -m text/html found a bunch of html files, but using tracker-search with terms in those html files produces no result as well, and this in log:

06 Sep 2007, 20:21:15:345 - Executing search with params Files, vpn
06 Sep 2007, 20:21:15:346 - search returned no results

[Ka-Hing Cheung (kahing)]

running `trackerd -v 3 -R` gives lots of:

(trackerd:4869): GLib-CRITICAL **: g_hash_table_insert: assertion `hash_table != NULL' failed

[Ka-Hing Cheung (kahing)]

Looking at the sqlite db itself it looks like it's populated:

$ sqlite3 file-content.db
sqlite> select COUNT(*) from ServiceContents;
6112

[Ka-Hing Cheung (kahing)]

Disabling Evolution indexing makes those hash_table != NULL messages mostly disappear, but still no search result.

[Jamie McCracken (jamiemcc-blueyonder)]

We noticed this problem and have already fixed in svn

We will have a new release out with this fix shortly

[Mathieu Pellerin (nirvn-asia)]

updated to 0.6.2-2ubuntu2 and problem still persists, searches return 0 result (both from the tracker-search-gtk and the command line tracker-search)

also get a bunch of GLib-CRITICAL message mentioned above if I run "trackerd -v 3 -R"

[Jamie McCracken (jamiemcc-blueyonder)]

fix will be in 0.6.3 release which is due this week

[Martin Pitt (pitti)]

tracker (0.6.3-0ubuntu1) gutsy; urgency=low

  [ Emilio Pozuelo Monfort ]
  * New upstream release (LP: #130794, #131983, #132320, #137352, #138331,
    #139173, #132505, #131559, #131735, #132710, #133246, #137873, #138778.
  * debian/patches/01-version_fix.patch,
    debian/patches/02-getenv.patch,
    - Removed, fixed upstream.
  * debian/patches/03-system_ioprio.patch: not needed anymore, as tracker
    now tries ioprio system syscalls if available.
  * debian/patches/01_from_upstream_fix_stemming.patch:
    - Added, fixes language selection.

  [ Martin Pitt ]
  * debian/control: Promote o3read to a dependency. That way, updates will get
    it, too, and we avoid making it a dependency of ubuntu-desktop. With the
    external dependency we can avoid installing the internal code copy.

 -- Martin Pitt <email address hidden> Fri, 28 Sep 2007 17:45:16 +0200

[Sakumatti Luukkonen (sakumatti-luukkonen)]

There seems to be another problem with this. I'm not getting any search results at all with the newest tracker. (0.6.3-0ubuntu2)

tracker-stats indicates that I have many files indexed, while tracker-search-tool or tracker-search can't find any.

saku@thor:~$ tracker-stats

-------fetching index stats---------

default : 0
Files : 21883
Folders : 1695
Documents : 72
Images : 5228
Music : 10999
Videos : 94
Text : 857
Development : 735
Other : 2203
Emails : 171
EvolutionEmails : 171
ThunderbirdEmails : 0
KMailEmails : 0
EmailAttachments : 0
EvolutionAttachments : 0
KMailAttachments : 0
Conversations : 0
GaimConversations : 0
Applications : 249
------------------------------------

There seems to be quite a bit of data too.

saku@thor:~/.cache/tracker$ ls -l
total 56856
-rw------- 1 saku saku 12288 2007-10-10 19:32 email-contents.db
-rw------- 1 saku saku 1227616 2007-10-11 19:02 email-index.db
-rw------- 1 saku saku 253952 2007-10-11 19:04 email-meta.db
-rw------- 1 saku saku 3796992 2007-10-11 18:41 file-contents.db
-rw------- 1 saku saku 17126720 2007-10-11 19:04 file-index.db
-rw------- 1 saku saku 34532352 2007-10-11 19:07 file-meta.db
-rw------- 1 saku saku 1180096 2007-10-11 19:02 file-update-index.db

In my tracker.log there are this kind of errors. My Music collection is mostly flac files, so there could be a problem with flac files. Most of them have also replaygain tags.

saku@thor:~/.cache/tracker$ cat ~/.local/share/tracker/tracker.log
11 Oct 2007, 19:07:11:104 - ERROR: metadata Audio:TrackPeakGain not found
11 Oct 2007, 19:07:11:580 - ERROR: metadata Audio:TrackPeakGain not found
11 Oct 2007, 19:07:11:796 - ERROR: metadata Audio:TrackPeakGain not found
11 Oct 2007, 19:07:12:065 - ERROR: metadata Audio:TrackPeakGain not found
11 Oct 2007, 19:07:12:316 - ERROR: metadata Audio:TrackPeakGain not found
11 Oct 2007, 19:07:12:668 - ERROR: metadata Audio:TrackPeakGain not found
11 Oct 2007, 19:07:12:992 - ERROR: metadata Audio:TrackPeakGain not found
11 Oct 2007, 19:07:14:545 - ERROR: metadata Audio:TrackPeakGain not found
11 Oct 2007, 19:07:14:785 - ERROR: metadata Audio:TrackPeakGain not found
11 Oct 2007, 19:07:15:019 - ERROR: metadata Audio:TrackPeakGain not found
11 Oct 2007, 19:07:15:309 - ERROR: metadata Audio:TrackPeakGain not found

[Jamie McCracken (jamiemcc-blueyonder)]

use tracker-status to see if indexing is complete

results will only be available when indexing is finished

if no help then also try :

killall trackerd
trackerd --reindex

and wait for index to finish

let us know if problem persists

[Ka-Hing Cheung (kahing)]

I am also not able to get any search result. It worked at one point after 0.6.3, but either a subsequent update broke it, or it just stopped working on its own. tracker-status returns that tracker is idle, and tracker-stats gives plenty of stuff indexed. I don't think doing a --reindex is going to help much here, since even if it "fixes" it that's just papering over the problem. Anything you want me to do to track down the problem?

[Jamie McCracken (jamiemcc-blueyonder)]

Ka-Hing,

it might just be update problem from 0.6.2 to 0.6.3

0.6.3 should automatically reindex when upgraded but you must make sure you do killall trackerd after upgrading to 0.6.3

doing a reindex is what it would do anyhow

[Ka-Hing Cheung (kahing)]

fair enough, I did killall trackerd and --reindex, when the cpu becomes idle again I did tracker-status:

$ time tracker-status
Tracker daemon's status is �Ë.

real 0m25.013s
user 0m0.004s
sys 0m0.000s

For some reason the returned status is garbled, trackerd memory corruption? Also, notice that it took 25 seconds, the entire time cpu is idle.

If I use tracker-search-tool to search, it tells me the search timed out. So next I tried to kill trackerd by Ctrl-C in the reindex terminal, but it did not terminate. Then I tried killall trackerd, and still not terminating. Then I attached gdb to it:

(gdb) bt
#0 0xffffe410 in __kernel_vsyscall ()
#1 0xb7ba0e96 in nanosleep () from /lib/tls/i686/cmov/libc.so.6
#2 0xb7ba0ca7 in sleep () from /lib/tls/i686/cmov/libc.so.6
#3 0xb7d9b94b in sqlite3UnixSleep () from /usr/lib/libsqlite3.so.0
#4 0xb7d99767 in ?? () from /usr/lib/libsqlite3.so.0
#5 0x000003e8 in ?? ()
#6 0xb7dd0e40 in ?? () from /usr/lib/libsqlite3.so.0
#7 0xbfb10528 in ?? ()
#8 0xb7d995c7 in sqlite3InvokeBusyHandler () from /usr/lib/libsqlite3.so.0
Backtrace stopped: frame did not save the PC

So next I kill -9'ed it, then restart without --reindex. Looks like it indexed a bunch more stuff, but I am going to sleep now...

[Ka-Hing Cheung (kahing)]

okay, works after it finished indexing again.

[Nicholas J Kreucher (kreucher)]

I also have this problem. I upgraded to 7.10 and Tracker worked great--for about a day. Now it returns no results. Also seems to have tons indexed.

$ tracker-stats

-------fetching index stats---------

default : 0
Files : 87202
Folders : 8017
Documents : 9854
Images : 6991
Music : 579
Videos : 71
Text : 13860
Development : 31384
Other : 16446
Emails : 0
EvolutionEmails : 0
ThunderbirdEmails : 0
KMailEmails : 0
EmailAttachments : 0
EvolutionAttachments : 0
KMailAttachments : 0
Conversations : 0
GaimConversations : 0
Applications : 604
------------------------------------

I will run trackerd --reindex to see if it fixes the problem, and will report back. Note that SIGTERM did not kill trackerd (I waited several minutes). It caught the signal, but never exited:

20 Oct 2007, 11:03:24:908 - Received signal 'Terminated' so now shutting down
20 Oct 2007, 11:03:24:908 - Total allocations = 9376, total deallocations = 9348

So I kill -9'ed it.

[Jamie McCracken (jamiemcc-blueyonder)]

kill -9 can corrupt the index and prevent searching again (until reindexed)

reindexing should solve the problem

[Nicholas J Kreucher (kreucher)]

Results so far... after several hours, but apparently before it was done indexing, trackerd exit'd with this error:

  ERROR: excessive busy count in query GetServiceID and thread main.

Running tracker-status appears to have started trackerd back up again:

  $ tracker-status
  Tracker daemon's status is Indexing.

  $ ps -lC trackerd
   F S UID PID PPID C PRI NI ADDR SZ WCHAN TTY TIME CMD
  0 S 1000 21348 1 5 94 19 - 8075 - ? 00:00:48 trackerd

It indexed for about 2min, and now there isn't any activity. Log says: file changes to /home/kreucher/nick/ipod_sync/temp.log is pausing tracker. However, this file doesn't exist and lsof reports no open files in that directory or it's parent at all. So not sure why the indexer is paused. tracker-status says status is Indexing.

Still unable to get any results from tracker-search :( Log always reports "search returned no results"

Hmm, after waiting a few minutes (wanted to see if it would start to index again), it appears something additionally is wrong: trackerd process is still running, but tracker-search just times out:

  $ tracker-search nick
  tracker-search: internal tracker error: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.

And tracker-status returns garbage:

  $ tracker-status
  Tracker daemon's status is �Ë
.

Any ideas?

[Jamie McCracken (jamiemcc-blueyonder)]

the bugs with busy count and trackerd status are fixed in svn so maybe that will help

searching while indexing can produce timeouts - its not really a bug as such although the error msg should be improved

I would suggest getting a subset of your home directory indexed (set settings in Tracker prefs) and the launch with --reindex

Let it finish indexing and then see if you can search

[Nicholas J Kreucher (kreucher)]

Ok, restricting trackerd to a small subset of my home directory allowed the --reindex to finish just fine. I was also then able to search.

Next I updated tracker to svn, and opened it up to my whole home directory again, and ran --reindex. Woke up this morning to a "hung" trackerd...

  $ tracker-status
  tracker-status: internal tracker error: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.

  $ ps -lC trackerd
  F S UID PID PPID C PRI NI ADDR SZ WCHAN TTY TIME CMD
  0 S 1000 13113 21589 2 94 19 - 12982 - pts/4 00:23:26 trackerd

Looks like it indexed quite a bit before it hung though, last status report and index entry in the log:

 Current memory usage is 30460, word count 92908 and hits 580117
 indexing #89100 - /home/kreucher/nick/house-notes.odt

And last message in the log was "file changes to [...] is pausing tracker". Sending a SIGTERM to trackerd yields no response :(

I don't want to clutter up this bug... shall I take this to a mailing list / or another bug instead?