beagled-helper loads CPU

[vilbara]

I've upgraded to edgy and now several times a day beagled-helper becomes active for quite a long time and fully loads CPU.

Here is a copy-paste from top:
5246 vilbara 15 0 109m 61m 39m S 92.1 6.1 35:06.43 beagled-helper

That's not nice as I use notebook, it heats up and becomes loud.

[Giuped (giuped)]

I can confirm this bug on two installation of edgy

[Chris Irwin (chrisirwin)]

Confirmed on my laptop (causes fan to come on). My issue is with beagled, apparently however.

All I need to do is sit leave my laptop idle for a few minutes, and about the same time it takes the screensaver to come on is when this issue occurs.

USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
chris 5174 73.0 1.7 113272 34832 ? Sl 01:37 487:24 beagled --debug /usr/lib/beagle/BeagleDaemon.exe --bg

[Eduardo Durany Fernández (edurany)]

Confirmed. I have the same problem as Chris Irwin.

[atie (atie-at-matrix)]

Status changed to "Confirmed", of course I have same problem with beagle 0.2.9-1ubuntu3. And it reminds me early version of beagle 2.x days.

[Thom Pischke (thom-pischke)]

I also have this problem since updating to edgy. This causes my laptop to heat up and the fan to turn on, so it's a very apparent problem. Hope this will be fixed soon.

Can I just uninstall beagle as a workaround?

[Thomas Wolfe (tomwolfe)]

looks like it is probably fixed in the new version just released
http://joeshaw.org/2006/10/17/426
0.2.11
edgy is still on 0.2.9 i think
hopefully we don't have to wait for the next release.

[Dean Loros (autocrosser)]

I also have had this problem for about a month--I went into my sessions & disabled beagled / beagle-helper ---seems to not made a difference to my system--will enable them with the next update to see if there is a change.

[Thom Pischke (thom-pischke)]

I just uninstalled beagle and beagled, that took care of the problem. Will try reinstalling in Edgy + 1, or when this bug is fixed, whichever comes first.

[Robin Sheat (eythian)]

It seems to be _very_ slow on some files, this may be causing it. E.g. currently it is taking up 50% of CPU on an SMP system running through files that it doesn't understand:
061024 1744598980 09420 IndexH DEBUG: +file:///home/robin/eclipse/workspace/Libraries/Old Jars/IconBank.jar
061024 1745025511 09420 IndexH DEBUG: Helper Size: VmRSS=45.7 MB, size=2.97, 49.2%
...
061024 1746176545 09420 IndexH DEBUG: Helper Size: VmRSS=46.2 MB, size=3.00, 50.0%
061024 1746184819 09420 IndexH DEBUG: No filter for /home/robin/eclipse/workspace/Libraries/Old Jars/IconBank.jar (application/zip)

That was about a minute and a half on one file, at full CPU, that it then gave up reading. I'm hoping that the percentage counter is how close to finishing it is, and that it'll give me my CPU back after then.

[Frank Niedermann (fbn)]

I have the same behavior here (after leaving computer / idle 10+ minutes CPU load is on 100% with beagle). After killing the beagle process everything is fine again.

Not having any word documents on my computer (read about issues with that).

Some other people seem to have this issue: http://www.ubuntuforums.org/showthread.php?t=162089&highlight=cpu+beagle

[Hannes Ovrén (kigurai)]

This problem is still present in released Edgy. Annoying.

[Robin Sheat (eythian)]

I'm seeing it in Edgy too. What I've found helps as a workaround is to find the problem filetypes (e.g. .jar files), and add them to the filetype exclusion list (open beagle -> search menu -> preferences -> indexing tab -> privacy section)

It's not ideal, but it means it doesn't take forever at full CPU.

[Frank Niedermann (fbn)]

I think I solved the issue here with enabling extended attributes for beagle:
http://beagle-project.org/Enabling_Extended_Attributes

But I'm not 100% sure ...

[Robin Sheat (eythian)]

I don't yet know if that helps beagle's CPU usage (I suspect it doesn't, but that's just an unfounded hunch), but an FYI for people interested: the instructions do work on ReiserFS 3 (the default ReiserFS) if it's been formatted reasonably recently. You can test by installing 'attr', and using attr -s test ~/file to set something, and then attr -g test ~/file to get it again.

The problem with the CPU usage that I've been observing is that it'll scan files (in my case, .jar and MS-DOS .exe files), and it'll take a few minutes to scan each one using 100% CPU at the time. Not too bad if you have one or two of them, but really bad if you have a directory full of a hundred or so. I don't think this'll be helped by how it knows whether or not they've been indexed.

[Dieter Komendera (dieter-komendera-com)]

I have the same problem with edgy. Very annoying with a laptop...

[Thom Pischke (thom-pischke)]

Ah, so jar files slow things down. That explains why the problem has been
especially irksome for me as a java developer.

On 10/28/06, Dieter Komendera <email address hidden> wrote:
>
> I have the same problem with edgy. Very annoying with a laptop...
>
> --
> beagled-helper loads CPU
> https://launchpad.net/bugs/64326
>

[Robin Sheat (eythian)]

I haven't had beagled-helper loading the CPU down recently, I think it's finished indexing everything. However, when I leave the computer alone for half an hour, now 'beagled' (not 'beagled-helper') starts up, using 100% of CPU. As soon as I move the mouse or type something, it stops. It seems like this is designed behaviour, but it's using a lot more than I'd expect:

$ ps aux|grep beagled
robin 6245 54.9 2.8 176764 59352 ? Sl Oct28 807:52 beagled --debug /usr/lib/beagle/BeagleDaemon.exe --bg

(i.e. it's used 807 minutes of CPU time in the past day)

In this case, unlike the case with beagled-helper, nothing is logged to indicate what it's doing.

[Matthew Caron (matt-mattcaron)]

I have the same problem since upgrading to Edgy.

[Sebastian Breier (tomcat42)]

Same problem.

[Stephen Chu (standby)]

if the process is really necessary, it should behave like Google Desktop - Just do the job in the background while the computer is idle; pause the indexing when the computer is busy. At least the priority is not that high.

[Erik (erik-erikwickstrom)]

In my case the CPU usage has been reasonable, but memory usage is another story! Beagld is using 250mb of ram on my system right now! Ridiculous!

$ ps auxh | grep beagled
erik 6691 0.4 24.2 314300 250512 ? Sl Oct28 5:38 beagled --debug /usr/lib/beagle/BeagleDaemon.exe --bg

[Sam Liddicott (sam-liddicott)]

it should NOT be running at default priority either!!!!

It should nice itself

Mines been running at 24 hours of 100% cpu.

Dumb mutt!

[Slogger (slogger)]

The process "beagled-helper" ID 5969 on my system runs fine for the first few minutes after booting into desktop, but all of a sudden it will suddenly start eating up all my memory, going from something like 8-14 megs to being with to taking up ALL of my system's RAM, like 1000+ megs...

Related?

[Slogger (slogger)]

https://launchpad.net/distros/ubuntu/+source/beagle/+bug/69752

[Erik Meitner (e.meitner)]

I have a similar situation to Eythian(above at 2006-10-28 22:32:25 UTC). It happens always after a few minutes of ide time. Stops upon user imput on keyboard or mouse.

Note CPU ussage increase is NOT accompanied by an increase in block device IO that indexing or database optimization,etc would cause. vmstat output during the transition:
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
 r b swpd free buff cache si so bi bo in cs us sy id wa
 0 0 82636 52508 163252 440940 0 0 0 0 334 960 4 2 94 0
 0 0 82636 52508 163252 440940 0 0 0 0 436 1041 4 3 93 0
 0 0 82636 52508 163252 440940 0 0 0 0 334 820 4 1 95 0
 0 0 82636 52508 163252 440940 0 0 0 0 455 1101 4 2 94 0
 0 0 82636 52384 163252 440940 0 0 0 0 335 844 4 1 95 0
 0 0 82636 52508 163252 440940 0 0 0 0 437 1145 6 2 92 0
 0 0 82636 52384 163252 440940 0 0 0 0 333 808 4 0 96 0
 1 0 82636 52384 163252 440940 0 0 0 0 435 1028 38 8 54 0
 1 0 82636 52384 163252 440940 0 0 4 0 353 719 66 12 22 0
 0 0 82636 52384 163252 440940 0 0 0 0 439 1014 39 10 51 0
 1 0 82636 52508 163252 440940 0 0 0 8 340 820 45 14 42 0
 1 0 82636 52384 163252 440940 0 0 0 0 461 1049 34 8 58 0
 2 0 82636 52384 163252 440940 0 0 0 0 335 775 63 16 21 0
 3 0 82636 52384 163252 440940 0 0 0 0 435 951 62 9 29 0

No lines were appended to these logs during this period :
~/.beagle/Log/current-Beagle
~/.beagle/Log/current-BeagleExceptions
~/.beagle/Log/current-IndexHelper

Following some tips I found by googling I temporarily enabled extended attributes on the partition that gets indexed:
sudo mount /dev/hda2 -oremount,user_xattr
This seems to have stopped the unusual resource consumption by beagled-helper. See:
http://beagle-project.org/Enabling_Extended_Attributes

More correctly, it seems that beagled-helper is no longer run.

[Cable (dacableguy)]

I also have this problem. Works fine when the computer starts, but CPU load goes up to 100% after being idle for a few minutes.

[Oliver Paulus (oliver-webprojekt)]

same problem since edgy.

[Msturm (msturm)]

I can confirm this bug. I've also upgraded from 6.06 to 6.10 and installed beagle afterwards. I've also enabled xattr on the partition on which my /home is located. Beagled seems to run fine when there are not many programs running, but especially when OpenOffice.org or Eclipse (using J2SE 1.4.2, i.e. not from the Ubuntu repositories), beagled and beagled-helper consumes so much resources that my laptop becomes totally unresponsive. Only killing beagled lowers the load of the system,
I've added .jar files to the list of filetypes which should not be indexed, but that doesn't seems to make any difference.

[Ali Sheikh (asheikh)]

I think there are two distinct bugs here. I also see the problem with CPU utilization when my X session goes idle. It seems that beagled is eating up too many cycles. I have not noticed any problems with beagle-helper though. Judging from other people's comments, it seems the X Idle problem is a distinct problem from the indexing problem.

I ran "beagled --fg" under strace. It seems that once the X session becomes idle, for some reason beagled decides to make a horrendous amount of gettimeofday system calls (once every ~35us -- ie. ~29000 system calls per second). This makes my laptop very hot.

[Joe Shaw (joeshaw)]

Hey everyone, I'm the upstream Beagle maintainer. There are a few different issues in this bug:

* Excessive CPU usage in beagled-helper on certain files (.jar, .js, .m being the main ones): This is a bug in the freedesktop.org xdgmime implementation which Beagle uses. It was fixed in the GTK+ copy of this code, which we now consider to be the "canonical" upstream release. We incorporated the fixes into Beagle 0.2.10, which was released on 18 September.

* 100% CPU usage in beagled or beagled-helper when the screensaver is active: This is not a bug. This is by design. The idea is that when you are not using your computer, Beagle can your data index as fast as possible and not worry about the impact on your user experience because, again, you're not using the machine interactively. This does not happen when you are on battery, however, so if you have a laptop you can test this by unplugging it from the wall and seeing if the CPU usage continues. (Give it a couple of seconds to finish indexing the file it's working on, though.)

* There are other instances where certain types of files (often MS PowerPoint documents) can cause 100% CPU utilization. There have been fixes for problems like these in newer releases, the latest of which is 0.2.12. Please pester your packagers to update to the latest code.

Any other instances where beagled or beagled-helper peg the CPU for extended periods of time is a bug. I don't regularly track bugs on Launchpad (nor do I in the Red Hat bugzilla, or any other vendor bug tracker), so filing them upstream in the GNOME Bugzilla is the best way to get the attention of not only me, but the other Beagle developers.

[Sebastian Breier (tomcat42)]

Thanks for the details, Joe. We usually file bugs upstream as well, I guess most people just forgot this time (at least I did, sorry...).

As for the "100% CPU usage in beagled or beagled-helper when the screensaver is active": I have not changed the power settings on my laptop and when I still had beagle installed, the gnome-power-manager had a little checkbox about indexing while on battery power, which was enabled. So everybody who is experiencing this should probably turn that option off; Which will of course not stop the laptop from activating the fan when on AC power, unfortunately.

[Joe Shaw (joeshaw)]

The "indexing while on battery power" checkbox will enable or disable *any* indexing, regardless of the screensaver. The main difference is that if the AC is unplugged, the screensaver has no impact on how "hard" beagle indexes.

[Robin Sheat (eythian)]

w.r.t the 100% CPU when idle, it doesn't seem to be doing anything. At least, an strace doesn't show much, and nothing is logged. It just seems to be looping over this:

sigreturn() = ? (mask now ~[INT QUIT ABRT KILL TERM STOP RTMIN])
futex(0x81f6660, FUTEX_WAKE, 5) = 0
sigreturn() = ? (mask now [])
gettimeofday({1163110344, 500400}, NULL) = 0
gettimeofday({1163110344, 500453}, NULL) = 0
poll([{fd=21, events=POLLIN}], 1, 1932) = -1 EINTR (Interrupted system call)
--- SIGPWR (Power failure) @ 0 (0) ---
futex(0x81f6660, FUTEX_WAKE, 5) = 0
rt_sigsuspend(~[INT QUIT ABRT TERM XCPU RTMIN RT_1]) = ? ERESTARTNOHAND (To be restarted)
--- SIGXCPU (CPU time limit exceeded) @ 0 (0) ---
sigreturn() = ? (mask now ~[INT QUIT ABRT KILL TERM STOP RTMIN])
futex(0x81f6660, FUTEX_WAKE, 5) = 0
...

As for fd 21, from the poll:
$ ls -l /proc/6152/fd/21
lr-x------ 1 robin robin 64 2006-11-10 11:14 /proc/6152/fd/21 -> pipe:[15633]

I don't know how to tell what's at the other end of the pipe however.

[Joe Shaw (joeshaw)]

Make sure you pass -f to strace. Beagle is a heavily threaded program.

[Robin Sheat (eythian)]

Ah, that is better. Now I can see it accessing files. One of the threads seems to be doing this a huge amount though:

[pid 6277] gettimeofday({1163112683, 715845}, NULL) = 0
[pid 6277] gettimeofday({1163112683, 715947}, NULL) = 0
[pid 6277] gettimeofday({1163112683, 716055}, NULL) = 0

Also, shouldn't it stop after a while, once it's indexed everything? After that, it just needs to find things that have changed.

[Paul Sladen (sladen)]

Joe: Thanks for the information.

As Joe says, there's several separate issues present in this report, that need to be separated out into separate individual reports.

(1) We should probably probably _untick_ the following setting by default:

  [ ] Index data while on battery pack.

What is particularly important is have the setting:

  [ ] Perform full index while on battery power.

This comes into use when the user first installs/upgrades Beagle and Beagle uses 100% CPU for an infinity amount of time doing the initial extra. (rm ~/.beagle/ if you want to repeat this!). By this time the user is annoyed enough with Beagle that they stop it; not realising that in the longer term Beagle might be a little saner.

(2) Joe: if Beagle detects a switch to ACPI power (how is it doing this, SIGPWR?), could we have beagle stop *immediately*. Having a feedback to the user is important so that they associate "remove AC power" with "stop of hard-disk activity".

Indexing of the interrupted file can easily be restarted later. If it was a particularly huge file that took 10minutes to index, I'd be wanting that battery power aswell!

(3) Joe: Could we have some level of self throttle (as opposed to just setting Nice) for Beagle. Running the CPU twice as fast uses eg. Four times the amount of electricity, produces more heat and the like. Perhaps Beagle could have a slider for how much CPU/Disk I/O is allowed and we have the default of having that slider quite low.

[Matthew Caron (matt-mattcaron)]

On my box, beagle plays nice until the screensaver starts. However, once the screensaver is stopped (wiggle mouse, enter password), beagle-helper still stays using a lot of CPU. Indeed - that's how I found the issue in the first place. (What's pegging the CPU? Run top. Aha, it's beagled-helper!) Upon further examination, beagled plays nice until the screensaver starts for the first time. After that, it stays running.

I'll have to check and see if it makes a difference that my screensaver just blanks the screen, rather than displaying something... I wonder if that's the issue? (No time to test that now, unfortunately).

[Edward Steel (eddsteel)]

I have the same high-processor usage while idle problem with beagled, (so not the beagled-helper initial index problem). It's also on a laptop and it's annoying.

I don't think this can really be as designed as joeshaw said, because unticking *all* the boxes about indexing services, so that really it shouldn't be doing anything, CPU still shoots up when idle. Only killing beagled allows me to have a quiet-running laptop overnight.

[Joe Shaw (joeshaw)]

Ok, going to try to answer these one at a time. :)

Paul: I don't understand what you mean by the second checkbox. That one doesn't exist, are you suggesting we add it? When you say "full index", do you mean the unthrottled indexing that happens when the screensaver is on? Note that this should *not* happen when you are on battery power. Battery power overrides the screensaver thing.

For (2), at present it monitors /proc/acpi/ac_adapter. In the future it will use HAL to determine this. There's no way at present to stop filtering a file in the middle, and in all but a few corner cases this won't come up. I'm not sure it's worth the effort; if the CPU is spinning for an extended period of time it's a bug that needs to be fixed because it affects all users negatively, not just battery users.

(3) is something that is possible to do, but I don't know that a slider is useful for the user because there isn't any unit of measure there, and users can't objectively make a decision about what setting is right up front. The only way they can see what is going on is to actively measure their CPU usage, and nobody will do that. It may make sense to have a few pre-defined modes that people can set, however.

Matthew: The real question here: Does beagled-helper ever release the CPU? Is it just finishing a fairly long stretch of indexing, or is there a bug which causes it to use 100% CPU forever? That's what we need to track down. Beagle just queries the X server as to whether or not it has a screensaver active, so whether it's blank or actually displaying something shouldn't matter.

Edward: Sounds like a different issue.

In all these cases, attaching your logs from ~/.beagle/Log (or emailing them directly to me, <email address hidden>) would help a lot in determining what the issue is. It would also be very helpful if people could try 0.2.12 (the latest version) and see how many of these problems are still present. Bugs similar to these have been fixed in large part because of the reports from Ubuntu users. (Thank you!)

[Brandon Hale (brandon)]

Beagle 0.2.13 should be available in Feisty shortly (in the queue) and contains many of the fixes Joe has cited.

[Matthew Caron (matt-mattcaron)]

Any chance of getting it into edgy-backports as well?

[Dieter Komendera (dieter-komendera-com)]

Yes, I also would like to see an update for edgy!

[Stephen Chu (standby)]

Yes, it would be nice to get it resolved in edgy. :)

[Markus Kienast (elias1884)]

I would not understand why you would not make this update available for
edgy? Do you prefer to keep a bug in there?

[Sebastien Bacher (seb128)]

elias, new version are not uploaded to stable usually (that's a standard policy) we prefer to backport fixes for specific bugs instead

[Brandon Hale (brandon)]

I subscribed ubuntu-backporters, as that is not really my thing.

[Brandon Hale (brandon)]

Can everyone please report on 0.2.13 in Feisty?
If someone would post an unofficial backport that would be helpful for people still on Edgy.

[Brandon Hale (brandon)]

Ugh I uploaded this a week ago but was waiting manual approval (new binary packages).
It should be in the archive today.

[Kevin Kubasik (kkubasik)]

Latest release in feisty should fix this problem.

[Rodrigo Tassinari (rto)]

Same problem in Edgy, beagled-helper eats 100% CPU indefinetely. It seems to have problems with large compressed files, jars and .exe's...

A backport or update would really be nice, I have disabled beagle for the time being...

[Markus Kienast (elias1884)]

So I have! Beagle is useless the way it is shipped with Edgy. Hope an
update will be available in backports soon. Hope an inofficial backport
will be available even sooner!

[Paul Sladen (sladen)]

Hello again Joe, sorry for the delay in replying.

(1) A full/initial scan (appears) to happen when there is no pre-existing index; such as on first-installation or upgrade. I'm suggesting that this huge gruntwork index /never/ takes place on battery power.

If my memory serves me correct, I have experienced what appears to be a never ending scan after an upgrade running on battery (this could have been an earlier development) and I quickly "kill -9'ed" this process.

(2) Yes, listening to HAL would be better for the battery/AC state querying than polling. What's the failure mode if the '/proc/acpi/ac_adapter' can't be opened. I'm wondering if perhaps some of the reports above are from machines with failed ACPI (or APM...). Would the daemon go into full-stream-ahead mode?

(3) For a slider to limit the extent of CPU|I/O that is occuring, perhaps suitable units could be in kilobytes/megabytes per limit; though I have a feeling that arbitrary units might be better this case. This is akeen to turning down the oven if the water is boiling over. I'd just turn the slider down until the condition (boiling over) is no longer occuring.

For the actual program there are probably *several* units. Load average, AC power, MB/second of block I/O, screensaver. Perhaps others? The daemon doesn't actually have to be using 100% CPU to bring a machine responsiveness down; could be as simple as starving other programs of disk I/O cycles I guess.

I'm as puzzled as you if Beagle is indeed not meant to ever run the CPU at full-whack.

[John Dong (jdong)]

Now that since Edgy we have CFQ and ionice, wouldn't it be a good idea to schedule beagle's indexing IO as idle/low-priority?

[Joe Shaw (joeshaw)]

Paul, answers to your questions. :)

(1) It's worth noting that the initial crawl happens whenever the beagle daemon is started, not just when there is no index. But what I'm saying is that Beagle already never does that crawl on battery power if you have the config setting set. This feature was added in 0.2.8.

(2) If we can't check a few known files like /proc/acpi/ac_adapter, beagle assumes you're on AC power. (Because the altnerative is to assume you're always on battery, which isn't the right thing either.)

(3) The main issue I have with this is that there isn't any real user feedback. Most users aren't monitoring their CPU usage, so I'm not sure "boiling over" analogy works very well here. Short bursts of CPU usage are normal, but if the system feels unresponsive then we need to do some fine tuning to our scheduler. Pushing that decision off to the user feels like the wrong thing to me.

[Joe Shaw (joeshaw)]

John: Beagle already tries to set itself to the idle scheduling class (which always fails, because it requires root), and falls back to setting its priority within the default "best effort" class to the lowest setting. Beagle also uses fadvise to preload and flush the buffer cache of the files it reads. Beagle should be *very* well behaved in terms of IO.

Starting in version 0.2.12 or 0.2.13 -- I don't remember which -- we also set the nice level of the helper process (which does the actual indexing) to +15, so in the cases where it does use the CPU (mostly normal cases, while indexing) it shouldn't affect things like watching movies, playing music, etc.

[fromport (ubuntu-dth)]

Running feisty, beagled-helper starts consuming all my cpu power as well:
ii beagle 0.2.13-0ubuntu1

this (newer) version definately doesn't fix the problem.

[Joe Shaw (joeshaw)]

fromport: Is the CPU usage for extended periods of time? Can you email me a tarball of your ~/.beagle/Log directory, with a rough timeline of the CPU usage? My email address is <email address hidden>.

[Danny (dth)]

On Tue, 2006-12-12 at 03:56 +0000, Joe Shaw wrote:
> fromport: Is the CPU usage for extended periods of time? Can you email
> me a tarball of your ~/.beagle/Log directory, with a rough timeline of
> the CPU usage? My email address is <email address hidden>.

Few things:
I recently replaced the cpu in my laptop with a faster one.
That's why i keep a close look on the temp of my cpu.
When i woke up this morning it was "blowing" like mad and according to
the attached graph i had been doing for a while.

During the day i walked away from my laptop a few times and each time it
was "hot". I switched to virtual console and with top identified beagled
eating all cpu.

My ~/.beagle dir:

dth@unubut:~/.beagle$ du -sh .
581M .
dth@unubut:~/.beagle$ du -sh Log/
551M Log/

Sure you want a dump of this ?
-rw-r--r-- 1 dth dth 1435803 2006-12-06 10:19
2006-12-06-07-57-11-IndexHelper
-rw-r--r-- 1 dth dth 1447436 2006-12-09 17:47
2006-12-09-16-32-22-IndexHelper
-rw-r--r-- 1 dth dth 1472629 2006-12-04 09:16
2006-12-04-07-46-05-IndexHelper
-rw-r--r-- 1 dth dth 2380193 2006-12-11 19:30
2006-12-10-18-12-10-Beagle
-rw-r--r-- 1 dth dth 554416549 2006-12-11 19:31
2006-12-10-22-16-19-IndexHelper

[fromport (ubuntu-dth)]

cpu temp file measured by munin/acpi.
last night it used all cpu from 01:00am till 07:00am

[Joe Shaw (joeshaw)]

Hi,

Danny wrote:
> During the day i walked away from my laptop a few times and each time it
> was "hot". I switched to virtual console and with top identified beagled
> eating all cpu.

Ok, good to know.

> Sure you want a dump of this ?
> -rw-r--r-- 1 dth dth 1435803 2006-12-06 10:19
> 2006-12-06-07-57-11-IndexHelper
> -rw-r--r-- 1 dth dth 1447436 2006-12-09 17:47
> 2006-12-09-16-32-22-IndexHelper
> -rw-r--r-- 1 dth dth 1472629 2006-12-04 09:16
> 2006-12-04-07-46-05-IndexHelper
> -rw-r--r-- 1 dth dth 2380193 2006-12-11 19:30
> 2006-12-10-18-12-10-Beagle
> -rw-r--r-- 1 dth dth 554416549 2006-12-11 19:31
> 2006-12-10-22-16-19-IndexHelper

Yeah, tar it up for me, please. The fact that the files are that big
means there is definitely a bug in there. I think a file that big might
be rejected via email, can you put it up on the web somewhere?

Joe

[drayen (alexmcfadyen)]

I found that the advice below (posted by Eythian) calmed it down instantly, i also added .exe to the exclusion list just to be on the safe side.

"What I've found helps as a workaround is to find the problem filetypes (e.g. .jar files), and add them to the filetype exclusion list (open beagle -> search menu -> preferences -> indexing tab -> privacy section)"

[Joe Shaw (joeshaw)]

Just FYI, the jar bug is this one:

http://bugzilla.gnome.org/show_bug.cgi?id=348139

And it was fixed in beagle 0.2.10. 0.2.14 is the currently released version.

[Dieter Komendera (dieter-komendera-com)]

I would really like to see a backport from the current version to edgy. This bug is very annoying.

[Dieter Komendera (dieter-komendera-com)]

It looks like 0.2.14 is in the edgy backports now!

But the build failed:
https://launchpad.net/ubuntu/+source/beagle/0.2.14-0ubuntu3~edgy1

Can anyone have a look at it?
Seems to be a dependency problem.

[Joe Shaw (joeshaw)]

Nice, but 0.2.15.1 is out now. ;)

[Dieter Komendera (dieter-komendera-com)]

Yeah, the release notes from 0.2.15 sound good!
Some parts:

* Many memory usage improvements, especially with IMAP accounts in the
  Evolution mail backend and when doing queries.

* A rewritten and massively improved RTF filter.

* The MS Word filter has been moved into an external process,
  improving service reliability tremendously.

* Dozens of improvements to the filter code(...)

Full releasenotes here: http://svn.gnome.org/viewcvs/*checkout*/beagle/trunk/beagle/NEWS

**want have!!** ;-)

[Joe (refactor77-misc)]

The last time I wasted this many CPU cycles on an idle, I was searching for intelligent life with SETI@home. It seems everyone with Edgy is affected by this virus, so how do we get it fixed? Is uninstall of Beagle the only solution?

[Sebastian Breier (tomcat42)]

No need to be so dramatic... a new version is in the backports, you can try that one. I know this bug makes beagle just unusable in edgy, but if nobody's backporting the fix to the beagle version in edgy, it's not gonna be fixed there.

[Joe Shaw (joeshaw)]

There are packages for 0.2.16 available now from here:

http://beagle-project.org/files/ubuntu/edgy/

They're not "official" Ubuntu packages, so caveat emptor, but hopefully they'll be useful to many of you.

[Dieter Komendera (dieter-komendera-com)]

Thanks for the link Joe, just installed it and it seems they work fine.

[Max Randor (max-randor)]

I followed that link and installed, it fixed the problem as far as I can tell, on coming out of screensaver there was just a little bit of CPU usage in the past but not too much and not for extended periods, (though I had excluded .exe and .jar files before I read to the end of the list :-) )

[Max Randor (max-randor)]

No I was wrong it is still broken, ubuntu edgy, installed from the previously given link, came back after having the screen locked for a while the cpu was at 100% and it was beagle-helper that was doing it.

[Joe Shaw (joeshaw)]

Max: For an extended period of time after the screensaver closed? I would expect a few seconds, but not a few minutes.

If that's the case you can send SIGUSR2 to the beagled-helper process and it will log to the ~/.beagle/Log/current-IndexHelper file what file it was processing. I'd like to see the output; if it's a problematic file if possible.

Also, make sure you get the 0.2.16.3 packages linked above and not the 0.2.16 ones, which have a particularly nasty looping bug with certain SVG files.

[murrayf (murrayf07)]

This problem still remains in Ubuntu 7.04 Feisty, beagled-helper process consumes a lot of memory and loads cpu at very high levels using beagle 0.2.16.3-0ubuntu4 package...

[Joe Shaw (joeshaw)]

murrayf: Can you send SIGUSR2 to the helper process and paste in the output from the logs? It should say which file it's spinning on.

[Brandon Wright (bearoso-deactivatedaccount)]

I've got a file that produces this bug consistently for me.

The file is at
http://svn.gnome.org/viewcvs/gnome-main-menu/trunk/main-menu/etc/empty.ods

Relevant line from sending the process SIGUSR2:
20070501 01:30:46.3526 24787 IndexH WARN: Filtering status (17h50m4s ago): determining filter for file:///home/bearoso/gnome-main-menu/main-menu/etc/empty.ods

[Joe Shaw (joeshaw)]

Brandon: Yeah, I've seen this bug with empty OOo files. It's already fixed in SVN (r3611), and I'm attaching the patch for it here.