randomly, update-apt-xapi will take 100% cpu for an extended period of time

[Bryan Moore]

Description: Ubuntu oneiric (development branch)
Release: 11.10

apt-xapian-index:
  Installed: 0.44ubuntu1
  Candidate: 0.44ubuntu1
  Version table:
 *** 0.44ubuntu1 0
        500 http://mirror.anl.gov/ubuntu/ oneiric/main i386 Packages
        100 /var/lib/dpkg/status

ProblemType: Bug
DistroRelease: Ubuntu 11.10
Package: apt-xapian-index 0.44ubuntu1
ProcVersionSignature: Ubuntu 3.0.0-8.11-generic 3.0.1
Uname: Linux 3.0.0-8-generic i686
Architecture: i386
Date: Sat Aug 20 21:10:16 2011
InstallationMedia: Ubuntu 11.10 "Oneiric Ocelot" - Alpha i386 (20110705.1)
PackageArchitecture: all
ProcEnviron:
 PATH=(custom, no user)
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: apt-xapian-index
UpgradeStatus: No upgrade log present (probably fresh install)

[Sam_ (and-sam)]

Similar Bug #363695

[primefalcon (primefalcon)]

one quick fix would be to run

sudo rm /etc/cron.weekly/apt-xapian-index

[Don Cady (doncady)]

Is this a dup of Bug #363695?

[mikewhatever (mikewhatever)]

Another workaround is to remove 'apt-xapian-index'.

sudo apt-get purge apt-xapian-index

This is an unbelievably persistent bug. It has tons of duplicates and has been marked as fixed as far back as 2009. ...and yet, here we are.

[Tal Liron (emblem-parade)]

Someone posted what appears to be a fix in the forums:

http://ubuntuforums.org/showpost.php?p=10453047&postcount=15

Can we verify this and patch?

Three years and this bug is still plaguing us...

[Madriddo (enero-vp)]

This problem still exist - Kubuntu Oneiric 11.10

[Tal Liron (emblem-parade)]

I can confirm that the fix mentioned above helps.

It seems that the Xapian updater is running for a much shorter time due to using '--update' instead of a full recreation of the index. However, since I don't really understand Xapian, I can't be sure that this doesn't create other problems.

However, my netbook still stutters while watching a video during this shorter period of time in which the Xapian updater is running. I think that all the "nice" use in the world won't help this sore go away. I hope Ubuntu can find an alternative solution.

Until then, can someone with authority look into confirming this fix and patching it?

Credit for the fix goes to Ubunu Forums user 'selanit': http://ubuntuforums.org/member.php?u=361185

[Numan Demirdöğen (if.gnu.linux)]

ApportVersion: 1.23-0ubuntu4
Architecture: i386
CheckboxSubmission: 670355b19452e48dc13bba8540f6da92
CheckboxSystem: 331fbefb4b1f6727f4a8261fee7507c9
DistroRelease: Ubuntu 11.10
InstallationMedia: Ubuntu 11.10 "Oneiric Ocelot" - Beta i386 (20110921.2)
Package: apt-xapian-index 0.44ubuntu4
PackageArchitecture: all
ProcVersionSignature: Ubuntu 3.0.0-13.22-generic 3.0.6
Tags: oneiric running-unity
Uname: Linux 3.0.0-13-generic i686
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: adm admin audio cdrom dialout lpadmin plugdev sambashare

[Numan Demirdöğen (if.gnu.linux)]

apport information

[Numan Demirdöğen (if.gnu.linux)]

apport information

[Tal Liron (emblem-parade)]

More information about the "fix" -- it helps, but does not solve the problem.

I still get jittery video about once a day when using the netbook. I'm sure a lot users on weaker machines are having this problem, but don't know enough to diagnose what's causing it, and thus just think "Ubuntu is very slow".

This bug, including its older dupes, is *years* old. Can someone who knows anything about Xapian fix this? Can we at least assign it to someone who can help?

[niplas (niplas)]

I can verify all of the above

This major annoyance (seems a bit harsh to call it a bug though :) is especially frustrating in an LTSP environment.

Firstly because the hardware even for fat clients usually is quite modest in terms of performance, and second (which is worst) the process is absolutely unnecessary when running the clients. All updates to the clientss is done manually via chroot and the image update or build process. In chroot and update I point to the index on the server itself which most people do.

As far as I can tell this cronjob is started as a separate process for all clients booted, fat as thin, and it seems that it is always "due" for updating even if you just do a 45 second reboot of your client (guess there is nothing there to tell the client it has been done)

This means you either put the server to a grinding halt when starting 20+ clients in the morning (naturally depending on your server specs), or your fat clients spend between 2 and ten (!) minutes at often near 100% CPU utilization updating an index that is probably sent to dev null or purged when the client is switched of (don't know, haven't investigated what happens with it)

The "Fix" helps (however i've choosen to purge apt-xapian-index completely in the client chroot, cant see a reason not to)

Not sure if this should go upstream..
Am a newbie when it comes to bug reporting so; copy and paste if you think the folks @ ltsp or someone else should take a look at this aswell.

[niplas (niplas)]

Whoops
No I cant verify all of the above :D but I sometimes do slip on ctrl-V

Also: In an LTSP FAT-client environment (generic 3.0.0.14/i386/allbuntu/verified) this should be considered a bug.
On a desktop or server default install it still is a default behaviour not acting nice at all (despite niceness level 10 :)

See Bug #655831 for even more reports (not marked as duplicate)

[Chris (fabricator4)]

Confirmed for 12.04 beta 1. The nice value is set high (20) but he priority is also set high (20). While things are definitely much better than they once were, it still impacts heavily on my Celeron test machine.

It's a non-critical cron job and it would be nice if wasn't quite so intrusive.

Chris

[vervelover (alessiopangos)]

confirmed on 12.04 with all updates. this bug makes two of the machines I have completely unusable for 10 minutes (I mean I can't do ANYTHING). It's a major annoyance, any non-geek user experiencing this would give up on ubuntu.

[Thomas Hood (jdthood)]

I just got this too. The update-apt-xapian-index command slowed down at around 30%, finally reached 70% or so before reporting

    Rebuilding Xapian index: done.
    Killed

While running, the top command showed:

    %CPU %MEM TIME+ COMMAND
    29.5 75.1 3:01.79 update-apt-xapi
    14.4 0.0 1:44.51 btrfs-endio-3
    13.8 0.0 0:17.87 btrfs-endio-2
    12.8 0.0 0:17.92 btrfs-endio-3

So it possibly failed in this case because btrfs-endio also happened to be running.

[Ville Ranki (ville-ranki)]

This just happened for me on quantal.

[Sarah Bennert (sarahbx)]

Experiencing this on Ubuntu & Xubuntu 12.04.2.

My workaround after reading posts here, there, and elsewhere:

sudo apt-get install cpulimit;

edit /etc/cron.weekly/apt-xapian-index
to utilize cpulimit

file attached

Keeps the process from taking over the cpu, works great.

[Vasily (applebear13)]

Still reproducible on fresh installed Ubuntu Utopic ((((

[Numan Demirdöğen (if.gnu.linux)]

It is fixed for me.

[Guy (guy-b)]

Not fixed in Xubuntu 14.04.2 LTS even after the changes from Eiríkr & selanit in given link @ #8 !

[Guy (guy-b)]

Seems even not to be fixed with the solution @ #19 but maybe after a reboot ? Let see...

[Jarno Suni (jarnos)]

Fixed in ubuntu 15.04?