hud-service frequently tries to accesses non existing files causing high disk/network activity

[Manfred Thole]

/usr/lib/indicator-appmenu/hud-service is frequently calling access() for the following three files:

access("<home>/.cache/indicator-appmenu/hud-usage-log.sqlite-journal", F_OK) = -1 ENOENT (No such file or directory)
access("<home>/.cache/indicator-appmenu/hud-usage-log.sqlite-wal", F_OK) = -1 ENOENT (No such file or directory)
access("/usr/share/indicator-appmenu/hud/app-info/multiload.hud-app-info", F_OK) = -1 ENOENT (No such file or directory)

which apparently do not exist (excerpt from strace log, <home> is users home directory).

For NFS mounted homes this causes a constant network traffic of ~300kB/s down and ~50kB/s up in idle state.
On slower machines this causes a more than noticable CPU load (with and without NFS mounted homes).

ProblemType: Bug
DistroRelease: Ubuntu 12.04
Package: indicator-appmenu 0.3.97-0ubuntu1
ProcVersionSignature: Ubuntu 3.2.0-24.37-generic 3.2.14
Uname: Linux 3.2.0-24-generic x86_64
NonfreeKernelModules: fglrx
ApportVersion: 2.0.1-0ubuntu7
Architecture: amd64
Date: Sun May 13 15:36:15 2012
ExecutablePath: /usr/lib/indicator-appmenu/hud-service
InstallationMedia: Ubuntu 11.10 "Oneiric Ocelot" - Release amd64 (20111012)
SourcePackage: indicator-appmenu
UpgradeStatus: Upgraded to precise on 2012-04-28 (15 days ago)

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[KVV (kouzminv-yahoo)]

If indicator-multiload is running, each indicator update cycle causes this hud-service activity. With the default 500ms indicator refresh time, my workstation is constantly sending ~300kB/s and receiving ~670kB/s from the NFS server hosting home directory, as well as ~9% CPU usage (mobile sandybridge CPU).

[Sebastien Bacher (seb128)]

Thank you for your bug report, it's not likely that those stats create the performance issues though, it's rather an indicator-multiload issue, sending status update over dbus every 500ms

[Bilal Akhtar (bilalakhtar)]

I think hud-service should (cache?) store stuff in memory in such situations, if calls are being made repeatedly, for example. But yeah, it's a low-priority task on this end, the rest of the load lies on indicator-multiload.

[KVV (kouzminv-yahoo)]

I see two problems:

1) indicator-multiload is causing (via dbus?) hud-service to do "something" (update/reload cache?) each time indicators get refreshed. indicator-multiload does not do any "strange" network activity by itself, strace shows quite sane behavior (access to /proc and /sys and such).

2) each hud-service "update" initiated by indicator-multiload is casuing some 0.5MB network exchange (if home directory is on NFS), about half with nfs port and half with nfs lockd. I tcpdumped both sessions. Lock protocol is all binary, looks like repeating lock requests. On nfs port, it does the same over and over again. I counted this snippet called 88 times during one hud-servide "update":

q.........O.W.7|N.O... ..yO... ..y............SQLite format 3......@ .........................................................................-.!
.....I....I.........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................W.../..windexapplication_indexusage.CREATE INDEX application_index on usage (application)\........tableusageusage.CREATE TABLE usage (application text, entry text, timestamp datetime)

As a workaround, I can:
1) reduce frequency of indicator refresh
2) kill hud-service (and so this network activity stops)

[KVV (kouzminv-yahoo)]

Technically this only affects the user if indicator-multiload is installed (non-default config but it's in the standard repositories in 12.04) AND home directory is on NFS (uncommon). If home directory is on local drive, this will probably have little impact due to caching.

But indicator refresh should cuase zero traffic with the NFS server, otherwise a network of tens or hurdreds NFS workstations will cause quite a troble to the server and the network.

[Ted Gould (ted)]

An indicator refresh would cause a check of the usage database if there was a search active, as it could possibly change the ordering of the search. Though, it sounds like that's not the case here.

Not sure of the exact cause, but a work around would be to tell the HUD to not save usage data so it'll only use an in-memory database:

  $ gsettings set com.canonical.indicator.appmenu.hud store-usage-data false

Which would mean that the locking files wouldn't be checked as the database is read.

[KVV (kouzminv-yahoo)]

> $ gsettings set com.canonical.indicator.appmenu.hud store-usage-data false

This worked, traffic dropped to zero!

[gcc (chris+ubuntu-qwirx)]

HUD is repeatedly opening and closing the database. That is very inefficient on any system, but especially over NFS. (I don't see 2 DBus requests per second as inappropriate, unlike comment #4).

Can't it just keep the database open? Nobody else should be using it at the same time.

[Jean Christophe André (progfou)]

I confirm on my side that the gsetting suggestion from Ted Gould (comment #8) is working for a user with his homedir over NFS. I have now put it in my default Ubuntu deployment strategy.