locate spawns very CPU hungry find process

[Alexander Mescheryakov]

/etc/cron.daily/locate job spawns find process which very aggressively uses CPU. Full scan of filesystem of HDD on my laptop takes about 40 minutes and almost all of that time one core is 100% occupied by find.

Here is the command that runs

/usr/bin/find / -ignore_readdir_race \( -fstype NFS -o -fstype nfs -o -fstype nfs4 -o -fstype afs -o -fstype binfmt_misc -o -fstype proc -o -fstype smbfs -o -fstype autofs -o -fstype iso9660 -o -fstype ncpfs -o -fstype coda -o -fstype devpts -o -fstype ftpfs -o -fstype devfs -o -fstype mfs -o -fstype shfs -o -fstype sysfs -o -fstype cifs -o -fstype lustre_lite -o -fstype tmpfs -o -fstype usbfs -o -fstype udf -o -fstype ocfs2 -o -type d -regex '\(^/tmp$\)\|\(^/usr/tmp$\)\|\(^/var/tmp$\)\|\(^/afs$\)\|\(^/amd$\)\|\(^/alex$\)\|\(^/var/spool$\)\|\(^/sfs$\)\|\(^/media$\)\|\(^/var/lib/schroot/mount$\)' \) -prune -o -print0

After brief investigation I found that CPU strain is produced by filesystem type checks. For example, here is original command benchmark (with additionally excluded /home/snapshots directory which contains dozens of BTRFS snapshots with too many files):

time sudo /usr/bin/find / -ignore_readdir_race \( -fstype NFS -o -fstype nfs -o -fstype nfs4 -o -fstype afs -o -fstype binfmt_misc -o -fstype proc -o -fstype smbfs -o -fstype autofs -o -fstype iso9660 -o -fstype ncpfs -o -fstype coda -o -fstype devpts -o -fstype ftpfs -o -fstype devfs -o -fstype mfs -o -fstype shfs -o -fstype sysfs -o -fstype cifs -o -fstype lustre_lite -o -fstype tmpfs -o -fstype usbfs -o -fstype udf -o -fstype ocfs2 -o -type d -regex '\(^/tmp$\)\|\(^/usr/tmp$\)\|\(^/var/tmp$\)\|\(^/afs$\)\|\(^/amd$\)\|\(^/alex$\)\|\(^/var/spool$\)\|\(^/sfs$\)\|\(^/media$\)\|\(^/var/lib/schroot/mount$\)\|\(^/home/snapshots$\)' \) -prune -o -print0 | pv > /dev/null
57,1MB 0:13:17 [73,3kB/s] [ <=> ]

real 13m17.647s
user 6m2.983s
sys 6m41.837s

And here is benchmark of command with severely shortened filesystem check list:

time sudo /usr/bin/find / -ignore_readdir_race \( -fstype afs -o -fstype binfmt_misc -o -fstype proc -o -fstype autofs -o -fstype devpts -o -fstype devfs -o -fstype sysfs -o -fstype tmpfs -o -fstype usbfs -o -type d -regex '\(^/tmp$\)\|\(^/usr/tmp$\)\|\(^/var/tmp$\)\|\(^/afs$\)\|\(^/amd$\)\|\(^/alex$\)\|\(^/var/spool$\)\|\(^/sfs$\)\|\(^/media$\)\|\(^/var/lib/schroot/mount$\)\|\(^/home/snapshots$\)' \) -prune -o -print0 | pv > /dev/null
57,1MB 0:05:04 [ 192kB/s] [ <=> ]

real 5m4.092s
user 2m24.112s
sys 2m39.951s

Each excluded filesystem greatly speeds up the process.

This is really annoying, as somehow daily cron jobs sometimes start even if my laptop is running on battery o_O (AFAIK anacron avoids running periodic jobs ).

I checked the same find command on onther PC with arch linux, and it used less than 10 seconds of CPU time with comparable number of files, so it is possible, that something is wrong with ubuntu find version or somehow related to ubuntu VFS specifics.

Also locate may check mounted filesystem and do not include unused filesystems when compiling command for find.

ProblemType: Bug
DistroRelease: Ubuntu 13.10
Package: locate 4.4.2-6ubuntu1
ProcVersionSignature: Ubuntu 3.11.0-15.23-generic 3.11.10
Uname: Linux 3.11.0-15-generic x86_64
ApportVersion: 2.12.5-0ubuntu2.2
Architecture: amd64
Date: Sun Jan 12 17:27:24 2014
Dependencies:
 findutils 4.4.2-6ubuntu1
 gcc-4.8-base 4.8.1-10ubuntu9
 libc6 2.17-93ubuntu4
 libgcc1 1:4.8.1-10ubuntu9
 multiarch-support 2.17-93ubuntu4
InstallationDate: Installed on 2012-07-01 (559 days ago)
InstallationMedia: Xubuntu 12.04 LTS "Precise Pangolin" - Release amd64 (20120425)
MarkForUpload: True
SourcePackage: findutils
UpgradeStatus: Upgraded to saucy on 2014-01-02 (10 days ago)

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Menachem Shapiro (menachem)]

Related:
https://bugs.launchpad.net/ubuntu/+source/findutils/+bug/332790
https://bugs.launchpad.net/ubuntu/+source/mlocate/+bug/1190696