updatedb.conf: CephFS should be included in PRUNEFS

[Wido den Hollander]

CephFS filesystems can become huge. It's networking filesystem for a client and clients should not index data on CephFS filesystems.

I suggest that "ceph" and "fuse.ceph" is added to "PRUNEFS" in /etc/updatedb.conf to prevent all the clients to start indexing this network filesystem.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Livio Soares (liviobs)]

Just wanted to cast my vote for adding "ceph" to the list of pruned file-system types. We have a fairly large Ceph file-system that is mounted throughout a cluster. Having the entire cluster crawl the file-system in synchronized form (due to cron) is really disruptive. Not only does it make the entire cluster halt due to network storm (causing all sorts of problems on the ceph mounts), it also typically causes problems on the Ceph servers (page allocation errors in kernel/kworker threads, etc).

Ceph should clearly be an opt-in for updatedb, definitely not the default behavior.

[Brian Murray (brian-murray)]

Thank you for providing a patch for this bug report. This particular package is synchronized with Debian so it would be quite helpful if you were to forward the patch to the Debian bug tracker. You can learn more about how to use the Debian bug tracker at https://wiki.ubuntu.com/Debian/Bugs. After you've forwarded the bug report and patch please add a bug watch, following the procedure at http://wiki.ubuntu.com/Bugs/Watches, so we can track the progress of the upstream bug report.

[Ken (ken-redhat-bugs)]

Description of problem:
Ceph is a software-defined storage project, and it stores its data in /var/lib/ceph as a big set of files. The problem is that updatedb indexes /var/lib/ceph along with everything else on the system. This /var/lib/ceph location is typically *very* large, and Ceph users are not supposed to go poking around at the files in there.

Version-Release number of selected component (if applicable):
mlocate-0.26-9.fc22

How reproducible:
always

Steps to Reproduce:
1. mount some large OSD disks under /var/lib/ceph
2. updatedb runs and indexes all of the OSDs' files

Actual results:
IO slows to a crawl

Expected results:
updatedb excludes Ceph data (add /var/lib/ceph to PRUNEPATHS)

Additional info:
Ceph also has a filesystem, so it would be nice to exclude the "ceph" and "fuse.ceph" filesystem types as well (via PRUNEFS).

For more background information, see a thread on the ceph-users mailing list: http://lists.ceph.com/pipermail/ceph-users-ceph.com/2014-February/037446.html

More discussion on ceph-devel: http://www.spinics.net/lists/ceph-devel/msg18098.html

[Ken Dreyer (Red Hat) (kdreyer-redhat)]

Wido, did you end up filing a bug with Debian on this? If not, I can do so.

[Wido den Hollander (wido)]

Hi Kevin,

No, I didn't. If you could do that it would be great.

Wido

> Op 21 mei 2015 om 01:20 heeft Ken Dreyer \(Red Hat\) <email address hidden> het volgende geschreven:
>
> Wido, did you end up filing a bug with Debian on this? If not, I can do
> so.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1281074
>
> Title:
> updatedb.conf: CephFS should be included in PRUNEFS
>
> Status in mlocate package in Ubuntu:
> Confirmed
> Status in Debian:
> New
>
> Bug description:
> CephFS filesystems can become huge. It's networking filesystem for a
> client and clients should not index data on CephFS filesystems.
>
> I suggest that "ceph" and "fuse.ceph" is added to "PRUNEFS" in
> /etc/updatedb.conf to prevent all the clients to start indexing this
> network filesystem.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/ubuntu/+source/mlocate/+bug/1281074/+subscriptions

[Fedora (fedora-redhat-bugs)]

mlocate-0.26-8.fc21 has been submitted as an update for Fedora 21.
https://admin.fedoraproject.org/updates/mlocate-0.26-8.fc21

[Fedora (fedora-redhat-bugs)]

Package mlocate-0.26-8.fc21:
* should fix your issue,
* was pushed to the Fedora 21 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing mlocate-0.26-8.fc21'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2015-8989/mlocate-0.26-8.fc21
then log in and leave karma (feedback).

[Fedora (fedora-redhat-bugs)]

mlocate-0.26-10.fc22 has been submitted as an update for Fedora 22.
https://admin.fedoraproject.org/updates/mlocate-0.26-10.fc22

[Ken (ken-redhat-bugs)]

Thanks for shipping this change!

Would you mind excluding the "ceph" and "fuse.ceph" filesystem types as well, adding them to PRUNEFS?

[Fedora (fedora-redhat-bugs)]

mlocate-0.26-10.fc22 has been pushed to the Fedora 22 stable repository. If problems still persist, please make note of it in this bug report.

[Fedora (fedora-redhat-bugs)]

mlocate-0.26-8.fc21 has been pushed to the Fedora 21 stable repository. If problems still persist, please make note of it in this bug report.

[Brian Murray (brian-murray)]

This is fixed in releases after and including Bionic.

 mlocate | 0.26-2ubuntu3.1 | bionic | source, amd64, arm64, armhf, i386, ppc64el, s390x
 mlocate | 0.26-2ubuntu3.1 | cosmic | source, amd64, arm64, armhf, i386, ppc64el, s390x
 mlocate | 0.26-2ubuntu3.1 | disco | source, amd64, arm64, armhf, i386, ppc64el, s390x