| 2021-08-24 10:17:57 |
Jonathan Wiltshire |
bug |
|
|
added bug |
| 2021-08-24 10:19:16 |
Chris Boot |
bug |
|
|
added subscriber Chris Boot |
| 2021-08-24 10:19:46 |
Launchpad Janitor |
monitoring-plugins (Ubuntu): status |
New |
Confirmed |
|
| 2021-08-24 10:20:02 |
Jonathan Wiltshire |
description |
$ lsb_release -d
Description: Ubuntu 20.04.3 LTS
$ apt-cache policy monitoring-plugins | head -n2
monitoring-plugins:
Installed: 2.2-6ubuntu1.1
check_disk returns UNKNOWN for tmpfs filesystems, even when directed to explicitly to check them. This is a regression introduced by the fix for LP: #1827159.
Steps to reproduce:
$ df -h /tmp
Filesystem Size Used Avail Use% Mounted on
tmpfs 16G 64K 16G 1% /tmp
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -p /tmp
DISK UNKNOWN - free space:|
Running the check as part of a group, even with just the one member, is a workaround:
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -g tmp -p /tmp
DISK OK - free space: tmp 16058 MB (99% inode=99%);| tmp=0MB;14452;15255;0;16058 |
$ lsb_release -d
Description: Ubuntu 20.04.3 LTS
$ apt-cache policy monitoring-plugins | head -n2
monitoring-plugins:
Installed: 2.2-6ubuntu1.1
check_disk returns UNKNOWN for tmpfs filesystems, even when directed explicitly to check them. This is a regression introduced by the fix for LP: #1827159.
Steps to reproduce:
$ df -h /tmp
Filesystem Size Used Avail Use% Mounted on
tmpfs 16G 64K 16G 1% /tmp
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -p /tmp
DISK UNKNOWN - free space:|
Running the check as part of a group, even with just the one member, is a workaround:
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -g tmp -p /tmp
DISK OK - free space: tmp 16058 MB (99% inode=99%);| tmp=0MB;14452;15255;0;16058 |
|
| 2021-08-25 19:16:59 |
Athos Ribeiro |
monitoring-plugins (Ubuntu): status |
Confirmed |
Triaged |
|
| 2021-08-26 11:31:53 |
Athos Ribeiro |
bug |
|
|
added subscriber Ubuntu Server |
| 2021-10-20 15:42:22 |
Bryce Harrington |
tags |
|
server-todo |
|
| 2021-10-20 15:42:27 |
Bryce Harrington |
monitoring-plugins (Ubuntu): importance |
Undecided |
Medium |
|
| 2021-11-25 07:36:23 |
Christian Ehrhardt |
monitoring-plugins (Ubuntu): assignee |
|
Miriam España Acebal (mirespace) |
|
| 2021-12-01 06:54:13 |
Miriam España Acebal |
monitoring-plugins (Ubuntu): status |
Triaged |
In Progress |
|
| 2021-12-01 09:04:48 |
Miriam España Acebal |
bug watch added |
|
https://github.com/monitoring-plugins/monitoring-plugins/issues/1608 |
|
| 2022-01-25 07:58:10 |
Utkarsh Gupta |
bug |
|
|
added subscriber Miriam España Acebal |
| 2022-01-28 11:47:56 |
Miriam España Acebal |
description |
$ lsb_release -d
Description: Ubuntu 20.04.3 LTS
$ apt-cache policy monitoring-plugins | head -n2
monitoring-plugins:
Installed: 2.2-6ubuntu1.1
check_disk returns UNKNOWN for tmpfs filesystems, even when directed explicitly to check them. This is a regression introduced by the fix for LP: #1827159.
Steps to reproduce:
$ df -h /tmp
Filesystem Size Used Avail Use% Mounted on
tmpfs 16G 64K 16G 1% /tmp
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -p /tmp
DISK UNKNOWN - free space:|
Running the check as part of a group, even with just the one member, is a workaround:
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -g tmp -p /tmp
DISK OK - free space: tmp 16058 MB (99% inode=99%);| tmp=0MB;14452;15255;0;16058 |
[Impact]
False positive reports are generated in monitoring tools when artificial file systems are mounted, as they show 100% disk utilisation, and therefore add an unnecessary (but fatal sounding) "CRITICAL DISK" noise. But, if we exclude them by default as we did in the first place in bug 1827159, we limit the use of command options that are commonly used in scripts and that, because of this exclusion, cannot operate on this kind of filesystems (e.g. the path option or the include filesystem option -as excluding has preference over including).
[Test Plan]
0.Preparing the test environment:
#Preparing the container
lxc launch images:ubuntu/focal checkdisks-focal
lxc shell checkdisks-focal
apt update -y
apt upgrade -y
#Installing necessary tools
apt install monitoring-plugins
apt install snapd
#Mount some synthetic systems and make use of snap space
mkdir /reproduce-tmpfs
mkdir /reproduce-squashfs
mkdir /reproduce-tracefs
mount -t tmpfs -o rw,size=2G tmpfs /reproduce-tmpfs
mount -t tmpfs -o rw,size=2G squashfs /reproduce-squashfs
mount -t tmpfs -o rw,size=2G tracefs /reproduce-tracefs
snap install gnome-calculator
1. Bad cases:
# "CRITICAL DISK" noise
$ /usr/lib/nagios/plugins/check_disk -w 10 -c 10
DISK CRITICAL - free space: /dev 448 MB (100% inode=99%); /run 96 MB (98% inode=99%); / 884 MB (18% inode=80%); /dev/shm 485 MB (100% inode=99%); /run/lock 5 MB (100% inode=99%); /snap/lxd/21835 0 MB (0% inode=0%); /snap/core20/1270 0 MB (0% inode=0%); /snap/snapd/14295 0 MB (0% inode=0%); /boot/efi 99 MB (95% inode=-); /run/snapd/ns 96 MB (98% inode=99%); /snap/snapd/14549 0 MB (0% inode=0%); /run/user/1000 97 MB (99% inode=99%); /snap/bare/5 0 MB (0% inode=0%); /snap/multipass-sshfs/145 0 MB (0% inode=0%); /home/ubuntu/monitoring-plugins 1024000 MB (100% inode=100%); /home/ubuntu/reproduce 2048 MB (100% inode=99%); /home/ubuntu/reproduce-squash 2048 MB (100% inode=99%); /snap/core20/1328 0 MB (0% inode=0%); /home/ubuntu/reproduce-tracefs 2048 MB (100% inode=99%);| /dev=0MB;438;438;0;448 /run=0MB;87;87;0;97 /=3886MB;4777;4777;0;4787 /dev/shm=0MB;475;475;0;485 /run/lock=0MB;-5;-5;0;5 /snap/lxd/21835=67MB;57;57;0;67 /snap/core20/1270=62MB;52;52;0;62 /snap/snapd/14295=43MB;33;33;0;43 /boot/efi=5MB;94;94;0;104 /run/snapd/ns=0MB;87;87;0;97 /snap/snapd/14549=43MB;33;33;0;43 /run/user/1000=0MB;87;87;0;97 /snap/bare/5=0MB;-10;-10;0;0 /snap/multipass-sshfs/145=1MB;-9;-9;0;1 /home/ubuntu/monitoring-plugins=0MB;1023990;1023990;0;1024000 /home/ubuntu/reproduce=0MB;2038;2038;0;2048 /home/ubuntu/reproduce-squash=0MB;2038;2038;0;2048 /snap/core20/1328=62MB;52;52;0;62 /home/ubuntu/reproduce-tracefs=0MB;2038;2038;0;2048
$ /usr/lib/nagios/plugins/check_disk -w 10 -c 10 -e -X squashfs
DISK CRITICAL - free space: /run/lock 5 MB (100% inode=99%);| /dev=0MB;438;438;0;448 /run=0MB;87;87;0;97 /=3886MB;4777;4777;0;4787 /dev/shm=0MB;475;475;0;485 /run/lock=0MB;-5;-5;0;5 /boot/efi=5MB;94;94;0;104 /run/snapd/ns=0MB;87;87;0;97 /run/user/1000=0MB;87;87;0;97 /home/ubuntu/monitoring-plugins=0MB;1023990;1023990;0;1024000 /home/ubuntu/reproduce=0MB;2038;2038;0;2048 /home/ubuntu/reproduce-squash=0MB;2038;2038;0;2048 /home/ubuntu/reproduce-tracefs=0MB;2038;2038;0;2048
$ /usr/lib/nagios/plugins/check_disk -w 10 -c 10 -e -X tracefs
DISK CRITICAL - free space: /run/lock 5 MB (100% inode=99%); /snap/lxd/21835 0 MB (0% inode=0%); /snap/core20/1270 0 MB (0% inode=0%); /snap/snapd/14295 0 MB (0% inode=0%); /snap/snapd/14549 0 MB (0% inode=0%); /snap/bare/5 0 MB (0% inode=0%); /snap/multipass-sshfs/145 0 MB (0% inode=0%); /snap/core20/1328 0 MB (0% inode=0%);| /dev=0MB;438;438;0;448 /run=0MB;87;87;0;97 /=3886MB;4777;4777;0;4787 /dev/shm=0MB;475;475;0;485 /run/lock=0MB;-5;-5;0;5 /snap/lxd/21835=67MB;57;57;0;67 /snap/core20/1270=62MB;52;52;0;62 /snap/snapd/14295=43MB;33;33;0;43 /boot/efi=5MB;94;94;0;104 /run/snapd/ns=0MB;87;87;0;97 /snap/snapd/14549=43MB;33;33;0;43 /run/user/1000=0MB;87;87;0;97 /snap/bare/5=0MB;-10;-10;0;0 /snap/multipass-sshfs/145=1MB;-9;-9;0;1 /home/ubuntu/monitoring-plugins=0MB;1023990;1023990;0;1024000 /home/ubuntu/reproduce=0MB;2038;2038;0;2048 /home/ubuntu/reproduce-squash=0MB;2038;2038;0;2048 /snap/core20/1328=62MB;52;52;0;62 /home/ubuntu/reproduce-tracefs=0MB;2038;2038;0;2048
$ /usr/lib/nagios/plugins/check_disk -w 10 -c 10 -e -X tracefs -X squashfs
DISK CRITICAL - free space: /run/lock 5 MB (100% inode=99%);| /dev=0MB;438;438;0;448 /run=0MB;87;87;0;97 /=3886MB;4777;4777;0;4787 /dev/shm=0MB;475;475;0;485 /run/lock=0MB;-5;-5;0;5 /boot/efi=5MB;94;94;0;104 /run/snapd/ns=0MB;87;87;0;97 /run/user/1000=0MB;87;87;0;97 /home/ubuntu/monitoring-plugins=0MB;1023990;1023990;0;1024000 /home/ubuntu/reproduce=0MB;2038;2038;0;2048 /home/ubuntu/reproduce-squash=0MB;2038;2038;0;2048 /home/ubuntu/reproduce-tracefs=0MB;2038;2038;0;2048
$ /usr/lib/nagios/plugins/check_disk -w 10 -c 10 -e -X tmpfs
DISK CRITICAL - free space: /snap/lxd/21835 0 MB (0% inode=0%); /snap/core20/1270 0 MB (0% inode=0%); /snap/snapd/14295 0 MB (0% inode=0%); /snap/snapd/14549 0 MB (0% inode=0%); /snap/bare/5 0 MB (0% inode=0%); /snap/multipass-sshfs/145 0 MB (0% inode=0%); /snap/core20/1328 0 MB (0% inode=0%);| /dev=0MB;438;438;0;448 /=3886MB;4777;4777;0;4787 /snap/lxd/21835=67MB;57;57;0;67 /snap/core20/1270=62MB;52;52;0;62 /snap/snapd/14295=43MB;33;33;0;43 /boot/efi=5MB;94;94;0;104 /snap/snapd/14549=43MB;33;33;0;43 /snap/bare/5=0MB;-10;-10;0;0 /snap/multipass-sshfs/145=1MB;-9;-9;0;1 /home/ubuntu/monitoring-plugins=0MB;1023990;1023990;0;1024000 /snap/core20/1328=62MB;52;52;0;62
# "DISK UNKNOWN" as synthetic filesystems are excluded
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -p /reproduce-tmpfs
DISK UNKNOWN - free space:|
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -N tmpfs
DISK UNKNOWN - free space:|
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -X squashfs -N tmpfs
DISK UNKNOWN - free space:|
2. Good cases:
# "CRITICAL DISK" noise
$ /usr/lib/nagios/plugins/check_disk -w 10 -c 10
DISK OK - free space: / 6971 MB (85% inode=99%); /dev/fuse 32071 MB (100% inode=99%); /dev/net/tun 32071 MB (100% inode=99%); /dev/full 32071 MB (100% inode=99%); /dev/null 32071 MB (100% inode=99%); /dev/random 32071 MB (100% inode=99%); /dev/tty 32071 MB (100% inode=99%); /dev/urandom 32071 MB (100% inode=99%); /dev/zero 32071 MB (100% inode=99%); /snap 6971 MB (85% inode=99%);| /=1204MB;8165;8165;0;8175 /dev/fuse=0MB;32061;32061;0;32071 /dev/net/tun=0MB;32061;32061;0;32071 /dev/full=0MB;32061;32061;0;32071 /dev/null=0MB;32061;32061;0;32071 /dev/random=0MB;32061;32061;0;32071 /dev/tty=0MB;32061;32061;0;32071 /dev/urandom=0MB;32061;32061;0;32071 /dev/zero=0MB;32061;32061;0;32071 /snap=1204MB;8165;8165;0;8175
# "DISK UNKNOWN" as synthetic filesystems are excluded
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -p /reproduce-tmpfs
DISK OK - free space: /reproduce-tmpfs 2048 MB (100% inode=99%);| /reproduce-tmpfs=0MB;1843;1945;0;2048
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -N tmpfs
DISK OK - free space: /dev 0 MB (99% inode=99%); /dev/lxd 0 MB (100% inode=99%); /dev/.lxd-mounts 0 MB (100% inode=99%); /proc/sys/kernel/random/boot_id 0 MB (99% inode=99%); /dev/shm 32106 MB (100% inode=99%); /run 6421 MB (99% inode=99%); /run/lock 5 MB (100% inode=99%); /sys/fs/cgroup 32106 MB (100% inode=99%); /run/user/0 6421 MB (100% inode=99%); /run/snapd/ns 6421 MB (99% inode=99%); /reproduce-tmpfs 2048 MB (100% inode=99%); /reproduce-squashfs 2048 MB (100% inode=99%); /reproduce-tracefs 2048 MB (100% inode=99%);| /dev=0MB;0;0;0;0 /dev/lxd=0MB;0;0;0;0 /dev/.lxd-mounts=0MB;0;0;0;0 /proc/sys/kernel/random/boot_id=0MB;0;0;0;0 /dev/shm=0MB;28895;30500;0;32106 /run=0MB;5778;6099;0;6421 /run/lock=0MB;4;4;0;5 /sys/fs/cgroup=0MB;28895;30500;0;32106 /run/user/0=0MB;5778;6099;0;6421 /run/snapd/ns=0MB;5778;6099;0;6421 /reproduce-tmpfs=0MB;1843;1945;0;2048 /reproduce-squashfs=0MB;1843;1945;0;2048 /reproduce-tracefs=0MB;1843;1945;0;2048
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -N tmpfs -p /reproduce-tmpfs
DISK OK - free space: /reproduce-tmpfs 2048 MB (100% inode=99%);| /reproduce-tmpfs=0MB;1843;1945;0;2048
//This case is fine, as I am explicitly asking for an fs that is being explicitly excluded too.
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -X tmpfs -p /reproduce-tmpfs
DISK UNKNOWN - free space:|
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -X squashfs -p /reproduce-tmpfs
DISK OK - free space: /reproduce-tmpfs 2048 MB (100% inode=99%);| /reproduce-tmpfs=0MB;1843;1945;0;2048
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -X squashfs -N tmpfs
DISK OK - free space: /dev 0 MB (99% inode=99%); /dev/lxd 0 MB (100% inode=99%); /dev/.lxd-mounts 0 MB (100% inode=99%); /proc/sys/kernel/random/boot_id 0 MB (99% inode=99%); /dev/shm 32106 MB (100% inode=99%); /run 6421 MB (99% inode=99%); /run/lock 5 MB (100% inode=99%); /sys/fs/cgroup 32106 MB (100% inode=99%); /run/user/0 6421 MB (100% inode=99%); /run/snapd/ns 6421 MB (99% inode=99%); /reproduce-tmpfs 2048 MB (100% inode=99%); /reproduce-squashfs 2048 MB (100% inode=99%); /reproduce-tracefs 2048 MB (100% inode=99%);| /dev=0MB;0;0;0;0 /dev/lxd=0MB;0;0;0;0 /dev/.lxd-mounts=0MB;0;0;0;0 /proc/sys/kernel/random/boot_id=0MB;0;0;0;0 /dev/shm=0MB;28895;30500;0;32106 /run=0MB;5778;6099;0;6421 /run/lock=0MB;4;4;0;5 /sys/fs/cgroup=0MB;28895;30500;0;32106 /run/user/0=0MB;5778;6099;0;6421 /run/snapd/ns=0MB;5778;6099;0;6421 /reproduce-tmpfs=0MB;1843;1945;0;2048 /reproduce-squashfs=0MB;1843;1945;0;2048 /reproduce-tracefs=0MB;1843;1945;0;2048
[Where problems could occur]
As this alters the logic of how out-of-space checks are handled, relevant issues to keep an eye out for would relate to filesystem checks reporting improperly. These tools underlay a few different front-ends, so regression bugs may get filed in a few different places, however they will tend to display error messages involving check_disk, nagios, and either tmpfs or tracefs. To minimize it, this changes has been done after all the arguments logic has been done (thanks Richard Laager), therefore excluding filesystems only if not required explicitly by the user (thanks Christian Ehrhardt).
Note that there are likely other synthetic filesystems beyond tmpfs, squashfs and tracefs (e.g. udev, usbfs, devtmpfs, fuse.*, ...) which might also cause similar false positives; these should be handled as separate bugs, although the proposed code here has the intention to be easily used to it.
[Other Info]
The intention is forward the patch to upstream once is accepted here, as the previous one (https://github.com/monitoring-plugins/monitoring-plugins/pull/1609 and https://github.com/monitoring-plugins/monitoring-plugins/issues/1608) was rejected.
[Original Report]
-------------------------------------------------------------------
$ lsb_release -d
Description: Ubuntu 20.04.3 LTS
$ apt-cache policy monitoring-plugins | head -n2
monitoring-plugins:
Installed: 2.2-6ubuntu1.1
check_disk returns UNKNOWN for tmpfs filesystems, even when directed explicitly to check them. This is a regression introduced by the fix for LP: #1827159.
Steps to reproduce:
$ df -h /tmp
Filesystem Size Used Avail Use% Mounted on
tmpfs 16G 64K 16G 1% /tmp
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -p /tmp
DISK UNKNOWN - free space:|
Running the check as part of a group, even with just the one member, is a workaround:
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -g tmp -p /tmp
DISK OK - free space: tmp 16058 MB (99% inode=99%);| tmp=0MB;14452;15255;0;16058 |
|
| 2022-01-28 11:48:52 |
Miriam España Acebal |
description |
[Impact]
False positive reports are generated in monitoring tools when artificial file systems are mounted, as they show 100% disk utilisation, and therefore add an unnecessary (but fatal sounding) "CRITICAL DISK" noise. But, if we exclude them by default as we did in the first place in bug 1827159, we limit the use of command options that are commonly used in scripts and that, because of this exclusion, cannot operate on this kind of filesystems (e.g. the path option or the include filesystem option -as excluding has preference over including).
[Test Plan]
0.Preparing the test environment:
#Preparing the container
lxc launch images:ubuntu/focal checkdisks-focal
lxc shell checkdisks-focal
apt update -y
apt upgrade -y
#Installing necessary tools
apt install monitoring-plugins
apt install snapd
#Mount some synthetic systems and make use of snap space
mkdir /reproduce-tmpfs
mkdir /reproduce-squashfs
mkdir /reproduce-tracefs
mount -t tmpfs -o rw,size=2G tmpfs /reproduce-tmpfs
mount -t tmpfs -o rw,size=2G squashfs /reproduce-squashfs
mount -t tmpfs -o rw,size=2G tracefs /reproduce-tracefs
snap install gnome-calculator
1. Bad cases:
# "CRITICAL DISK" noise
$ /usr/lib/nagios/plugins/check_disk -w 10 -c 10
DISK CRITICAL - free space: /dev 448 MB (100% inode=99%); /run 96 MB (98% inode=99%); / 884 MB (18% inode=80%); /dev/shm 485 MB (100% inode=99%); /run/lock 5 MB (100% inode=99%); /snap/lxd/21835 0 MB (0% inode=0%); /snap/core20/1270 0 MB (0% inode=0%); /snap/snapd/14295 0 MB (0% inode=0%); /boot/efi 99 MB (95% inode=-); /run/snapd/ns 96 MB (98% inode=99%); /snap/snapd/14549 0 MB (0% inode=0%); /run/user/1000 97 MB (99% inode=99%); /snap/bare/5 0 MB (0% inode=0%); /snap/multipass-sshfs/145 0 MB (0% inode=0%); /home/ubuntu/monitoring-plugins 1024000 MB (100% inode=100%); /home/ubuntu/reproduce 2048 MB (100% inode=99%); /home/ubuntu/reproduce-squash 2048 MB (100% inode=99%); /snap/core20/1328 0 MB (0% inode=0%); /home/ubuntu/reproduce-tracefs 2048 MB (100% inode=99%);| /dev=0MB;438;438;0;448 /run=0MB;87;87;0;97 /=3886MB;4777;4777;0;4787 /dev/shm=0MB;475;475;0;485 /run/lock=0MB;-5;-5;0;5 /snap/lxd/21835=67MB;57;57;0;67 /snap/core20/1270=62MB;52;52;0;62 /snap/snapd/14295=43MB;33;33;0;43 /boot/efi=5MB;94;94;0;104 /run/snapd/ns=0MB;87;87;0;97 /snap/snapd/14549=43MB;33;33;0;43 /run/user/1000=0MB;87;87;0;97 /snap/bare/5=0MB;-10;-10;0;0 /snap/multipass-sshfs/145=1MB;-9;-9;0;1 /home/ubuntu/monitoring-plugins=0MB;1023990;1023990;0;1024000 /home/ubuntu/reproduce=0MB;2038;2038;0;2048 /home/ubuntu/reproduce-squash=0MB;2038;2038;0;2048 /snap/core20/1328=62MB;52;52;0;62 /home/ubuntu/reproduce-tracefs=0MB;2038;2038;0;2048
$ /usr/lib/nagios/plugins/check_disk -w 10 -c 10 -e -X squashfs
DISK CRITICAL - free space: /run/lock 5 MB (100% inode=99%);| /dev=0MB;438;438;0;448 /run=0MB;87;87;0;97 /=3886MB;4777;4777;0;4787 /dev/shm=0MB;475;475;0;485 /run/lock=0MB;-5;-5;0;5 /boot/efi=5MB;94;94;0;104 /run/snapd/ns=0MB;87;87;0;97 /run/user/1000=0MB;87;87;0;97 /home/ubuntu/monitoring-plugins=0MB;1023990;1023990;0;1024000 /home/ubuntu/reproduce=0MB;2038;2038;0;2048 /home/ubuntu/reproduce-squash=0MB;2038;2038;0;2048 /home/ubuntu/reproduce-tracefs=0MB;2038;2038;0;2048
$ /usr/lib/nagios/plugins/check_disk -w 10 -c 10 -e -X tracefs
DISK CRITICAL - free space: /run/lock 5 MB (100% inode=99%); /snap/lxd/21835 0 MB (0% inode=0%); /snap/core20/1270 0 MB (0% inode=0%); /snap/snapd/14295 0 MB (0% inode=0%); /snap/snapd/14549 0 MB (0% inode=0%); /snap/bare/5 0 MB (0% inode=0%); /snap/multipass-sshfs/145 0 MB (0% inode=0%); /snap/core20/1328 0 MB (0% inode=0%);| /dev=0MB;438;438;0;448 /run=0MB;87;87;0;97 /=3886MB;4777;4777;0;4787 /dev/shm=0MB;475;475;0;485 /run/lock=0MB;-5;-5;0;5 /snap/lxd/21835=67MB;57;57;0;67 /snap/core20/1270=62MB;52;52;0;62 /snap/snapd/14295=43MB;33;33;0;43 /boot/efi=5MB;94;94;0;104 /run/snapd/ns=0MB;87;87;0;97 /snap/snapd/14549=43MB;33;33;0;43 /run/user/1000=0MB;87;87;0;97 /snap/bare/5=0MB;-10;-10;0;0 /snap/multipass-sshfs/145=1MB;-9;-9;0;1 /home/ubuntu/monitoring-plugins=0MB;1023990;1023990;0;1024000 /home/ubuntu/reproduce=0MB;2038;2038;0;2048 /home/ubuntu/reproduce-squash=0MB;2038;2038;0;2048 /snap/core20/1328=62MB;52;52;0;62 /home/ubuntu/reproduce-tracefs=0MB;2038;2038;0;2048
$ /usr/lib/nagios/plugins/check_disk -w 10 -c 10 -e -X tracefs -X squashfs
DISK CRITICAL - free space: /run/lock 5 MB (100% inode=99%);| /dev=0MB;438;438;0;448 /run=0MB;87;87;0;97 /=3886MB;4777;4777;0;4787 /dev/shm=0MB;475;475;0;485 /run/lock=0MB;-5;-5;0;5 /boot/efi=5MB;94;94;0;104 /run/snapd/ns=0MB;87;87;0;97 /run/user/1000=0MB;87;87;0;97 /home/ubuntu/monitoring-plugins=0MB;1023990;1023990;0;1024000 /home/ubuntu/reproduce=0MB;2038;2038;0;2048 /home/ubuntu/reproduce-squash=0MB;2038;2038;0;2048 /home/ubuntu/reproduce-tracefs=0MB;2038;2038;0;2048
$ /usr/lib/nagios/plugins/check_disk -w 10 -c 10 -e -X tmpfs
DISK CRITICAL - free space: /snap/lxd/21835 0 MB (0% inode=0%); /snap/core20/1270 0 MB (0% inode=0%); /snap/snapd/14295 0 MB (0% inode=0%); /snap/snapd/14549 0 MB (0% inode=0%); /snap/bare/5 0 MB (0% inode=0%); /snap/multipass-sshfs/145 0 MB (0% inode=0%); /snap/core20/1328 0 MB (0% inode=0%);| /dev=0MB;438;438;0;448 /=3886MB;4777;4777;0;4787 /snap/lxd/21835=67MB;57;57;0;67 /snap/core20/1270=62MB;52;52;0;62 /snap/snapd/14295=43MB;33;33;0;43 /boot/efi=5MB;94;94;0;104 /snap/snapd/14549=43MB;33;33;0;43 /snap/bare/5=0MB;-10;-10;0;0 /snap/multipass-sshfs/145=1MB;-9;-9;0;1 /home/ubuntu/monitoring-plugins=0MB;1023990;1023990;0;1024000 /snap/core20/1328=62MB;52;52;0;62
# "DISK UNKNOWN" as synthetic filesystems are excluded
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -p /reproduce-tmpfs
DISK UNKNOWN - free space:|
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -N tmpfs
DISK UNKNOWN - free space:|
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -X squashfs -N tmpfs
DISK UNKNOWN - free space:|
2. Good cases:
# "CRITICAL DISK" noise
$ /usr/lib/nagios/plugins/check_disk -w 10 -c 10
DISK OK - free space: / 6971 MB (85% inode=99%); /dev/fuse 32071 MB (100% inode=99%); /dev/net/tun 32071 MB (100% inode=99%); /dev/full 32071 MB (100% inode=99%); /dev/null 32071 MB (100% inode=99%); /dev/random 32071 MB (100% inode=99%); /dev/tty 32071 MB (100% inode=99%); /dev/urandom 32071 MB (100% inode=99%); /dev/zero 32071 MB (100% inode=99%); /snap 6971 MB (85% inode=99%);| /=1204MB;8165;8165;0;8175 /dev/fuse=0MB;32061;32061;0;32071 /dev/net/tun=0MB;32061;32061;0;32071 /dev/full=0MB;32061;32061;0;32071 /dev/null=0MB;32061;32061;0;32071 /dev/random=0MB;32061;32061;0;32071 /dev/tty=0MB;32061;32061;0;32071 /dev/urandom=0MB;32061;32061;0;32071 /dev/zero=0MB;32061;32061;0;32071 /snap=1204MB;8165;8165;0;8175
# "DISK UNKNOWN" as synthetic filesystems are excluded
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -p /reproduce-tmpfs
DISK OK - free space: /reproduce-tmpfs 2048 MB (100% inode=99%);| /reproduce-tmpfs=0MB;1843;1945;0;2048
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -N tmpfs
DISK OK - free space: /dev 0 MB (99% inode=99%); /dev/lxd 0 MB (100% inode=99%); /dev/.lxd-mounts 0 MB (100% inode=99%); /proc/sys/kernel/random/boot_id 0 MB (99% inode=99%); /dev/shm 32106 MB (100% inode=99%); /run 6421 MB (99% inode=99%); /run/lock 5 MB (100% inode=99%); /sys/fs/cgroup 32106 MB (100% inode=99%); /run/user/0 6421 MB (100% inode=99%); /run/snapd/ns 6421 MB (99% inode=99%); /reproduce-tmpfs 2048 MB (100% inode=99%); /reproduce-squashfs 2048 MB (100% inode=99%); /reproduce-tracefs 2048 MB (100% inode=99%);| /dev=0MB;0;0;0;0 /dev/lxd=0MB;0;0;0;0 /dev/.lxd-mounts=0MB;0;0;0;0 /proc/sys/kernel/random/boot_id=0MB;0;0;0;0 /dev/shm=0MB;28895;30500;0;32106 /run=0MB;5778;6099;0;6421 /run/lock=0MB;4;4;0;5 /sys/fs/cgroup=0MB;28895;30500;0;32106 /run/user/0=0MB;5778;6099;0;6421 /run/snapd/ns=0MB;5778;6099;0;6421 /reproduce-tmpfs=0MB;1843;1945;0;2048 /reproduce-squashfs=0MB;1843;1945;0;2048 /reproduce-tracefs=0MB;1843;1945;0;2048
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -N tmpfs -p /reproduce-tmpfs
DISK OK - free space: /reproduce-tmpfs 2048 MB (100% inode=99%);| /reproduce-tmpfs=0MB;1843;1945;0;2048
//This case is fine, as I am explicitly asking for an fs that is being explicitly excluded too.
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -X tmpfs -p /reproduce-tmpfs
DISK UNKNOWN - free space:|
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -X squashfs -p /reproduce-tmpfs
DISK OK - free space: /reproduce-tmpfs 2048 MB (100% inode=99%);| /reproduce-tmpfs=0MB;1843;1945;0;2048
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -X squashfs -N tmpfs
DISK OK - free space: /dev 0 MB (99% inode=99%); /dev/lxd 0 MB (100% inode=99%); /dev/.lxd-mounts 0 MB (100% inode=99%); /proc/sys/kernel/random/boot_id 0 MB (99% inode=99%); /dev/shm 32106 MB (100% inode=99%); /run 6421 MB (99% inode=99%); /run/lock 5 MB (100% inode=99%); /sys/fs/cgroup 32106 MB (100% inode=99%); /run/user/0 6421 MB (100% inode=99%); /run/snapd/ns 6421 MB (99% inode=99%); /reproduce-tmpfs 2048 MB (100% inode=99%); /reproduce-squashfs 2048 MB (100% inode=99%); /reproduce-tracefs 2048 MB (100% inode=99%);| /dev=0MB;0;0;0;0 /dev/lxd=0MB;0;0;0;0 /dev/.lxd-mounts=0MB;0;0;0;0 /proc/sys/kernel/random/boot_id=0MB;0;0;0;0 /dev/shm=0MB;28895;30500;0;32106 /run=0MB;5778;6099;0;6421 /run/lock=0MB;4;4;0;5 /sys/fs/cgroup=0MB;28895;30500;0;32106 /run/user/0=0MB;5778;6099;0;6421 /run/snapd/ns=0MB;5778;6099;0;6421 /reproduce-tmpfs=0MB;1843;1945;0;2048 /reproduce-squashfs=0MB;1843;1945;0;2048 /reproduce-tracefs=0MB;1843;1945;0;2048
[Where problems could occur]
As this alters the logic of how out-of-space checks are handled, relevant issues to keep an eye out for would relate to filesystem checks reporting improperly. These tools underlay a few different front-ends, so regression bugs may get filed in a few different places, however they will tend to display error messages involving check_disk, nagios, and either tmpfs or tracefs. To minimize it, this changes has been done after all the arguments logic has been done (thanks Richard Laager), therefore excluding filesystems only if not required explicitly by the user (thanks Christian Ehrhardt).
Note that there are likely other synthetic filesystems beyond tmpfs, squashfs and tracefs (e.g. udev, usbfs, devtmpfs, fuse.*, ...) which might also cause similar false positives; these should be handled as separate bugs, although the proposed code here has the intention to be easily used to it.
[Other Info]
The intention is forward the patch to upstream once is accepted here, as the previous one (https://github.com/monitoring-plugins/monitoring-plugins/pull/1609 and https://github.com/monitoring-plugins/monitoring-plugins/issues/1608) was rejected.
[Original Report]
-------------------------------------------------------------------
$ lsb_release -d
Description: Ubuntu 20.04.3 LTS
$ apt-cache policy monitoring-plugins | head -n2
monitoring-plugins:
Installed: 2.2-6ubuntu1.1
check_disk returns UNKNOWN for tmpfs filesystems, even when directed explicitly to check them. This is a regression introduced by the fix for LP: #1827159.
Steps to reproduce:
$ df -h /tmp
Filesystem Size Used Avail Use% Mounted on
tmpfs 16G 64K 16G 1% /tmp
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -p /tmp
DISK UNKNOWN - free space:|
Running the check as part of a group, even with just the one member, is a workaround:
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -g tmp -p /tmp
DISK OK - free space: tmp 16058 MB (99% inode=99%);| tmp=0MB;14452;15255;0;16058 |
[SRU]
[Impact]
False positive reports are generated in monitoring tools when artificial file systems are mounted, as they show 100% disk utilisation, and therefore add an unnecessary (but fatal sounding) "CRITICAL DISK" noise. But, if we exclude them by default as we did in the first place in bug 1827159, we limit the use of command options that are commonly used in scripts and that, because of this exclusion, cannot operate on this kind of filesystems (e.g. the path option or the include filesystem option -as excluding has preference over including).
[Test Plan]
0.Preparing the test environment:
#Preparing the container
lxc launch images:ubuntu/focal checkdisks-focal
lxc shell checkdisks-focal
apt update -y
apt upgrade -y
#Installing necessary tools
apt install monitoring-plugins
apt install snapd
#Mount some synthetic systems and make use of snap space
mkdir /reproduce-tmpfs
mkdir /reproduce-squashfs
mkdir /reproduce-tracefs
mount -t tmpfs -o rw,size=2G tmpfs /reproduce-tmpfs
mount -t tmpfs -o rw,size=2G squashfs /reproduce-squashfs
mount -t tmpfs -o rw,size=2G tracefs /reproduce-tracefs
snap install gnome-calculator
1. Bad cases:
# "CRITICAL DISK" noise
$ /usr/lib/nagios/plugins/check_disk -w 10 -c 10
DISK CRITICAL - free space: /dev 448 MB (100% inode=99%); /run 96 MB (98% inode=99%); / 884 MB (18% inode=80%); /dev/shm 485 MB (100% inode=99%); /run/lock 5 MB (100% inode=99%); /snap/lxd/21835 0 MB (0% inode=0%); /snap/core20/1270 0 MB (0% inode=0%); /snap/snapd/14295 0 MB (0% inode=0%); /boot/efi 99 MB (95% inode=-); /run/snapd/ns 96 MB (98% inode=99%); /snap/snapd/14549 0 MB (0% inode=0%); /run/user/1000 97 MB (99% inode=99%); /snap/bare/5 0 MB (0% inode=0%); /snap/multipass-sshfs/145 0 MB (0% inode=0%); /home/ubuntu/monitoring-plugins 1024000 MB (100% inode=100%); /home/ubuntu/reproduce 2048 MB (100% inode=99%); /home/ubuntu/reproduce-squash 2048 MB (100% inode=99%); /snap/core20/1328 0 MB (0% inode=0%); /home/ubuntu/reproduce-tracefs 2048 MB (100% inode=99%);| /dev=0MB;438;438;0;448 /run=0MB;87;87;0;97 /=3886MB;4777;4777;0;4787 /dev/shm=0MB;475;475;0;485 /run/lock=0MB;-5;-5;0;5 /snap/lxd/21835=67MB;57;57;0;67 /snap/core20/1270=62MB;52;52;0;62 /snap/snapd/14295=43MB;33;33;0;43 /boot/efi=5MB;94;94;0;104 /run/snapd/ns=0MB;87;87;0;97 /snap/snapd/14549=43MB;33;33;0;43 /run/user/1000=0MB;87;87;0;97 /snap/bare/5=0MB;-10;-10;0;0 /snap/multipass-sshfs/145=1MB;-9;-9;0;1 /home/ubuntu/monitoring-plugins=0MB;1023990;1023990;0;1024000 /home/ubuntu/reproduce=0MB;2038;2038;0;2048 /home/ubuntu/reproduce-squash=0MB;2038;2038;0;2048 /snap/core20/1328=62MB;52;52;0;62 /home/ubuntu/reproduce-tracefs=0MB;2038;2038;0;2048
$ /usr/lib/nagios/plugins/check_disk -w 10 -c 10 -e -X squashfs
DISK CRITICAL - free space: /run/lock 5 MB (100% inode=99%);| /dev=0MB;438;438;0;448 /run=0MB;87;87;0;97 /=3886MB;4777;4777;0;4787 /dev/shm=0MB;475;475;0;485 /run/lock=0MB;-5;-5;0;5 /boot/efi=5MB;94;94;0;104 /run/snapd/ns=0MB;87;87;0;97 /run/user/1000=0MB;87;87;0;97 /home/ubuntu/monitoring-plugins=0MB;1023990;1023990;0;1024000 /home/ubuntu/reproduce=0MB;2038;2038;0;2048 /home/ubuntu/reproduce-squash=0MB;2038;2038;0;2048 /home/ubuntu/reproduce-tracefs=0MB;2038;2038;0;2048
$ /usr/lib/nagios/plugins/check_disk -w 10 -c 10 -e -X tracefs
DISK CRITICAL - free space: /run/lock 5 MB (100% inode=99%); /snap/lxd/21835 0 MB (0% inode=0%); /snap/core20/1270 0 MB (0% inode=0%); /snap/snapd/14295 0 MB (0% inode=0%); /snap/snapd/14549 0 MB (0% inode=0%); /snap/bare/5 0 MB (0% inode=0%); /snap/multipass-sshfs/145 0 MB (0% inode=0%); /snap/core20/1328 0 MB (0% inode=0%);| /dev=0MB;438;438;0;448 /run=0MB;87;87;0;97 /=3886MB;4777;4777;0;4787 /dev/shm=0MB;475;475;0;485 /run/lock=0MB;-5;-5;0;5 /snap/lxd/21835=67MB;57;57;0;67 /snap/core20/1270=62MB;52;52;0;62 /snap/snapd/14295=43MB;33;33;0;43 /boot/efi=5MB;94;94;0;104 /run/snapd/ns=0MB;87;87;0;97 /snap/snapd/14549=43MB;33;33;0;43 /run/user/1000=0MB;87;87;0;97 /snap/bare/5=0MB;-10;-10;0;0 /snap/multipass-sshfs/145=1MB;-9;-9;0;1 /home/ubuntu/monitoring-plugins=0MB;1023990;1023990;0;1024000 /home/ubuntu/reproduce=0MB;2038;2038;0;2048 /home/ubuntu/reproduce-squash=0MB;2038;2038;0;2048 /snap/core20/1328=62MB;52;52;0;62 /home/ubuntu/reproduce-tracefs=0MB;2038;2038;0;2048
$ /usr/lib/nagios/plugins/check_disk -w 10 -c 10 -e -X tracefs -X squashfs
DISK CRITICAL - free space: /run/lock 5 MB (100% inode=99%);| /dev=0MB;438;438;0;448 /run=0MB;87;87;0;97 /=3886MB;4777;4777;0;4787 /dev/shm=0MB;475;475;0;485 /run/lock=0MB;-5;-5;0;5 /boot/efi=5MB;94;94;0;104 /run/snapd/ns=0MB;87;87;0;97 /run/user/1000=0MB;87;87;0;97 /home/ubuntu/monitoring-plugins=0MB;1023990;1023990;0;1024000 /home/ubuntu/reproduce=0MB;2038;2038;0;2048 /home/ubuntu/reproduce-squash=0MB;2038;2038;0;2048 /home/ubuntu/reproduce-tracefs=0MB;2038;2038;0;2048
$ /usr/lib/nagios/plugins/check_disk -w 10 -c 10 -e -X tmpfs
DISK CRITICAL - free space: /snap/lxd/21835 0 MB (0% inode=0%); /snap/core20/1270 0 MB (0% inode=0%); /snap/snapd/14295 0 MB (0% inode=0%); /snap/snapd/14549 0 MB (0% inode=0%); /snap/bare/5 0 MB (0% inode=0%); /snap/multipass-sshfs/145 0 MB (0% inode=0%); /snap/core20/1328 0 MB (0% inode=0%);| /dev=0MB;438;438;0;448 /=3886MB;4777;4777;0;4787 /snap/lxd/21835=67MB;57;57;0;67 /snap/core20/1270=62MB;52;52;0;62 /snap/snapd/14295=43MB;33;33;0;43 /boot/efi=5MB;94;94;0;104 /snap/snapd/14549=43MB;33;33;0;43 /snap/bare/5=0MB;-10;-10;0;0 /snap/multipass-sshfs/145=1MB;-9;-9;0;1 /home/ubuntu/monitoring-plugins=0MB;1023990;1023990;0;1024000 /snap/core20/1328=62MB;52;52;0;62
# "DISK UNKNOWN" as synthetic filesystems are excluded
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -p /reproduce-tmpfs
DISK UNKNOWN - free space:|
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -N tmpfs
DISK UNKNOWN - free space:|
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -X squashfs -N tmpfs
DISK UNKNOWN - free space:|
2. Good cases:
# "CRITICAL DISK" noise
$ /usr/lib/nagios/plugins/check_disk -w 10 -c 10
DISK OK - free space: / 6971 MB (85% inode=99%); /dev/fuse 32071 MB (100% inode=99%); /dev/net/tun 32071 MB (100% inode=99%); /dev/full 32071 MB (100% inode=99%); /dev/null 32071 MB (100% inode=99%); /dev/random 32071 MB (100% inode=99%); /dev/tty 32071 MB (100% inode=99%); /dev/urandom 32071 MB (100% inode=99%); /dev/zero 32071 MB (100% inode=99%); /snap 6971 MB (85% inode=99%);| /=1204MB;8165;8165;0;8175 /dev/fuse=0MB;32061;32061;0;32071 /dev/net/tun=0MB;32061;32061;0;32071 /dev/full=0MB;32061;32061;0;32071 /dev/null=0MB;32061;32061;0;32071 /dev/random=0MB;32061;32061;0;32071 /dev/tty=0MB;32061;32061;0;32071 /dev/urandom=0MB;32061;32061;0;32071 /dev/zero=0MB;32061;32061;0;32071 /snap=1204MB;8165;8165;0;8175
# "DISK UNKNOWN" as synthetic filesystems are excluded
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -p /reproduce-tmpfs
DISK OK - free space: /reproduce-tmpfs 2048 MB (100% inode=99%);| /reproduce-tmpfs=0MB;1843;1945;0;2048
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -N tmpfs
DISK OK - free space: /dev 0 MB (99% inode=99%); /dev/lxd 0 MB (100% inode=99%); /dev/.lxd-mounts 0 MB (100% inode=99%); /proc/sys/kernel/random/boot_id 0 MB (99% inode=99%); /dev/shm 32106 MB (100% inode=99%); /run 6421 MB (99% inode=99%); /run/lock 5 MB (100% inode=99%); /sys/fs/cgroup 32106 MB (100% inode=99%); /run/user/0 6421 MB (100% inode=99%); /run/snapd/ns 6421 MB (99% inode=99%); /reproduce-tmpfs 2048 MB (100% inode=99%); /reproduce-squashfs 2048 MB (100% inode=99%); /reproduce-tracefs 2048 MB (100% inode=99%);| /dev=0MB;0;0;0;0 /dev/lxd=0MB;0;0;0;0 /dev/.lxd-mounts=0MB;0;0;0;0 /proc/sys/kernel/random/boot_id=0MB;0;0;0;0 /dev/shm=0MB;28895;30500;0;32106 /run=0MB;5778;6099;0;6421 /run/lock=0MB;4;4;0;5 /sys/fs/cgroup=0MB;28895;30500;0;32106 /run/user/0=0MB;5778;6099;0;6421 /run/snapd/ns=0MB;5778;6099;0;6421 /reproduce-tmpfs=0MB;1843;1945;0;2048 /reproduce-squashfs=0MB;1843;1945;0;2048 /reproduce-tracefs=0MB;1843;1945;0;2048
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -N tmpfs -p /reproduce-tmpfs
DISK OK - free space: /reproduce-tmpfs 2048 MB (100% inode=99%);| /reproduce-tmpfs=0MB;1843;1945;0;2048
//This case is fine, as I am explicitly asking for an fs that is being explicitly excluded too.
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -X tmpfs -p /reproduce-tmpfs
DISK UNKNOWN - free space:|
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -X squashfs -p /reproduce-tmpfs
DISK OK - free space: /reproduce-tmpfs 2048 MB (100% inode=99%);| /reproduce-tmpfs=0MB;1843;1945;0;2048
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -X squashfs -N tmpfs
DISK OK - free space: /dev 0 MB (99% inode=99%); /dev/lxd 0 MB (100% inode=99%); /dev/.lxd-mounts 0 MB (100% inode=99%); /proc/sys/kernel/random/boot_id 0 MB (99% inode=99%); /dev/shm 32106 MB (100% inode=99%); /run 6421 MB (99% inode=99%); /run/lock 5 MB (100% inode=99%); /sys/fs/cgroup 32106 MB (100% inode=99%); /run/user/0 6421 MB (100% inode=99%); /run/snapd/ns 6421 MB (99% inode=99%); /reproduce-tmpfs 2048 MB (100% inode=99%); /reproduce-squashfs 2048 MB (100% inode=99%); /reproduce-tracefs 2048 MB (100% inode=99%);| /dev=0MB;0;0;0;0 /dev/lxd=0MB;0;0;0;0 /dev/.lxd-mounts=0MB;0;0;0;0 /proc/sys/kernel/random/boot_id=0MB;0;0;0;0 /dev/shm=0MB;28895;30500;0;32106 /run=0MB;5778;6099;0;6421 /run/lock=0MB;4;4;0;5 /sys/fs/cgroup=0MB;28895;30500;0;32106 /run/user/0=0MB;5778;6099;0;6421 /run/snapd/ns=0MB;5778;6099;0;6421 /reproduce-tmpfs=0MB;1843;1945;0;2048 /reproduce-squashfs=0MB;1843;1945;0;2048 /reproduce-tracefs=0MB;1843;1945;0;2048
[Where problems could occur]
As this alters the logic of how out-of-space checks are handled, relevant issues to keep an eye out for would relate to filesystem checks reporting improperly. These tools underlay a few different front-ends, so regression bugs may get filed in a few different places, however they will tend to display error messages involving check_disk, nagios, and either tmpfs or tracefs. To minimize it, this changes has been done after all the arguments logic has been done (thanks Richard Laager), therefore excluding filesystems only if not required explicitly by the user (thanks Christian Ehrhardt).
Note that there are likely other synthetic filesystems beyond tmpfs, squashfs and tracefs (e.g. udev, usbfs, devtmpfs, fuse.*, ...) which might also cause similar false positives; these should be handled as separate bugs, although the proposed code here has the intention to be easily used to it.
[Other Info]
The intention is forward the patch to upstream once is accepted here, as the previous one (https://github.com/monitoring-plugins/monitoring-plugins/pull/1609 and https://github.com/monitoring-plugins/monitoring-plugins/issues/1608) was rejected.
[Original Report]
-------------------------------------------------------------------
$ lsb_release -d
Description: Ubuntu 20.04.3 LTS
$ apt-cache policy monitoring-plugins | head -n2
monitoring-plugins:
Installed: 2.2-6ubuntu1.1
check_disk returns UNKNOWN for tmpfs filesystems, even when directed explicitly to check them. This is a regression introduced by the fix for LP: #1827159.
Steps to reproduce:
$ df -h /tmp
Filesystem Size Used Avail Use% Mounted on
tmpfs 16G 64K 16G 1% /tmp
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -p /tmp
DISK UNKNOWN - free space:|
Running the check as part of a group, even with just the one member, is a workaround:
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -g tmp -p /tmp
DISK OK - free space: tmp 16058 MB (99% inode=99%);| tmp=0MB;14452;15255;0;16058 |
|
| 2022-01-28 12:13:00 |
Launchpad Janitor |
merge proposal linked |
|
https://code.launchpad.net/~mirespace/ubuntu/+source/monitoring-plugins/+git/monitoring-plugins/+merge/414725 |
|
| 2022-02-01 03:01:48 |
Bryce Harrington |
bug task added |
|
percona-monitoring-plugins |
|
| 2022-02-01 03:02:18 |
Bryce Harrington |
bug task deleted |
percona-monitoring-plugins |
|
|
| 2022-02-01 14:57:18 |
Miriam España Acebal |
description |
[SRU]
[Impact]
False positive reports are generated in monitoring tools when artificial file systems are mounted, as they show 100% disk utilisation, and therefore add an unnecessary (but fatal sounding) "CRITICAL DISK" noise. But, if we exclude them by default as we did in the first place in bug 1827159, we limit the use of command options that are commonly used in scripts and that, because of this exclusion, cannot operate on this kind of filesystems (e.g. the path option or the include filesystem option -as excluding has preference over including).
[Test Plan]
0.Preparing the test environment:
#Preparing the container
lxc launch images:ubuntu/focal checkdisks-focal
lxc shell checkdisks-focal
apt update -y
apt upgrade -y
#Installing necessary tools
apt install monitoring-plugins
apt install snapd
#Mount some synthetic systems and make use of snap space
mkdir /reproduce-tmpfs
mkdir /reproduce-squashfs
mkdir /reproduce-tracefs
mount -t tmpfs -o rw,size=2G tmpfs /reproduce-tmpfs
mount -t tmpfs -o rw,size=2G squashfs /reproduce-squashfs
mount -t tmpfs -o rw,size=2G tracefs /reproduce-tracefs
snap install gnome-calculator
1. Bad cases:
# "CRITICAL DISK" noise
$ /usr/lib/nagios/plugins/check_disk -w 10 -c 10
DISK CRITICAL - free space: /dev 448 MB (100% inode=99%); /run 96 MB (98% inode=99%); / 884 MB (18% inode=80%); /dev/shm 485 MB (100% inode=99%); /run/lock 5 MB (100% inode=99%); /snap/lxd/21835 0 MB (0% inode=0%); /snap/core20/1270 0 MB (0% inode=0%); /snap/snapd/14295 0 MB (0% inode=0%); /boot/efi 99 MB (95% inode=-); /run/snapd/ns 96 MB (98% inode=99%); /snap/snapd/14549 0 MB (0% inode=0%); /run/user/1000 97 MB (99% inode=99%); /snap/bare/5 0 MB (0% inode=0%); /snap/multipass-sshfs/145 0 MB (0% inode=0%); /home/ubuntu/monitoring-plugins 1024000 MB (100% inode=100%); /home/ubuntu/reproduce 2048 MB (100% inode=99%); /home/ubuntu/reproduce-squash 2048 MB (100% inode=99%); /snap/core20/1328 0 MB (0% inode=0%); /home/ubuntu/reproduce-tracefs 2048 MB (100% inode=99%);| /dev=0MB;438;438;0;448 /run=0MB;87;87;0;97 /=3886MB;4777;4777;0;4787 /dev/shm=0MB;475;475;0;485 /run/lock=0MB;-5;-5;0;5 /snap/lxd/21835=67MB;57;57;0;67 /snap/core20/1270=62MB;52;52;0;62 /snap/snapd/14295=43MB;33;33;0;43 /boot/efi=5MB;94;94;0;104 /run/snapd/ns=0MB;87;87;0;97 /snap/snapd/14549=43MB;33;33;0;43 /run/user/1000=0MB;87;87;0;97 /snap/bare/5=0MB;-10;-10;0;0 /snap/multipass-sshfs/145=1MB;-9;-9;0;1 /home/ubuntu/monitoring-plugins=0MB;1023990;1023990;0;1024000 /home/ubuntu/reproduce=0MB;2038;2038;0;2048 /home/ubuntu/reproduce-squash=0MB;2038;2038;0;2048 /snap/core20/1328=62MB;52;52;0;62 /home/ubuntu/reproduce-tracefs=0MB;2038;2038;0;2048
$ /usr/lib/nagios/plugins/check_disk -w 10 -c 10 -e -X squashfs
DISK CRITICAL - free space: /run/lock 5 MB (100% inode=99%);| /dev=0MB;438;438;0;448 /run=0MB;87;87;0;97 /=3886MB;4777;4777;0;4787 /dev/shm=0MB;475;475;0;485 /run/lock=0MB;-5;-5;0;5 /boot/efi=5MB;94;94;0;104 /run/snapd/ns=0MB;87;87;0;97 /run/user/1000=0MB;87;87;0;97 /home/ubuntu/monitoring-plugins=0MB;1023990;1023990;0;1024000 /home/ubuntu/reproduce=0MB;2038;2038;0;2048 /home/ubuntu/reproduce-squash=0MB;2038;2038;0;2048 /home/ubuntu/reproduce-tracefs=0MB;2038;2038;0;2048
$ /usr/lib/nagios/plugins/check_disk -w 10 -c 10 -e -X tracefs
DISK CRITICAL - free space: /run/lock 5 MB (100% inode=99%); /snap/lxd/21835 0 MB (0% inode=0%); /snap/core20/1270 0 MB (0% inode=0%); /snap/snapd/14295 0 MB (0% inode=0%); /snap/snapd/14549 0 MB (0% inode=0%); /snap/bare/5 0 MB (0% inode=0%); /snap/multipass-sshfs/145 0 MB (0% inode=0%); /snap/core20/1328 0 MB (0% inode=0%);| /dev=0MB;438;438;0;448 /run=0MB;87;87;0;97 /=3886MB;4777;4777;0;4787 /dev/shm=0MB;475;475;0;485 /run/lock=0MB;-5;-5;0;5 /snap/lxd/21835=67MB;57;57;0;67 /snap/core20/1270=62MB;52;52;0;62 /snap/snapd/14295=43MB;33;33;0;43 /boot/efi=5MB;94;94;0;104 /run/snapd/ns=0MB;87;87;0;97 /snap/snapd/14549=43MB;33;33;0;43 /run/user/1000=0MB;87;87;0;97 /snap/bare/5=0MB;-10;-10;0;0 /snap/multipass-sshfs/145=1MB;-9;-9;0;1 /home/ubuntu/monitoring-plugins=0MB;1023990;1023990;0;1024000 /home/ubuntu/reproduce=0MB;2038;2038;0;2048 /home/ubuntu/reproduce-squash=0MB;2038;2038;0;2048 /snap/core20/1328=62MB;52;52;0;62 /home/ubuntu/reproduce-tracefs=0MB;2038;2038;0;2048
$ /usr/lib/nagios/plugins/check_disk -w 10 -c 10 -e -X tracefs -X squashfs
DISK CRITICAL - free space: /run/lock 5 MB (100% inode=99%);| /dev=0MB;438;438;0;448 /run=0MB;87;87;0;97 /=3886MB;4777;4777;0;4787 /dev/shm=0MB;475;475;0;485 /run/lock=0MB;-5;-5;0;5 /boot/efi=5MB;94;94;0;104 /run/snapd/ns=0MB;87;87;0;97 /run/user/1000=0MB;87;87;0;97 /home/ubuntu/monitoring-plugins=0MB;1023990;1023990;0;1024000 /home/ubuntu/reproduce=0MB;2038;2038;0;2048 /home/ubuntu/reproduce-squash=0MB;2038;2038;0;2048 /home/ubuntu/reproduce-tracefs=0MB;2038;2038;0;2048
$ /usr/lib/nagios/plugins/check_disk -w 10 -c 10 -e -X tmpfs
DISK CRITICAL - free space: /snap/lxd/21835 0 MB (0% inode=0%); /snap/core20/1270 0 MB (0% inode=0%); /snap/snapd/14295 0 MB (0% inode=0%); /snap/snapd/14549 0 MB (0% inode=0%); /snap/bare/5 0 MB (0% inode=0%); /snap/multipass-sshfs/145 0 MB (0% inode=0%); /snap/core20/1328 0 MB (0% inode=0%);| /dev=0MB;438;438;0;448 /=3886MB;4777;4777;0;4787 /snap/lxd/21835=67MB;57;57;0;67 /snap/core20/1270=62MB;52;52;0;62 /snap/snapd/14295=43MB;33;33;0;43 /boot/efi=5MB;94;94;0;104 /snap/snapd/14549=43MB;33;33;0;43 /snap/bare/5=0MB;-10;-10;0;0 /snap/multipass-sshfs/145=1MB;-9;-9;0;1 /home/ubuntu/monitoring-plugins=0MB;1023990;1023990;0;1024000 /snap/core20/1328=62MB;52;52;0;62
# "DISK UNKNOWN" as synthetic filesystems are excluded
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -p /reproduce-tmpfs
DISK UNKNOWN - free space:|
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -N tmpfs
DISK UNKNOWN - free space:|
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -X squashfs -N tmpfs
DISK UNKNOWN - free space:|
2. Good cases:
# "CRITICAL DISK" noise
$ /usr/lib/nagios/plugins/check_disk -w 10 -c 10
DISK OK - free space: / 6971 MB (85% inode=99%); /dev/fuse 32071 MB (100% inode=99%); /dev/net/tun 32071 MB (100% inode=99%); /dev/full 32071 MB (100% inode=99%); /dev/null 32071 MB (100% inode=99%); /dev/random 32071 MB (100% inode=99%); /dev/tty 32071 MB (100% inode=99%); /dev/urandom 32071 MB (100% inode=99%); /dev/zero 32071 MB (100% inode=99%); /snap 6971 MB (85% inode=99%);| /=1204MB;8165;8165;0;8175 /dev/fuse=0MB;32061;32061;0;32071 /dev/net/tun=0MB;32061;32061;0;32071 /dev/full=0MB;32061;32061;0;32071 /dev/null=0MB;32061;32061;0;32071 /dev/random=0MB;32061;32061;0;32071 /dev/tty=0MB;32061;32061;0;32071 /dev/urandom=0MB;32061;32061;0;32071 /dev/zero=0MB;32061;32061;0;32071 /snap=1204MB;8165;8165;0;8175
# "DISK UNKNOWN" as synthetic filesystems are excluded
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -p /reproduce-tmpfs
DISK OK - free space: /reproduce-tmpfs 2048 MB (100% inode=99%);| /reproduce-tmpfs=0MB;1843;1945;0;2048
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -N tmpfs
DISK OK - free space: /dev 0 MB (99% inode=99%); /dev/lxd 0 MB (100% inode=99%); /dev/.lxd-mounts 0 MB (100% inode=99%); /proc/sys/kernel/random/boot_id 0 MB (99% inode=99%); /dev/shm 32106 MB (100% inode=99%); /run 6421 MB (99% inode=99%); /run/lock 5 MB (100% inode=99%); /sys/fs/cgroup 32106 MB (100% inode=99%); /run/user/0 6421 MB (100% inode=99%); /run/snapd/ns 6421 MB (99% inode=99%); /reproduce-tmpfs 2048 MB (100% inode=99%); /reproduce-squashfs 2048 MB (100% inode=99%); /reproduce-tracefs 2048 MB (100% inode=99%);| /dev=0MB;0;0;0;0 /dev/lxd=0MB;0;0;0;0 /dev/.lxd-mounts=0MB;0;0;0;0 /proc/sys/kernel/random/boot_id=0MB;0;0;0;0 /dev/shm=0MB;28895;30500;0;32106 /run=0MB;5778;6099;0;6421 /run/lock=0MB;4;4;0;5 /sys/fs/cgroup=0MB;28895;30500;0;32106 /run/user/0=0MB;5778;6099;0;6421 /run/snapd/ns=0MB;5778;6099;0;6421 /reproduce-tmpfs=0MB;1843;1945;0;2048 /reproduce-squashfs=0MB;1843;1945;0;2048 /reproduce-tracefs=0MB;1843;1945;0;2048
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -N tmpfs -p /reproduce-tmpfs
DISK OK - free space: /reproduce-tmpfs 2048 MB (100% inode=99%);| /reproduce-tmpfs=0MB;1843;1945;0;2048
//This case is fine, as I am explicitly asking for an fs that is being explicitly excluded too.
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -X tmpfs -p /reproduce-tmpfs
DISK UNKNOWN - free space:|
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -X squashfs -p /reproduce-tmpfs
DISK OK - free space: /reproduce-tmpfs 2048 MB (100% inode=99%);| /reproduce-tmpfs=0MB;1843;1945;0;2048
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -X squashfs -N tmpfs
DISK OK - free space: /dev 0 MB (99% inode=99%); /dev/lxd 0 MB (100% inode=99%); /dev/.lxd-mounts 0 MB (100% inode=99%); /proc/sys/kernel/random/boot_id 0 MB (99% inode=99%); /dev/shm 32106 MB (100% inode=99%); /run 6421 MB (99% inode=99%); /run/lock 5 MB (100% inode=99%); /sys/fs/cgroup 32106 MB (100% inode=99%); /run/user/0 6421 MB (100% inode=99%); /run/snapd/ns 6421 MB (99% inode=99%); /reproduce-tmpfs 2048 MB (100% inode=99%); /reproduce-squashfs 2048 MB (100% inode=99%); /reproduce-tracefs 2048 MB (100% inode=99%);| /dev=0MB;0;0;0;0 /dev/lxd=0MB;0;0;0;0 /dev/.lxd-mounts=0MB;0;0;0;0 /proc/sys/kernel/random/boot_id=0MB;0;0;0;0 /dev/shm=0MB;28895;30500;0;32106 /run=0MB;5778;6099;0;6421 /run/lock=0MB;4;4;0;5 /sys/fs/cgroup=0MB;28895;30500;0;32106 /run/user/0=0MB;5778;6099;0;6421 /run/snapd/ns=0MB;5778;6099;0;6421 /reproduce-tmpfs=0MB;1843;1945;0;2048 /reproduce-squashfs=0MB;1843;1945;0;2048 /reproduce-tracefs=0MB;1843;1945;0;2048
[Where problems could occur]
As this alters the logic of how out-of-space checks are handled, relevant issues to keep an eye out for would relate to filesystem checks reporting improperly. These tools underlay a few different front-ends, so regression bugs may get filed in a few different places, however they will tend to display error messages involving check_disk, nagios, and either tmpfs or tracefs. To minimize it, this changes has been done after all the arguments logic has been done (thanks Richard Laager), therefore excluding filesystems only if not required explicitly by the user (thanks Christian Ehrhardt).
Note that there are likely other synthetic filesystems beyond tmpfs, squashfs and tracefs (e.g. udev, usbfs, devtmpfs, fuse.*, ...) which might also cause similar false positives; these should be handled as separate bugs, although the proposed code here has the intention to be easily used to it.
[Other Info]
The intention is forward the patch to upstream once is accepted here, as the previous one (https://github.com/monitoring-plugins/monitoring-plugins/pull/1609 and https://github.com/monitoring-plugins/monitoring-plugins/issues/1608) was rejected.
[Original Report]
-------------------------------------------------------------------
$ lsb_release -d
Description: Ubuntu 20.04.3 LTS
$ apt-cache policy monitoring-plugins | head -n2
monitoring-plugins:
Installed: 2.2-6ubuntu1.1
check_disk returns UNKNOWN for tmpfs filesystems, even when directed explicitly to check them. This is a regression introduced by the fix for LP: #1827159.
Steps to reproduce:
$ df -h /tmp
Filesystem Size Used Avail Use% Mounted on
tmpfs 16G 64K 16G 1% /tmp
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -p /tmp
DISK UNKNOWN - free space:|
Running the check as part of a group, even with just the one member, is a workaround:
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -g tmp -p /tmp
DISK OK - free space: tmp 16058 MB (99% inode=99%);| tmp=0MB;14452;15255;0;16058 |
[SRU]
[Impact]
False positive reports are generated in monitoring tools when artificial file systems are mounted, as they show 100% disk utilisation, and therefore add an unnecessary (but fatal sounding) "CRITICAL DISK" noise. But, if we exclude them by default as we did in the first place in bug 1827159, we limit the use of command options that are commonly used in scripts and that, because of this exclusion, cannot operate on this kind of filesystems (e.g. the path option or the include filesystem option -as excluding has preference over including).
[Test Plan]
0.Preparing the test environment:
#Preparing the container
lxc launch images:ubuntu/focal checkdisks-focal
lxc shell checkdisks-focal
apt update -y
apt upgrade -y
#Installing necessary tools
apt install monitoring-plugins
apt install snapd
#Mount some synthetic systems and make use of snap space
mkdir /reproduce-tmpfs
mkdir /reproduce-squashfs
mkdir /reproduce-tracefs
mount -t tmpfs -o rw,size=2G tmpfs /reproduce-tmpfs
mount -t tmpfs -o rw,size=2G squashfs /reproduce-squashfs
mount -t tmpfs -o rw,size=2G tracefs /reproduce-tracefs
snap install gnome-calculator
1. Bad cases:
# "CRITICAL DISK" noise
$ /usr/lib/nagios/plugins/check_disk -w 10 -c 10
DISK CRITICAL - free space: /dev 448 MB (100% inode=99%); /run 96 MB (98% inode=99%); / 884 MB (18% inode=80%); /dev/shm 485 MB (100% inode=99%); /run/lock 5 MB (100% inode=99%); /snap/lxd/21835 0 MB (0% inode=0%); /snap/core20/1270 0 MB (0% inode=0%); /snap/snapd/14295 0 MB (0% inode=0%); /boot/efi 99 MB (95% inode=-); /run/snapd/ns 96 MB (98% inode=99%); /snap/snapd/14549 0 MB (0% inode=0%); /run/user/1000 97 MB (99% inode=99%); /snap/bare/5 0 MB (0% inode=0%); /snap/multipass-sshfs/145 0 MB (0% inode=0%); /home/ubuntu/monitoring-plugins 1024000 MB (100% inode=100%); /home/ubuntu/reproduce 2048 MB (100% inode=99%); /home/ubuntu/reproduce-squash 2048 MB (100% inode=99%); /snap/core20/1328 0 MB (0% inode=0%); /home/ubuntu/reproduce-tracefs 2048 MB (100% inode=99%);| /dev=0MB;438;438;0;448 /run=0MB;87;87;0;97 /=3886MB;4777;4777;0;4787 /dev/shm=0MB;475;475;0;485 /run/lock=0MB;-5;-5;0;5 /snap/lxd/21835=67MB;57;57;0;67 /snap/core20/1270=62MB;52;52;0;62 /snap/snapd/14295=43MB;33;33;0;43 /boot/efi=5MB;94;94;0;104 /run/snapd/ns=0MB;87;87;0;97 /snap/snapd/14549=43MB;33;33;0;43 /run/user/1000=0MB;87;87;0;97 /snap/bare/5=0MB;-10;-10;0;0 /snap/multipass-sshfs/145=1MB;-9;-9;0;1 /home/ubuntu/monitoring-plugins=0MB;1023990;1023990;0;1024000 /home/ubuntu/reproduce=0MB;2038;2038;0;2048 /home/ubuntu/reproduce-squash=0MB;2038;2038;0;2048 /snap/core20/1328=62MB;52;52;0;62 /home/ubuntu/reproduce-tracefs=0MB;2038;2038;0;2048
$ /usr/lib/nagios/plugins/check_disk -w 10 -c 10 -e -X squashfs
DISK CRITICAL - free space: /run/lock 5 MB (100% inode=99%);| /dev=0MB;438;438;0;448 /run=0MB;87;87;0;97 /=3886MB;4777;4777;0;4787 /dev/shm=0MB;475;475;0;485 /run/lock=0MB;-5;-5;0;5 /boot/efi=5MB;94;94;0;104 /run/snapd/ns=0MB;87;87;0;97 /run/user/1000=0MB;87;87;0;97 /home/ubuntu/monitoring-plugins=0MB;1023990;1023990;0;1024000 /home/ubuntu/reproduce=0MB;2038;2038;0;2048 /home/ubuntu/reproduce-squash=0MB;2038;2038;0;2048 /home/ubuntu/reproduce-tracefs=0MB;2038;2038;0;2048
$ /usr/lib/nagios/plugins/check_disk -w 10 -c 10 -e -X tracefs
DISK CRITICAL - free space: /run/lock 5 MB (100% inode=99%); /snap/lxd/21835 0 MB (0% inode=0%); /snap/core20/1270 0 MB (0% inode=0%); /snap/snapd/14295 0 MB (0% inode=0%); /snap/snapd/14549 0 MB (0% inode=0%); /snap/bare/5 0 MB (0% inode=0%); /snap/multipass-sshfs/145 0 MB (0% inode=0%); /snap/core20/1328 0 MB (0% inode=0%);| /dev=0MB;438;438;0;448 /run=0MB;87;87;0;97 /=3886MB;4777;4777;0;4787 /dev/shm=0MB;475;475;0;485 /run/lock=0MB;-5;-5;0;5 /snap/lxd/21835=67MB;57;57;0;67 /snap/core20/1270=62MB;52;52;0;62 /snap/snapd/14295=43MB;33;33;0;43 /boot/efi=5MB;94;94;0;104 /run/snapd/ns=0MB;87;87;0;97 /snap/snapd/14549=43MB;33;33;0;43 /run/user/1000=0MB;87;87;0;97 /snap/bare/5=0MB;-10;-10;0;0 /snap/multipass-sshfs/145=1MB;-9;-9;0;1 /home/ubuntu/monitoring-plugins=0MB;1023990;1023990;0;1024000 /home/ubuntu/reproduce=0MB;2038;2038;0;2048 /home/ubuntu/reproduce-squash=0MB;2038;2038;0;2048 /snap/core20/1328=62MB;52;52;0;62 /home/ubuntu/reproduce-tracefs=0MB;2038;2038;0;2048
$ /usr/lib/nagios/plugins/check_disk -w 10 -c 10 -e -X tracefs -X squashfs
DISK CRITICAL - free space: /run/lock 5 MB (100% inode=99%);| /dev=0MB;438;438;0;448 /run=0MB;87;87;0;97 /=3886MB;4777;4777;0;4787 /dev/shm=0MB;475;475;0;485 /run/lock=0MB;-5;-5;0;5 /boot/efi=5MB;94;94;0;104 /run/snapd/ns=0MB;87;87;0;97 /run/user/1000=0MB;87;87;0;97 /home/ubuntu/monitoring-plugins=0MB;1023990;1023990;0;1024000 /home/ubuntu/reproduce=0MB;2038;2038;0;2048 /home/ubuntu/reproduce-squash=0MB;2038;2038;0;2048 /home/ubuntu/reproduce-tracefs=0MB;2038;2038;0;2048
$ /usr/lib/nagios/plugins/check_disk -w 10 -c 10 -e -X tmpfs
DISK CRITICAL - free space: /snap/lxd/21835 0 MB (0% inode=0%); /snap/core20/1270 0 MB (0% inode=0%); /snap/snapd/14295 0 MB (0% inode=0%); /snap/snapd/14549 0 MB (0% inode=0%); /snap/bare/5 0 MB (0% inode=0%); /snap/multipass-sshfs/145 0 MB (0% inode=0%); /snap/core20/1328 0 MB (0% inode=0%);| /dev=0MB;438;438;0;448 /=3886MB;4777;4777;0;4787 /snap/lxd/21835=67MB;57;57;0;67 /snap/core20/1270=62MB;52;52;0;62 /snap/snapd/14295=43MB;33;33;0;43 /boot/efi=5MB;94;94;0;104 /snap/snapd/14549=43MB;33;33;0;43 /snap/bare/5=0MB;-10;-10;0;0 /snap/multipass-sshfs/145=1MB;-9;-9;0;1 /home/ubuntu/monitoring-plugins=0MB;1023990;1023990;0;1024000 /snap/core20/1328=62MB;52;52;0;62
# "DISK UNKNOWN" as synthetic filesystems are excluded
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -p /reproduce-tmpfs
DISK UNKNOWN - free space:|
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -N tmpfs
DISK UNKNOWN - free space:|
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -X squashfs -N tmpfs
DISK UNKNOWN - free space:|
2. Good cases:
# "CRITICAL DISK" noise
$ /usr/lib/nagios/plugins/check_disk -w 10 -c 10
DISK OK - free space: / 6971 MB (85% inode=99%); /dev/fuse 32071 MB (100% inode=99%); /dev/net/tun 32071 MB (100% inode=99%); /dev/full 32071 MB (100% inode=99%); /dev/null 32071 MB (100% inode=99%); /dev/random 32071 MB (100% inode=99%); /dev/tty 32071 MB (100% inode=99%); /dev/urandom 32071 MB (100% inode=99%); /dev/zero 32071 MB (100% inode=99%); /snap 6971 MB (85% inode=99%);| /=1204MB;8165;8165;0;8175 /dev/fuse=0MB;32061;32061;0;32071 /dev/net/tun=0MB;32061;32061;0;32071 /dev/full=0MB;32061;32061;0;32071 /dev/null=0MB;32061;32061;0;32071 /dev/random=0MB;32061;32061;0;32071 /dev/tty=0MB;32061;32061;0;32071 /dev/urandom=0MB;32061;32061;0;32071 /dev/zero=0MB;32061;32061;0;32071 /snap=1204MB;8165;8165;0;8175
# "DISK UNKNOWN" as synthetic filesystems are excluded
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -p /reproduce-tmpfs
DISK OK - free space: /reproduce-tmpfs 2048 MB (100% inode=99%);| /reproduce-tmpfs=0MB;1843;1945;0;2048
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -N tmpfs
DISK OK - free space: /dev 0 MB (99% inode=99%); /dev/lxd 0 MB (100% inode=99%); /dev/.lxd-mounts 0 MB (100% inode=99%); /proc/sys/kernel/random/boot_id 0 MB (99% inode=99%); /dev/shm 32106 MB (100% inode=99%); /run 6421 MB (99% inode=99%); /run/lock 5 MB (100% inode=99%); /sys/fs/cgroup 32106 MB (100% inode=99%); /run/user/0 6421 MB (100% inode=99%); /run/snapd/ns 6421 MB (99% inode=99%); /reproduce-tmpfs 2048 MB (100% inode=99%); /reproduce-squashfs 2048 MB (100% inode=99%); /reproduce-tracefs 2048 MB (100% inode=99%);| /dev=0MB;0;0;0;0 /dev/lxd=0MB;0;0;0;0 /dev/.lxd-mounts=0MB;0;0;0;0 /proc/sys/kernel/random/boot_id=0MB;0;0;0;0 /dev/shm=0MB;28895;30500;0;32106 /run=0MB;5778;6099;0;6421 /run/lock=0MB;4;4;0;5 /sys/fs/cgroup=0MB;28895;30500;0;32106 /run/user/0=0MB;5778;6099;0;6421 /run/snapd/ns=0MB;5778;6099;0;6421 /reproduce-tmpfs=0MB;1843;1945;0;2048 /reproduce-squashfs=0MB;1843;1945;0;2048 /reproduce-tracefs=0MB;1843;1945;0;2048
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -N tmpfs -p /reproduce-tmpfs
DISK OK - free space: /reproduce-tmpfs 2048 MB (100% inode=99%);| /reproduce-tmpfs=0MB;1843;1945;0;2048
//This case is fine, as I am explicitly asking for an fs that is being explicitly excluded too.
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -X tmpfs -p /reproduce-tmpfs
DISK UNKNOWN - free space:|
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -X squashfs -p /reproduce-tmpfs
DISK OK - free space: /reproduce-tmpfs 2048 MB (100% inode=99%);| /reproduce-tmpfs=0MB;1843;1945;0;2048
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -X squashfs -N tmpfs
DISK OK - free space: /dev 0 MB (99% inode=99%); /dev/lxd 0 MB (100% inode=99%); /dev/.lxd-mounts 0 MB (100% inode=99%); /proc/sys/kernel/random/boot_id 0 MB (99% inode=99%); /dev/shm 32106 MB (100% inode=99%); /run 6421 MB (99% inode=99%); /run/lock 5 MB (100% inode=99%); /sys/fs/cgroup 32106 MB (100% inode=99%); /run/user/0 6421 MB (100% inode=99%); /run/snapd/ns 6421 MB (99% inode=99%); /reproduce-tmpfs 2048 MB (100% inode=99%); /reproduce-squashfs 2048 MB (100% inode=99%); /reproduce-tracefs 2048 MB (100% inode=99%);| /dev=0MB;0;0;0;0 /dev/lxd=0MB;0;0;0;0 /dev/.lxd-mounts=0MB;0;0;0;0 /proc/sys/kernel/random/boot_id=0MB;0;0;0;0 /dev/shm=0MB;28895;30500;0;32106 /run=0MB;5778;6099;0;6421 /run/lock=0MB;4;4;0;5 /sys/fs/cgroup=0MB;28895;30500;0;32106 /run/user/0=0MB;5778;6099;0;6421 /run/snapd/ns=0MB;5778;6099;0;6421 /reproduce-tmpfs=0MB;1843;1945;0;2048 /reproduce-squashfs=0MB;1843;1945;0;2048 /reproduce-tracefs=0MB;1843;1945;0;2048
[Where problems could occur]
As this alters the logic of how out-of-space checks are handled, relevant issues to keep an eye out for would relate to filesystem checks reporting improperly. These tools underlay a few different front-ends, so regression bugs may get filed in a few different places, however they will tend to display error messages involving check_disk, nagios, and either tmpfs or tracefs. To minimize it, this changes has been done after all the arguments logic has been done (thanks Richard Laager), therefore excluding filesystems only if not required explicitly by the user (thanks Christian Ehrhardt).
Note that there are likely other synthetic filesystems beyond tmpfs, squashfs and tracefs (e.g. udev, usbfs, devtmpfs, fuse.*, ...) which might also cause similar false positives; these should be handled as separate bugs, although the proposed code here has the intention to be easily used to it.
[Other Info]
The intention is to forward the patch to upstream once is accepted here, as the previous one (https://github.com/monitoring-plugins/monitoring-plugins/pull/1609 and https://github.com/monitoring-plugins/monitoring-plugins/issues/1608) was rejected. Update: Issue created to upstream at https://github.com/monitoring-plugins/monitoring-plugins/issues/1748 with PR at https://github.com/monitoring-plugins/monitoring-plugins/pull/1749 .
[Original Report]
-------------------------------------------------------------------
$ lsb_release -d
Description: Ubuntu 20.04.3 LTS
$ apt-cache policy monitoring-plugins | head -n2
monitoring-plugins:
Installed: 2.2-6ubuntu1.1
check_disk returns UNKNOWN for tmpfs filesystems, even when directed explicitly to check them. This is a regression introduced by the fix for LP: #1827159.
Steps to reproduce:
$ df -h /tmp
Filesystem Size Used Avail Use% Mounted on
tmpfs 16G 64K 16G 1% /tmp
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -p /tmp
DISK UNKNOWN - free space:|
Running the check as part of a group, even with just the one member, is a workaround:
$ /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -g tmp -p /tmp
DISK OK - free space: tmp 16058 MB (99% inode=99%);| tmp=0MB;14452;15255;0;16058 |
|
| 2022-02-01 18:13:29 |
Brian Murray |
nominated for series |
|
Ubuntu Focal |
|
| 2022-02-01 18:13:29 |
Brian Murray |
bug task added |
|
monitoring-plugins (Ubuntu Focal) |
|
| 2022-02-01 18:13:35 |
Brian Murray |
monitoring-plugins (Ubuntu Focal): status |
New |
Incomplete |
|
| 2022-02-01 18:13:39 |
Brian Murray |
bug |
|
|
added subscriber Brian Murray |
| 2022-02-02 10:16:50 |
Miriam España Acebal |
monitoring-plugins (Ubuntu Focal): assignee |
|
Miriam España Acebal (mirespace) |
|
| 2022-02-02 10:16:55 |
Miriam España Acebal |
monitoring-plugins (Ubuntu Focal): status |
Incomplete |
In Progress |
|
| 2022-02-02 10:17:17 |
Miriam España Acebal |
monitoring-plugins (Ubuntu Focal): importance |
Undecided |
Medium |
|
| 2022-02-02 11:26:49 |
Miriam España Acebal |
nominated for series |
|
Ubuntu Impish |
|
| 2022-02-02 11:26:49 |
Miriam España Acebal |
bug task added |
|
monitoring-plugins (Ubuntu Impish) |
|
| 2022-02-02 11:26:54 |
Miriam España Acebal |
monitoring-plugins (Ubuntu Impish): assignee |
|
Miriam España Acebal (mirespace) |
|
| 2022-02-02 11:26:57 |
Miriam España Acebal |
monitoring-plugins (Ubuntu Impish): importance |
Undecided |
Medium |
|
| 2022-02-02 11:27:01 |
Miriam España Acebal |
monitoring-plugins (Ubuntu Impish): status |
New |
In Progress |
|
| 2022-02-02 12:16:54 |
Miriam España Acebal |
nominated for series |
|
Ubuntu Jammy |
|
| 2022-02-02 12:16:54 |
Miriam España Acebal |
bug task added |
|
monitoring-plugins (Ubuntu Jammy) |
|
| 2022-02-02 12:27:34 |
Launchpad Janitor |
merge proposal linked |
|
https://code.launchpad.net/~mirespace/ubuntu/+source/monitoring-plugins/+git/monitoring-plugins/+merge/414954 |
|
| 2022-02-02 12:53:10 |
Launchpad Janitor |
merge proposal linked |
|
https://code.launchpad.net/~mirespace/ubuntu/+source/monitoring-plugins/+git/monitoring-plugins/+merge/414956 |
|
| 2022-02-04 04:02:15 |
Launchpad Janitor |
monitoring-plugins (Ubuntu Jammy): status |
In Progress |
Fix Released |
|
| 2022-02-08 22:41:33 |
Brian Murray |
monitoring-plugins (Ubuntu Impish): status |
In Progress |
Fix Committed |
|
| 2022-02-08 22:41:35 |
Brian Murray |
bug |
|
|
added subscriber Ubuntu Stable Release Updates Team |
| 2022-02-08 22:41:37 |
Brian Murray |
bug |
|
|
added subscriber SRU Verification |
| 2022-02-08 22:41:41 |
Brian Murray |
tags |
server-todo |
server-todo verification-needed verification-needed-impish |
|
| 2022-02-08 22:44:04 |
Brian Murray |
monitoring-plugins (Ubuntu Focal): status |
In Progress |
Fix Committed |
|
| 2022-02-08 22:44:11 |
Brian Murray |
tags |
server-todo verification-needed verification-needed-impish |
server-todo verification-needed verification-needed-focal verification-needed-impish |
|
| 2022-02-09 08:51:17 |
Miriam España Acebal |
bug |
|
|
added subscriber Richard Laager |
| 2022-02-10 05:46:24 |
Richard Laager |
tags |
server-todo verification-needed verification-needed-focal verification-needed-impish |
server-todo verification-done-focal verification-needed verification-needed-impish |
|
| 2022-02-10 12:54:37 |
Miriam España Acebal |
tags |
server-todo verification-done-focal verification-needed verification-needed-impish |
server-todo verification-done verification-done-focal verification-done-impish |
|
| 2022-02-16 12:35:43 |
Launchpad Janitor |
monitoring-plugins (Ubuntu Impish): status |
Fix Committed |
Fix Released |
|
| 2022-02-16 12:35:47 |
Robie Basak |
removed subscriber Ubuntu Stable Release Updates Team |
|
|
|
| 2022-02-16 12:35:57 |
Launchpad Janitor |
monitoring-plugins (Ubuntu Focal): status |
Fix Committed |
Fix Released |
|
| 2022-02-16 15:26:48 |
Miriam España Acebal |
tags |
server-todo verification-done verification-done-focal verification-done-impish |
verification-done verification-done-focal verification-done-impish |
|
| 2022-07-06 08:57:15 |
Hua Zhang |
bug |
|
|
added subscriber Hua Zhang |
| 2022-07-06 09:03:11 |
Dariusz Gadomski |
bug |
|
|
added subscriber Dariusz Gadomski |
| 2022-07-06 09:03:15 |
Dariusz Gadomski |
nominated for series |
|
Ubuntu Bionic |
|
| 2022-07-06 09:03:15 |
Dariusz Gadomski |
bug task added |
|
monitoring-plugins (Ubuntu Bionic) |
|
| 2022-07-06 09:03:36 |
Dariusz Gadomski |
monitoring-plugins (Ubuntu Bionic): assignee |
|
Hua Zhang (zhhuabj) |
|
| 2022-07-06 09:29:24 |
Dariusz Gadomski |
monitoring-plugins (Ubuntu Bionic): importance |
Undecided |
Medium |
|
| 2022-07-06 11:56:35 |
Dariusz Gadomski |
tags |
verification-done verification-done-focal verification-done-impish |
sts sts-sponsor-dgadomski verification-done verification-done-focal verification-done-impish |
|
| 2022-07-09 18:16:13 |
Patrick Olson |
bug |
|
|
added subscriber Patrick Olson |
| 2022-07-13 11:07:15 |
Robie Basak |
monitoring-plugins (Ubuntu Bionic): status |
New |
Fix Committed |
|
| 2022-07-13 11:07:17 |
Robie Basak |
bug |
|
|
added subscriber Ubuntu Stable Release Updates Team |
| 2022-07-13 11:07:21 |
Robie Basak |
tags |
sts sts-sponsor-dgadomski verification-done verification-done-focal verification-done-impish |
sts sts-sponsor-dgadomski verification-done-focal verification-done-impish verification-needed verification-needed-bionic |
|
| 2022-07-14 05:43:06 |
Hua Zhang |
tags |
sts sts-sponsor-dgadomski verification-done-focal verification-done-impish verification-needed verification-needed-bionic |
sts sts-sponsor-dgadomski verification-done-bionic verification-done-focal verification-done-impish verification-needed |
|
| 2022-07-27 00:21:18 |
Launchpad Janitor |
monitoring-plugins (Ubuntu Bionic): status |
Fix Committed |
Fix Released |
|
