Comment 8 for bug 1901783

Revision history for this message
Lee Yarwood (lyarwood) wrote :

FWIW we are also seeing this in the nova-lvm job where n-cpu calls lvs via privsep:

https://a59f59e90e7507f14599-2cecd17ff9ea5f4d02c12f6d5fc5aedd.ssl.cf1.rackcdn.com/776681/7/check/nova-lvm/a19ff22/controller/logs/screen-n-cpu.txt

Mar 09 16:45:48.496630 ubuntu-focal-rax-ord-0023393373 nova-compute[91725]: ERROR oslo_messaging.rpc.server File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 1525, in _cleanup
Mar 09 16:45:48.496630 ubuntu-focal-rax-ord-0023393373 nova-compute[91725]: ERROR oslo_messaging.rpc.server self._cleanup_lvm(instance, block_device_info)
Mar 09 16:45:48.496630 ubuntu-focal-rax-ord-0023393373 nova-compute[91725]: ERROR oslo_messaging.rpc.server File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 1615, in _cleanup_lvm
Mar 09 16:45:48.496630 ubuntu-focal-rax-ord-0023393373 nova-compute[91725]: ERROR oslo_messaging.rpc.server disks = self._lvm_disks(instance)
Mar 09 16:45:48.496630 ubuntu-focal-rax-ord-0023393373 nova-compute[91725]: ERROR oslo_messaging.rpc.server File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 1633, in _lvm_disks
Mar 09 16:45:48.496630 ubuntu-focal-rax-ord-0023393373 nova-compute[91725]: ERROR oslo_messaging.rpc.server logical_volumes = lvm.list_volumes(vg)
Mar 09 16:45:48.496630 ubuntu-focal-rax-ord-0023393373 nova-compute[91725]: ERROR oslo_messaging.rpc.server File "/opt/stack/nova/nova/virt/libvirt/storage/lvm.py", line 111, in list_volumes
Mar 09 16:45:48.496630 ubuntu-focal-rax-ord-0023393373 nova-compute[91725]: ERROR oslo_messaging.rpc.server out, err = nova.privsep.fs.lvlist(vg)
Mar 09 16:45:48.496630 ubuntu-focal-rax-ord-0023393373 nova-compute[91725]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 247, in _wrap
Mar 09 16:45:48.496630 ubuntu-focal-rax-ord-0023393373 nova-compute[91725]: ERROR oslo_messaging.rpc.server return self.channel.remote_call(name, args, kwargs)
Mar 09 16:45:48.496630 ubuntu-focal-rax-ord-0023393373 nova-compute[91725]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 224, in remote_call
Mar 09 16:45:48.496630 ubuntu-focal-rax-ord-0023393373 nova-compute[91725]: ERROR oslo_messaging.rpc.server raise exc_type(*result[2])
Mar 09 16:45:48.496630 ubuntu-focal-rax-ord-0023393373 nova-compute[91725]: ERROR oslo_messaging.rpc.server oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.
Mar 09 16:45:48.496630 ubuntu-focal-rax-ord-0023393373 nova-compute[91725]: ERROR oslo_messaging.rpc.server Command: lvs --noheadings -o lv_name /dev/stack-volumes-default
Mar 09 16:45:48.496630 ubuntu-focal-rax-ord-0023393373 nova-compute[91725]: ERROR oslo_messaging.rpc.server Exit code: -11
Mar 09 16:45:48.496630 ubuntu-focal-rax-ord-0023393373 nova-compute[91725]: ERROR oslo_messaging.rpc.server Stdout: ''
Mar 09 16:45:48.496630 ubuntu-focal-rax-ord-0023393373 nova-compute[91725]: ERROR oslo_messaging.rpc.server Stderr: ' WARNING: Failed to get udev device handler for device /dev/sda1.\n /dev/sda15: stat failed: No such file or directory\n Path /dev/sda15 no longer valid for device(8,15)\n /dev/sda15: stat failed: No such file or directory\n Path /dev/sda15 no longer valid for device(8,15)\n Device open /dev/sda 8:0 failed errno 2\n Device open /dev/sda 8:0 failed errno 2\n Device open /dev/sda1 8:1 failed errno 2\n Device open /dev/sda1 8:1 failed errno 2\n WARNING: Scan ignoring device 8:0 with no paths.\n WARNING: Scan ignoring device 8:1 with no paths.\n'