initiator/connectors/nvmeof.py _get_host_uuid uses `findmnt / -n -o SOURCE` to get the path of the block device to use with `blkid`. For the server I'm working on we have btrfs snapshots with snapper and that makes this not work.
I'm using openstack 2023.1 installed with devstack on Ubuntu Server 2022.04.
$ findmnt / -n -o SOURCE
/dev/sda2[/.snapshots/3/snapshot]
Adding the "-v" flag might help?
$ findmnt / -n -o SOURCE -v
/dev/sda2
I moved the findmnt binary to findmnt.bak and made this shell script:
$ cat /usr/bin/findmnt
#!/bin/bash
/usr/bin/findmnt.bak $@ -v
And now the python backtrace below doesn't happen when I try to live migrate a VM to this server:
DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=52748) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.009s {{(pid=52748) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
DEBUG oslo.privsep.daemon [-] privsep: reply[e2abaa37-8e82-46e4-83ea-da43dfb24608]: (4, ('/dev/sdc2[/.snapshots/3/snapshot]\n', '')) {{(pid=52748) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sdc2[/.snapshots/3/snapshot] -s UUID -o value {{(pid=52748) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sdc2[/.snapshots/3/snapshot] -s UUID -o value" returned: 2 in 0.007s {{(pid=52748) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
DEBUG oslo_concurrency.processutils [-] 'blkid /dev/sdc2[/.snapshots/3/snapshot] -s UUID -o value' failed. Not Retrying. {{(pid=52748) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
DEBUG oslo.privsep.daemon [-] privsep: Exception during request[a7caad1d-6c58-4372-a947-6e19bcaa109c]: Unexpected error while running command.
Command: blkid /dev/sdc2[/.snapshots/3/snapshot] -s UUID -o value
Exit code: 2
Stdout: ''
Stderr: '' {{(pid=52748) _process_cmd /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:479}}
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py", line 476, in _process_cmd
ret = func(*f_args, **f_kwargs)
File "/usr/local/lib/python3.10/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
return func(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
return putils.execute(on_execute=on_execute,
File "/usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py", line 438, in execute
raise ProcessExecutionError(exit_code=_returncode,
oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.
Command: blkid /dev/sdc2[/.snapshots/3/snapshot] -s UUID -o value
Exit code: 2
Stdout: ''
Stderr: ''
I've started using the following sed command on each server for now as a simpler fix:
sudo sed -i "s|'findmnt', '/', '-n', '-o', 'SOURCE'|'findmnt', '/', '-n', '-o', 'SOURCE', '-v'|" /usr/local/
This means I don't need to replace the findmnt binary anymore.