Volume group "cinder-volumes" not found - Cannot process volume group cinder-volumes

[Aurelien Lourot]

Might be a re-occurrence of lp:1480504.

Happened here [0] while a Zaza test tried to backup a cinder volume. cinder-volume.log show a lot of:

2020-05-13 00:00:40.339 36017 ERROR cinder.volume.manager Traceback (most recent call last):
2020-05-13 00:00:40.339 36017 ERROR cinder.volume.manager File "/usr/lib/python3/dist-packages/cinder/volume/manager.py", line 467, in _init_host
2020-05-13 00:00:40.339 36017 ERROR cinder.volume.manager self.driver.check_for_setup_error()
2020-05-13 00:00:40.339 36017 ERROR cinder.volume.manager File "/usr/lib/python3/dist-packages/cinder/volume/drivers/lvm.py", line 312, in check_for_setup_error
2020-05-13 00:00:40.339 36017 ERROR cinder.volume.manager self.vg = lvm.LVM(
2020-05-13 00:00:40.339 36017 ERROR cinder.volume.manager File "/usr/lib/python3/dist-packages/cinder/brick/local_dev/lvm.py", line 107, in __init__
2020-05-13 00:00:40.339 36017 ERROR cinder.volume.manager if self._vg_exists() is False:
2020-05-13 00:00:40.339 36017 ERROR cinder.volume.manager File "/usr/lib/python3/dist-packages/cinder/brick/local_dev/lvm.py", line 138, in _vg_exists
2020-05-13 00:00:40.339 36017 ERROR cinder.volume.manager (out, _err) = self._execute(*cmd,
2020-05-13 00:00:40.339 36017 ERROR cinder.volume.manager File "/usr/lib/python3/dist-packages/os_brick/executor.py", line 52, in _execute
2020-05-13 00:00:40.339 36017 ERROR cinder.volume.manager result = self.__execute(*args, **kwargs)
2020-05-13 00:00:40.339 36017 ERROR cinder.volume.manager File "/usr/lib/python3/dist-packages/cinder/utils.py", line 126, in execute
2020-05-13 00:00:40.339 36017 ERROR cinder.volume.manager return processutils.execute(*cmd, **kwargs)
2020-05-13 00:00:40.339 36017 ERROR cinder.volume.manager File "/usr/lib/python3/dist-packages/oslo_concurrency/processutils.py", line 421, in execute
2020-05-13 00:00:40.339 36017 ERROR cinder.volume.manager raise ProcessExecutionError(exit_code=_returncode,
2020-05-13 00:00:40.339 36017 ERROR cinder.volume.manager oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.
2020-05-13 00:00:40.339 36017 ERROR cinder.volume.manager Command: sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C vgs --noheadings -o name cinder-volumes
2020-05-13 00:00:40.339 36017 ERROR cinder.volume.manager Exit code: 5
2020-05-13 00:00:40.339 36017 ERROR cinder.volume.manager Stdout: ''
2020-05-13 00:00:40.339 36017 ERROR cinder.volume.manager Stderr: ' Volume group "cinder-volumes" not found\n Cannot process volume group cinder-volumes\n'
2020-05-13 00:00:40.339 36017 ERROR cinder.volume.manager
2020-05-13 00:00:41.348 36017 INFO cinder.volume.manager [req-e092d3d1-fb54-4c29-b3cc-c7d104217569 - - - - -] Starting volume driver LVMVolumeDriver (3.0.0)
2020-05-13 00:00:42.026 36017 ERROR cinder.volume.manager [req-e092d3d1-fb54-4c29-b3cc-c7d104217569 - - - - -] Failed to initialize driver.: oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running com
mand.
Command: sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C vgs --noheadings -o name cinder-volumes
Exit code: 5
Stdout: ''
Stderr: ' Volume group "cinder-volumes" not found\n Cannot process volume group cinder-volumes\n'

For the record, for this particular occurrence, the hypervisor where cinder was running was 'cuvier'.

This might be a spurious issue that has existed for years [1][2][3]

0: https://openstack-ci-reports.ubuntu.com/artifacts/test_charm_pipeline_func_full/openstack/charm-cinder-backup-swift-proxy/726798/1/5674/index.html
1: https://bugs.launchpad.net/openstack-ansible/+bug/1698831
2: https://bugzilla.redhat.com/show_bug.cgi?id=1412661
3: https://stackoverflow.com/questions/15791931/error-when-creating-volume-with-openstack-dashboard

[Peter Matulis (petermatulis)]

Maybe related: bug 1862392.

[James Page (james-page)]

When cinder is installed from packaging it default to using the LVM driver and will attempt to automatically startup - which is kinda unfortunate as this then generates a error messages in the log.

I dug into the linked test crashdump - I don't see anything materially broken once configuration is complete and ceph is being used by the cinder-volume daemon.

Marking Invalid.