LVM/iSCSI c-vol backend lvs call during delete volume fails with exit code 139 WARNING: Scan ignoring device 8:0 with no paths

Bug #1918261 reported by Lee Yarwood
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Cinder
New
Undecided
Unassigned

Bug Description

https://zuul.opendev.org/t/openstack/build/e3f8ea65dedb43d88bbc8d35f71cc887/log/controller/logs/screen-c-vol.txt#4469

Mar 08 18:51:20.296852 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: DEBUG oslo_concurrency.processutils [None req-a1184e68-d9d9-4acc-8125-6e01d4bc86cb tempest-ServerActionsTestJSON-1795437865 None] 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C lvs --noheadings --unit=g -o vg_name,name,size --nosuffix --readonly stack-volumes-lvmdriver-1/volume-8ddcc170-826c-45e0-b149-b63652912e8e' failed. Not Retrying. {{(pid=96013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Mar 08 18:51:20.322747 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: DEBUG cinder.coordination [None req-a1184e68-d9d9-4acc-8125-6e01d4bc86cb tempest-ServerActionsTestJSON-1795437865 None] Lock "b'cinder-8ddcc170-826c-45e0-b149-b63652912e8e-delete_volume'" released by "delete_volume" :: held 0.609s {{(pid=96013) _synchronized /opt/stack/cinder/cinder/coordination.py:158}}
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server [None req-a1184e68-d9d9-4acc-8125-6e01d4bc86cb tempest-ServerActionsTestJSON-1795437865 None] Exception during message handling: oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: Command: sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C lvs --noheadings --unit=g -o vg_name,name,size --nosuffix --readonly stack-volumes-lvmdriver-1/volume-8ddcc170-826c-45e0-b149-b63652912e8e
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: Exit code: 139
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: Stdout: ''
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: Stderr: ' Device open /dev/sda 8:0 failed errno 2\n Device open /dev/sda 8:0 failed errno 2\n WARNING: Scan ignoring device 8:0 with no paths.\n'
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.8/dist-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.8/dist-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.8/dist-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server File "<decorator-gen-754>", line 2, in delete_volume
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/coordination.py", line 151, in _synchronized
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server return f(*a, **k)
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server File "<decorator-gen-753>", line 2, in delete_volume
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/objects/cleanable.py", line 208, in wrapper
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server result = f(*args, **kwargs)
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/manager.py", line 939, in delete_volume
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server self._clear_db(is_migrating_dest, volume, new_status)
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server self.force_reraise()
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server raise self.value
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/manager.py", line 923, in delete_volume
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server self.driver.delete_volume(volume)
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/drivers/lvm.py", line 460, in delete_volume
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server if self._volume_not_present(volume['name']):
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/drivers/lvm.py", line 139, in _volume_not_present
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server return self.vg.get_volume(volume_name) is None
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/brick/local_dev/lvm.py", line 345, in get_volume
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server ref_list = self.get_volumes(name)
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/brick/local_dev/lvm.py", line 335, in get_volumes
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server return self.get_lv_info(self._root_helper,
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/brick/local_dev/lvm.py", line 318, in get_lv_info
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server out = None
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server self.force_reraise()
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server raise self.value
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/brick/local_dev/lvm.py", line 308, in get_lv_info
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server (out, _err) = putils.execute(*cmd,
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 438, in execute
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server raise ProcessExecutionError(exit_code=_returncode,
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server Command: sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C lvs --noheadings --unit=g -o vg_name,name,size --nosuffix --readonly stack-volumes-lvmdriver-1/volume-8ddcc170-826c-45e0-b149-b63652912e8e
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server Exit code: 139
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server Stdout: ''
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server Stderr: ' Device open /dev/sda 8:0 failed errno 2\n Device open /dev/sda 8:0 failed errno 2\n WARNING: Scan ignoring device 8:0 with no paths.\n'
Mar 08 18:51:20.342827 ubuntu-focal-rax-iad-0023375011 cinder-volume[96013]: ERROR oslo_messaging.rpc.server

Tags: gate-failure
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.