Openstack Details:
Base OS - Ubuntu 201.04 LTS
HW - 3 Controller and multi computes
Openstack Version - Yoga(Kolla-Ansible)
Cinder Backend: Powerstore 5000T using ISCSI
Hello, I have this error when I resize or migrate instances
2024-04-12 09:49:35.284 7 WARNING os_brick.exception [req-ea4c2c71-d1f1-45ab-82b5-9b384bc93101 5875e342bc6c39052c9b291df7da1a4f4960cb5578defbadb0b432f5b5dbd087 a14e68c02f6344cb8acdd279dce5aab1 - default default] Flushing 36000d3100578ec0000000000000005ba failed: oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.
2024-04-12 09:49:35.309 7 ERROR nova.compute.manager [req-ea4c2c71-d1f1-45ab-82b5-9b384bc93101 5875e342bc6c39052c9b291df7da1a4f4960cb5578defbadb0b432f5b5dbd087 a14e68c02f6344cb8acdd279dce5aab1 - default default] [instance: 17a71500-ebb8-42cf-82b8-0d380929d380] Setting instance vm_state to ERROR: oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.
Command: multipath -f 36000d3100578ec0000000000000005ba
Exit code: 1
Stdout: ''
Stderr: 'Apr 12 09:48:55 | 36000d3100578ec0000000000000005ba: map in use\n'
2024-04-12 09:49:35.309 7 ERROR nova.compute.manager [instance: 17a71500-ebb8-42cf-82b8-0d380929d380] Traceback (most recent call last):
2024-04-12 09:49:35.309 7 ERROR nova.compute.manager [instance: 17a71500-ebb8-42cf-82b8-0d380929d380] File "/var/lib/kolla/venv/lib/python3.8/site-packages/nova/compute/manager.py", line 10415, in _error_out_instance_on_exception
2024-04-12 09:49:35.309 7 ERROR nova.compute.manager [instance: 17a71500-ebb8-42cf-82b8-0d380929d380] yield
2024-04-12 09:49:35.309 7 ERROR nova.compute.manager [instance: 17a71500-ebb8-42cf-82b8-0d380929d380] File "/var/lib/kolla/venv/lib/python3.8/site-packages/nova/compute/manager.py", line 5712, in _resize_instance
2024-04-12 09:49:35.309 7 ERROR nova.compute.manager [instance: 17a71500-ebb8-42cf-82b8-0d380929d380] disk_info = self.driver.migrate_disk_and_power_off(
2024-04-12 09:49:35.309 7 ERROR nova.compute.manager [instance: 17a71500-ebb8-42cf-82b8-0d380929d380] File "/var/lib/kolla/venv/lib/python3.8/site-packages/nova/virt/libvirt/driver.py", line 11252, in migrate_disk_and_power_off
2024-04-12 09:49:35.309 7 ERROR nova.compute.manager [instance: 17a71500-ebb8-42cf-82b8-0d380929d380] self._disconnect_volume(context, connection_info, instance)
2024-04-12 09:49:35.309 7 ERROR nova.compute.manager [instance: 17a71500-ebb8-42cf-82b8-0d380929d380] File "/var/lib/kolla/venv/lib/python3.8/site-packages/nova/virt/libvirt/driver.py", line 1971, in _disconnect_volume
2024-04-12 09:49:35.309 7 ERROR nova.compute.manager [instance: 17a71500-ebb8-42cf-82b8-0d380929d380] vol_driver.disconnect_volume(
2024-04-12 09:49:35.309 7 ERROR nova.compute.manager [instance: 17a71500-ebb8-42cf-82b8-0d380929d380] File "/var/lib/kolla/venv/lib/python3.8/site-packages/nova/virt/libvirt/volume/iscsi.py", line 74, in disconnect_volume
2024-04-12 09:49:35.309 7 ERROR nova.compute.manager [instance: 17a71500-ebb8-42cf-82b8-0d380929d380] self.connector.disconnect_volume(
2024-04-12 09:49:35.309 7 ERROR nova.compute.manager [instance: 17a71500-ebb8-42cf-82b8-0d380929d380] File "/var/lib/kolla/venv/lib/python3.8/site-packages/os_brick/utils.py", line 169, in trace_logging_wrapper
2024-04-12 09:49:35.309 7 ERROR nova.compute.manager [instance: 17a71500-ebb8-42cf-82b8-0d380929d380] return f(*args, **kwargs)
2024-04-12 09:49:35.309 7 ERROR nova.compute.manager [instance: 17a71500-ebb8-42cf-82b8-0d380929d380] File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_concurrency/lockutils.py", line 391, in inner
2024-04-12 09:49:35.309 7 ERROR nova.compute.manager [instance: 17a71500-ebb8-42cf-82b8-0d380929d380] return f(*args, **kwargs)
2024-04-12 09:49:35.309 7 ERROR nova.compute.manager [instance: 17a71500-ebb8-42cf-82b8-0d380929d380] File "/var/lib/kolla/venv/lib/python3.8/site-packages/os_brick/utils.py", line 382, in change_encrypted2024-04-12 09:49:35.309 7 ERROR nova.compute.manager [instance: 17a71500-ebb8-42cf-82b8-0d380929d380] res = func(**call_args)
2024-04-12 09:49:35.309 7 ERROR nova.compute.manager [instance: 17a71500-ebb8-42cf-82b8-0d380929d380] File "/var/lib/kolla/venv/lib/python3.8/site-packages/os_brick/initiator/connectors/iscsi.py", line 855, in disconnect_volume
2024-04-12 09:49:35.309 7 ERROR nova.compute.manager [instance: 17a71500-ebb8-42cf-82b8-0d380929d380] return self._cleanup_connection(connection_properties, force=force,
2024-04-12 09:49:35.309 7 ERROR nova.compute.manager [instance: 17a71500-ebb8-42cf-82b8-0d380929d380] File "/var/lib/kolla/venv/lib/python3.8/site-packages/os_brick/initiator/connectors/iscsi.py", line 904, in _cleanup_connection
2024-04-12 09:49:35.309 7 ERROR nova.compute.manager [instance: 17a71500-ebb8-42cf-82b8-0d380929d380] multipath_name = self._linuxscsi.remove_connection(
2024-04-12 09:49:35.309 7 ERROR nova.compute.manager [instance: 17a71500-ebb8-42cf-82b8-0d380929d380] File "/var/lib/kolla/venv/lib/python3.8/site-packages/os_brick/initiator/linuxscsi.py", line 310, in remove_connection
2024-04-12 09:49:35.309 7 ERROR nova.compute.manager [instance: 17a71500-ebb8-42cf-82b8-0d380929d380] self.flush_multipath_device(multipath_name)
2024-04-12 09:49:35.309 7 ERROR nova.compute.manager [instance: 17a71500-ebb8-42cf-82b8-0d380929d380] File "/var/lib/kolla/venv/lib/python3.8/site-packages/os_brick/initiator/linuxscsi.py", line 378, in flush_multipath_device
2024-04-12 09:49:35.309 7 ERROR nova.compute.manager [instance: 17a71500-ebb8-42cf-82b8-0d380929d380] self._execute('multipath', '-f', device_map_name, run_as_root=True,
2024-04-12 09:49:35.309 7 ERROR nova.compute.manager [instance: 17a71500-ebb8-42cf-82b8-0d380929d380] File "/var/lib/kolla/venv/lib/python3.8/site-packages/os_brick/executor.py", line 53, in _execute
2024-04-12 09:49:35.309 7 ERROR nova.compute.manager [instance: 17a71500-ebb8-42cf-82b8-0d380929d380] result = self.__execute(*args, **kwargs)
2024-04-12 09:49:35.309 7 ERROR nova.compute.manager [instance: 17a71500-ebb8-42cf-82b8-0d380929d380] File "/var/lib/kolla/venv/lib/python3.8/site-packages/os_brick/privileged/rootwrap.py", line 172, in execute
2024-04-12 09:49:35.309 7 ERROR nova.compute.manager [instance: 17a71500-ebb8-42cf-82b8-0d380929d380] return execute_root(*cmd, **kwargs)
2024-04-12 09:49:35.309 7 ERROR nova.compute.manager [instance: 17a71500-ebb8-42cf-82b8-0d380929d380] File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_privsep/priv_context.py", line 271, in _wrap2024-04-12 09:49:35.309 7 ERROR nova.compute.manager [instance: 17a71500-ebb8-42cf-82b8-0d380929d380] return self.channel.remote_call(name, args, kwargs,
2024-04-12 09:49:35.309 7 ERROR nova.compute.manager [instance: 17a71500-ebb8-42cf-82b8-0d380929d380] File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_privsep/daemon.py", line 215, in remote_call2024-04-12 09:49:35.309 7 ERROR nova.compute.manager [instance: 17a71500-ebb8-42cf-82b8-0d380929d380] raise exc_type(*result[2])
2024-04-12 09:49:35.309 7 ERROR nova.compute.manager [instance: 17a71500-ebb8-42cf-82b8-0d380929d380] oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.
2024-04-12 09:49:35.309 7 ERROR nova.compute.manager [instance: 17a71500-ebb8-42cf-82b8-0d380929d380] Command: multipath -f 36000d3100578ec0000000000000005ba
2024-04-12 09:49:35.309 7 ERROR nova.compute.manager [instance: 17a71500-ebb8-42cf-82b8-0d380929d380] Exit code: 1
2024-04-12 09:49:35.309 7 ERROR nova.compute.manager [instance: 17a71500-ebb8-42cf-82b8-0d380929d380] Stdout: ''
2024-04-12 09:49:35.309 7 ERROR nova.compute.manager [instance: 17a71500-ebb8-42cf-82b8-0d380929d380] Stderr: 'Apr 12 09:48:55 | 36000d3100578ec0000000000000005ba: map in use\n'
2024-04-12 09:49:35.309 7 ERROR nova.compute.manager [instance: 17a71500-ebb8-42cf-82b8-0d380929d380]
