Cinder

  1. Bug #2035404
  2. Activity log

Activity log for bug #2035404

Date Who What changed Old value New value Message
2023-09-13 22:03:13 Simon Dodsley bug added bug
2023-09-13 22:03:44 Simon Dodsley summary [Pure Storage] Replication Failover with sync-repl cvolumes faiure [Pure Storage] Missing replication pod can cause driver failure on restart
2023-09-13 22:03:49 Simon Dodsley cinder: assignee Simon Dodsley (simon-dodsley)
2023-09-13 23:00:49 Simon Dodsley description If synchromous replication is configured, under certain circumstances, if the replication pod defined by the parameter `pure_replication_pod_name` does not exist on the backend at driver restart. Forexample, if someone accidentally deletes the pod, this will cause the restart of the driver to fail. The driver should be capable of correctly recreating the pod. Error traceback example: Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager Traceback (most recent call last): Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager File "/opt/stack/cinder/cinder/volume/drivers/pure.py", line 2864, in _get_current_array Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager pod_info = target_array.get_pod(self._replication_pod_name) Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager File "/opt/stack/.local/lib/python3.10/site-packages/purestorage/purestorage.py", line 3291, in get_pod Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager return self._request("GET", "pod/{0}".format(pod), kwargs) Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager File "/opt/stack/cinder/cinder/volume/drivers/pure.py", line 1884, in wrapper Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager ret = fn(*args, **kwargs) Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager File "/opt/stack/.local/lib/python3.10/site-packages/purestorage/purestorage.py", line 202, in _request Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager raise PureHTTPError(self._target, str(self._rest_version), response) Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager purestorage.purestorage.PureHTTPError: PureHTTPError status code 400 returned by REST version 1.19 at 10.21.228.28: BAD REQUEST Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager [{"msg": "Pod does not exist.", "ctx": "simon-cinder"}] Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager During handling of the above exception, another exception occurred: Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager Traceback (most recent call last): Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager File "/opt/stack/cinder/cinder/volume/manager.py", line 524, in _init_host Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager self.driver.do_setup(ctxt) Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager File "/opt/stack/cinder/cinder/volume/drivers/pure.py", line 432, in do_setup Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager self.do_setup_replication() Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager File "/opt/stack/cinder/cinder/volume/drivers/pure.py", line 525, in do_setup_replication Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager self._setup_replicated_pods( Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager File "/opt/stack/cinder/cinder/volume/drivers/pure.py", line 200, in wrapper Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager backend_name = driver._get_current_array().backend_id Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager File "/opt/stack/cinder/cinder/volume/drivers/pure.py", line 2877, in _get_current_array Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager raise purestorage.PureError('No functional arrays ' Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager purestorage.purestorage.PureError: PureError: No functional arrays available If synchronous replication is configured, under certain circumstances, if the replication pod defined by the parameter `pure_replication_pod_name` does not exist on the backend at driver restart. Forexample, if someone accidentally deletes the pod, this will cause the restart of the driver to fail. The driver should be capable of correctly recreating the pod. Error traceback example: Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager Traceback (most recent call last): Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager File "/opt/stack/cinder/cinder/volume/drivers/pure.py", line 2864, in _get_current_array Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager pod_info = target_array.get_pod(self._replication_pod_name) Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager File "/opt/stack/.local/lib/python3.10/site-packages/purestorage/purestorage.py", line 3291, in get_pod Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager return self._request("GET", "pod/{0}".format(pod), kwargs) Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager File "/opt/stack/cinder/cinder/volume/drivers/pure.py", line 1884, in wrapper Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager ret = fn(*args, **kwargs) Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager File "/opt/stack/.local/lib/python3.10/site-packages/purestorage/purestorage.py", line 202, in _request Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager raise PureHTTPError(self._target, str(self._rest_version), response) Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager purestorage.purestorage.PureHTTPError: PureHTTPError status code 400 returned by REST version 1.19 at 10.21.228.28: BAD REQUEST Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager [{"msg": "Pod does not exist.", "ctx": "simon-cinder"}] Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager During handling of the above exception, another exception occurred: Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager Traceback (most recent call last): Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager File "/opt/stack/cinder/cinder/volume/manager.py", line 524, in _init_host Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager self.driver.do_setup(ctxt) Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager File "/opt/stack/cinder/cinder/volume/drivers/pure.py", line 432, in do_setup Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager self.do_setup_replication() Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager File "/opt/stack/cinder/cinder/volume/drivers/pure.py", line 525, in do_setup_replication Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager self._setup_replicated_pods( Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager File "/opt/stack/cinder/cinder/volume/drivers/pure.py", line 200, in wrapper Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager backend_name = driver._get_current_array().backend_id Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager File "/opt/stack/cinder/cinder/volume/drivers/pure.py", line 2877, in _get_current_array Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager raise purestorage.PureError('No functional arrays ' Sep 13 20:36:13 sn1-pool-c07-03 cinder-volume[2654547]: ERROR cinder.volume.manager purestorage.purestorage.PureError: PureError: No functional arrays available
2023-09-13 23:07:06 OpenStack Infra cinder: status New In Progress
2024-03-05 17:57:31 OpenStack Infra cinder: status In Progress Fix Released