Dell PowerStore - cinder cannot delete volumes

[Oleg Nesterenkov]

Cannot delete volumes from Cinder with Dell PowerStore driver backend.

Logs below:

cinder-volume[21492]: ERROR cinder.volume.drivers.dell_emc.powerstore.client [None req-7b9ca729-e08d-4e82-b7ac-46c115f6ac1b demo None] REST Request: DELETE https://{SAN IP}/api/rest/volume/45d3e0df-4384-448e-b4ff-0547eee4bac7 with body {}
cinder-volume[21492]: ERROR cinder.volume.drivers.dell_emc.powerstore.client [None req-7b9ca729-e08d-4e82-b7ac-46c115f6ac1b demo None] REST Response: 400 with data {"messages":[{"code":"0xE0404003000B","severity":"Error","message_l10n":"The REST body parameter is not accepted.","arguments":[""]}]}
cinder-volume[21492]: ERROR cinder.volume.drivers.dell_emc.powerstore.client [None req-7b9ca729-e08d-4e82-b7ac-46c115f6ac1b demo None] Failed to delete PowerStore volume with id 45d3e0df-4384-448e-b4ff-0547eee4bac7.
cinder-volume[21492]: DEBUG cinder.coordination [None req-7b9ca729-e08d-4e82-b7ac-46c115f6ac1b demo None] Lock "b'cinder-f4bb7abf-6e9c-46ee-b314-42fdc8903afc-delete_volume'" released by "delete_volume" :: held 0.289s {{(pid=21492) _synchronized /opt/stack/cinder/cinder/coordination.py:198}}
cinder-volume[21492]: ERROR oslo_messaging.rpc.server [None req-7b9ca729-e08d-4e82-b7ac-46c115f6ac1b demo None] Exception during message handling: cinder.exception.VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Failed to delete PowerStore volume with id 45d3e0df-4384-448e-b4ff-0547eee4bac7.
cinder-volume[21492]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
cinder-volume[21492]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.8/dist-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
cinder-volume[21492]: ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
cinder-volume[21492]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.8/dist-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
cinder-volume[21492]: ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
cinder-volume[21492]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.8/dist-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
cinder-volume[21492]: ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
cinder-volume[21492]: ERROR oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/manager.py", line 203, in wrapper
cinder-volume[21492]: ERROR oslo_messaging.rpc.server skip_clean = func(self, context, volume, *args, **kwargs)
cinder-volume[21492]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun
cinder-volume[21492]: ERROR oslo_messaging.rpc.server return caller(func, *(extras + args), **kw)
cinder-volume[21492]: ERROR oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/coordination.py", line 191, in _synchronized
cinder-volume[21492]: ERROR oslo_messaging.rpc.server return f(*a, **k)
cinder-volume[21492]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun
cinder-volume[21492]: ERROR oslo_messaging.rpc.server return caller(func, *(extras + args), **kw)
cinder-volume[21492]: ERROR oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/objects/cleanable.py", line 208, in wrapper
cinder-volume[21492]: ERROR oslo_messaging.rpc.server result = f(*args, **kwargs)
cinder-volume[21492]: ERROR oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/manager.py", line 1002, in delete_volume
cinder-volume[21492]: ERROR oslo_messaging.rpc.server self._clear_db(volume, new_status)
cinder-volume[21492]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__
cinder-volume[21492]: ERROR oslo_messaging.rpc.server self.force_reraise()
cinder-volume[21492]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise
cinder-volume[21492]: ERROR oslo_messaging.rpc.server raise self.value
cinder-volume[21492]: ERROR oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/manager.py", line 986, in delete_volume
cinder-volume[21492]: ERROR oslo_messaging.rpc.server self.driver.delete_volume(volume)
cinder-volume[21492]: ERROR oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/drivers/dell_emc/powerstore/driver.py", line 134, in delete_volume
cinder-volume[21492]: ERROR oslo_messaging.rpc.server self.adapter.delete_volume(volume)
cinder-volume[21492]: ERROR oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/drivers/dell_emc/powerstore/adapter.py", line 173, in delete_volume
cinder-volume[21492]: ERROR oslo_messaging.rpc.server self.client.delete_volume_or_snapshot(provider_id)
cinder-volume[21492]: ERROR oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/drivers/dell_emc/powerstore/client.py", line 201, in delete_volume_or_snapshot
cinder-volume[21492]: ERROR oslo_messaging.rpc.server raise exception.VolumeBackendAPIException(data=msg)
cinder-volume[21492]: ERROR oslo_messaging.rpc.server cinder.exception.VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Failed to delete PowerStore volume with id 45d3e0df-4384-448e-b4ff-0547eee4bac7.
cinder-volume[21492]: ERROR oslo_messaging.rpc.server

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/cinder/+/849626

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/cinder/+/849626
Committed: https://opendev.org/openstack/cinder/commit/a4ab9220fcfd4afd3019a0f3bd4f4b452296166f
Submitter: "Zuul (22348)"
Branch: master

commit a4ab9220fcfd4afd3019a0f3bd4f4b452296166f
Author: olegnest <email address hidden>
Date: Wed Jul 13 11:06:07 2022 +0300

    PowerStore driver - Request data validation fix

    Due to json.dumps({}) method returns {}, an error occurs in some
    client's methods, such as when deleting volumes. This fix validate
    if any payload was provided for the request.

    Closes-Bug: #1981068
    Change-Id: I0b389f4f53d3b34eddc23a7aebc85a5b42f3a09f

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/cinder 21.0.0.0rc1 release candidate.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/cinder/+/863457

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/yoga
Review: https://review.opendev.org/c/openstack/cinder/+/863458

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/xena
Review: https://review.opendev.org/c/openstack/cinder/+/863459

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/cinder/+/863458
Committed: https://opendev.org/openstack/cinder/commit/f57576769024f8cb0c9e310e4552b1ae714ea855
Submitter: "Zuul (22348)"
Branch: stable/yoga

commit f57576769024f8cb0c9e310e4552b1ae714ea855
Author: olegnest <email address hidden>
Date: Wed Jul 13 11:06:07 2022 +0300

    PowerStore driver - Request data validation fix

    Due to json.dumps({}) method returns {}, an error occurs in some
    client's methods, such as when deleting volumes. This fix validate
    if any payload was provided for the request.

    Closes-Bug: #1981068
    Change-Id: I0b389f4f53d3b34eddc23a7aebc85a5b42f3a09f
    (cherry picked from commit a4ab9220fcfd4afd3019a0f3bd4f4b452296166f)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/cinder 20.1.0 release.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/cinder/+/863459
Committed: https://opendev.org/openstack/cinder/commit/c5c35dfa248782f3287ace09655a1e29410c29df
Submitter: "Zuul (22348)"
Branch: stable/xena

commit c5c35dfa248782f3287ace09655a1e29410c29df
Author: olegnest <email address hidden>
Date: Wed Jul 13 11:06:07 2022 +0300

    PowerStore driver - Request data validation fix

    Due to json.dumps({}) method returns {}, an error occurs in some
    client's methods, such as when deleting volumes. This fix validate
    if any payload was provided for the request.

    Closes-Bug: #1981068
    Change-Id: I0b389f4f53d3b34eddc23a7aebc85a5b42f3a09f
    (cherry picked from commit a4ab9220fcfd4afd3019a0f3bd4f4b452296166f)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/cinder/+/863457
Committed: https://opendev.org/openstack/cinder/commit/aa9b860eeaa7b3f88eadfcf6de61a6cd21c22685
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit aa9b860eeaa7b3f88eadfcf6de61a6cd21c22685
Author: olegnest <email address hidden>
Date: Wed Jul 13 11:06:07 2022 +0300

    PowerStore driver - Request data validation fix

    Due to json.dumps({}) method returns {}, an error occurs in some
    client's methods, such as when deleting volumes. This fix validate
    if any payload was provided for the request.

    Closes-Bug: #1981068
    Change-Id: I0b389f4f53d3b34eddc23a7aebc85a5b42f3a09f
    (cherry picked from commit a4ab9220fcfd4afd3019a0f3bd4f4b452296166f)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/cinder 19.3.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/cinder wallaby-eom release.