NetApp cDOT NFS driver sends the wrong flexvol name to the filer to delete volumes/snapshots

[Goutham Pacha Ravi]

Premise: On cDOT, flexvol names need not correspond to the names of the junction paths;
for example:

If I've a Flexvol called gouthamr_cinder_1. I can junction this volume at /cinder/gouthamr_pool_1.

This confuses the cDOT driver atleast in one path, noticably:

Steps to reproduce:

* Create FlexVol with name xyzzy
* Junction FlexVol at /foobar
* Supply FlexVol to cDOT NFS cinder driver via the shares.conf (see: http://netapp.github.io/openstack-deploy-ops-guide/ocata/content/cinder.cdot.nfs.configuration.html)
* Create cinder volume
* Create snapshot of cinder volume
* Delete snapshot (Check logs)
* Delete volume (Check logs)

The driver will fail the deletion of the volume/snapshot on first attempt and will resort to removing the files via the mounted filesystem, sample log:

client_cmode [req-20cd71c9-3a8e-48ed-b55a-bbcb5b8a523c c348f53ccf6e46c8b4a9b472350c5b49 ab610d234daf4ed2a2a87f71e27c8f04 - - -] ==> delete_file: call {'args': (<cinder.volume.drivers.netapp.dataontap.client.client_cmode.Client object at 0x36a6f50>, u'/vol/gouthamr_cinder_vol_2/snapshot-0de08672-d231-4871-b3c1-4c5cbe75fc2e'), 'kwargs': {}} trace_logging_wrapper /usr/lib/python2.7/site-packages/cinder/utils.py:920
2017-05-15 18:59:05.595 11360 DEBUG cinder.volume.drivers.netapp.dataontap.client.client_cmode [req-20cd71c9-3a8e-48ed-b55a-bbcb5b8a523c c348f53ccf6e46c8b4a9b472350c5b49 ab610d234daf4ed2a2a87f71e27c8f04 - - -] ==> delete_file: call {'args': (<cinder.volume.drivers.netapp.dataontap.client.client_cmode.Client object at 0x36a6f50>, u'/vol/gouthamr_cinder_vol_2/snapshot-0de08672-d231-4871-b3c1-4c5cbe75fc2e'), 'kwargs': {}} trace_logging_wrapper /usr/lib/python2.7/site-packages/cinder/utils.py:920
2017-05-15 18:59:05.595 11360 DEBUG cinder.volume.drivers.netapp.dataontap.client.client_cmode [req-20cd71c9-3a8e-48ed-b55a-bbcb5b8a523c c348f53ccf6e46c8b4a9b472350c5b49 ab610d234daf4ed2a2a87f71e27c8f04 - - -] ==> delete_file: call {'path_to_file': u'/vol/gouthamr_cinder_vol_2/snapshot-0de08672-d231-4871-b3c1-4c5cbe75fc2e', 'self': <cinder.volume.drivers.netapp.dataontap.client.client_cmode.Client object at 0x36a6f50>} trace_logging_wrapper /usr/lib/python2.7/site-packages/cinder/utils.py:920
2017-05-15 18:59:05.596 11360 DEBUG cinder.volume.drivers.netapp.dataontap.client.client_cmode [req-20cd71c9-3a8e-48ed-b55a-bbcb5b8a523c c348f53ccf6e46c8b4a9b472350c5b49 ab610d234daf4ed2a2a87f71e27c8f04 - - -] ==> send_request: call {'enable_tunneling': True, 'self': <cinder.volume.drivers.netapp.dataontap.client.client_cmode.Client object at 0x36a6f50>, 'api_args': {'path': u'/vol/gouthamr_cinder_vol_2/snapshot-0de08672-d231-4871-b3c1-4c5cbe75fc2e', 'is-clone-file': 'true'}, 'api_name': 'file-delete-file'} trace_logging_wrapper /usr/lib/python2.7/site-packages/cinder/utils.py:920
2017-05-15 18:59:05.596 11360 DEBUG cinder.volume.drivers.netapp.dataontap.client.api [req-20cd71c9-3a8e-48ed-b55a-bbcb5b8a523c c348f53ccf6e46c8b4a9b472350c5b49 ab610d234daf4ed2a2a87f71e27c8f04 - - -] ==> send_http_request: call {'self': <cinder.volume.drivers.netapp.dataontap.client.api.NaServer object at 0x36ab450>, 'na_element': <file-delete-file>
  <path>/vol/gouthamr_cinder_vol_2/snapshot-0de08672-d231-4871-b3c1-4c5cbe75fc2e</path>
  <is-clone-file>true</is-clone-file>
</file-delete-file>
, 'enable_tunneling': True} trace_logging_wrapper /usr/lib/python2.7/site-packages/cinder/utils.py:920
2017-05-15 18:59:05.628 11360 DEBUG cinder.volume.drivers.netapp.dataontap.client.api [req-20cd71c9-3a8e-48ed-b55a-bbcb5b8a523c c348f53ccf6e46c8b4a9b472350c5b49 ab610d234daf4ed2a2a87f71e27c8f04 - - -] <== send_http_request: return (31ms) <results xmlns="http://www.netapp.com/filer/admin" reason="Volume &quot;gouthamr_cinder_vol_2&quot; does not exist in Vserver &quot;gouthamr&quot;. Reason: entry doesn't exist. " status="failed" errno="13040"/>
 trace_logging_wrapper /usr/lib/python2.7/site-packages/cinder/utils.py:937
2017-05-15 18:59:05.628 11360 DEBUG cinder.volume.drivers.netapp.dataontap.client.client_cmode [req-20cd71c9-3a8e-48ed-b55a-bbcb5b8a523c c348f53ccf6e46c8b4a9b472350c5b49 ab610d234daf4ed2a2a87f71e27c8f04 - - -] <== send_request: exception (32ms) NaApiError() trace_logging_wrapper /usr/lib/python2.7/site-packages/cinder/utils.py:930
2017-05-15 18:59:05.629 11360 DEBUG cinder.volume.drivers.netapp.dataontap.client.client_cmode [req-20cd71c9-3a8e-48ed-b55a-bbcb5b8a523c c348f53ccf6e46c8b4a9b472350c5b49 ab610d234daf4ed2a2a87f71e27c8f04 - - -] <== delete_file: exception (33ms) NaApiError() trace_logging_wrapper /usr/lib/python2.7/site-packages/cinder/utils.py:930
2017-05-15 18:59:05.629 11360 DEBUG cinder.volume.drivers.netapp.dataontap.client.client_cmode [req-20cd71c9-3a8e-48ed-b55a-bbcb5b8a523c c348f53ccf6e46c8b4a9b472350c5b49 ab610d234daf4ed2a2a87f71e27c8f04 - - -] <== delete_file: exception (33ms) NaApiError() trace_logging_wrapper /usr/lib/python2.7/site-packages/cinder/utils.py:930
2017-05-15 18:59:05.629 11360 DEBUG cinder.volume.drivers.netapp.dataontap.client.client_cmode [req-20cd71c9-3a8e-48ed-b55a-bbcb5b8a523c c348f53ccf6e46c8b4a9b472350c5b49 ab610d234daf4ed2a2a87f71e27c8f04 - - -] <== delete_file: exception (34ms) NaApiError() trace_logging_wrapper /usr/lib/python2.7/site-packages/cinder/utils.py:930
2017-05-15 18:59:05.630 11360 DEBUG cinder.volume.drivers.netapp.dataontap.nfs_cmode [req-20cd71c9-3a8e-48ed-b55a-bbcb5b8a523c c348f53ccf6e46c8b4a9b472350c5b49 ab610d234daf4ed2a2a87f71e27c8f04 - - -] <== _delete_snapshot_on_filer: exception (138ms) NaApiError() trace_logging_wrapper /usr/lib/python2.7/site-packages/cinder/utils.py:930
2017-05-15 18:59:05.630 11360 DEBUG cinder.volume.drivers.netapp.dataontap.nfs_cmode [req-20cd71c9-3a8e-48ed-b55a-bbcb5b8a523c c348f53ccf6e46c8b4a9b472350c5b49 ab610d234daf4ed2a2a87f71e27c8f04 - - -] <== _delete_snapshot_on_filer: exception (139ms) NaApiError() trace_logging_wrapper /usr/lib/python2.7/site-packages/cinder/utils.py:930
2017-05-15 18:59:05.630 11360 DEBUG cinder.volume.drivers.netapp.dataontap.nfs_cmode [req-20cd71c9-3a8e-48ed-b55a-bbcb5b8a523c c348f53ccf6e46c8b4a9b472350c5b49 ab610d234daf4ed2a2a87f71e27c8f04 - - -] <== _delete_snapshot_on_filer: exception (141ms) NaApiError() trace_logging_wrapper /usr/lib/python2.7/site-packages/cinder/utils.py:930
2017-05-15 18:59:05.631 11360 ERROR cinder.volume.drivers.netapp.dataontap.nfs_cmode [req-20cd71c9-3a8e-48ed-b55a-bbcb5b8a523c c348f53ccf6e46c8b4a9b472350c5b49 ab610d234daf4ed2a2a87f71e27c8f04 - - -] Could not do delete of snapshot 0de08672-d231-4871-b3c1-4c5cbe75fc2e on filer, falling back to exec of "rm" command.
2017-05-15 18:59:05.631 11360 ERROR cinder.volume.drivers.netapp.dataontap.nfs_cmode Traceback (most recent call last):
2017-05-15 18:59:05.631 11360 ERROR cinder.volume.drivers.netapp.dataontap.nfs_cmode File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/netapp/dataontap/nfs_cmode.py", line 380, in _delete_backing_file_for_snapshot
2017-05-15 18:59:05.631 11360 ERROR cinder.volume.drivers.netapp.dataontap.nfs_cmode self._delete_snapshot_on_filer(snapshot)
2017-05-15 18:59:05.631 11360 ERROR cinder.volume.drivers.netapp.dataontap.nfs_cmode File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 871, in trace_method_logging_wrapper
2017-05-15 18:59:05.631 11360 ERROR cinder.volume.drivers.netapp.dataontap.nfs_cmode return trace(f)(*args, **kwargs)
2017-05-15 18:59:05.631 11360 ERROR cinder.volume.drivers.netapp.dataontap.nfs_cmode File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 924, in trace_logging_wrapper
2017-05-15 18:59:05.631 11360 ERROR cinder.volume.drivers.netapp.dataontap.nfs_cmode result = f(*args, **kwargs)
2017-05-15 18:59:05.631 11360 ERROR cinder.volume.drivers.netapp.dataontap.nfs_cmode File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 871, in trace_method_logging_wrapper
2017-05-15 18:59:05.631 11360 ERROR cinder.volume.drivers.netapp.dataontap.nfs_cmode return trace(f)(*args, **kwargs)
2017-05-15 18:59:05.631 11360 ERROR cinder.volume.drivers.netapp.dataontap.nfs_cmode File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 924, in trace_logging_wrapper
2017-05-15 18:59:05.631 11360 ERROR cinder.volume.drivers.netapp.dataontap.nfs_cmode result = f(*args, **kwargs)
2017-05-15 18:59:05.631 11360 ERROR cinder.volume.drivers.netapp.dataontap.nfs_cmode File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 871, in trace_method_logging_wrapper
2017-05-15 18:59:05.631 11360 ERROR cinder.volume.drivers.netapp.dataontap.nfs_cmode return trace(f)(*args, **kwargs)
2017-05-15 18:59:05.631 11360 ERROR cinder.volume.drivers.netapp.dataontap.nfs_cmode File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 924, in trace_logging_wrapper
2017-05-15 18:59:05.631 11360 ERROR cinder.volume.drivers.netapp.dataontap.nfs_cmode result = f(*args, **kwargs)
2017-05-15 18:59:05.631 11360 ERROR cinder.volume.drivers.netapp.dataontap.nfs_cmode File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/netapp/dataontap/nfs_cmode.py", line 398, in _delete_snapshot_on_filer
2017-05-15 18:59:05.631 11360 ERROR cinder.volume.drivers.netapp.dataontap.nfs_cmode self.zapi_client.delete_file(path_on_filer)
2017-05-15 18:59:05.631 11360 ERROR cinder.volume.drivers.netapp.dataontap.nfs_cmode File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 871, in trace_method_logging_wrapper
2017-05-15 18:59:05.631 11360 ERROR cinder.volume.drivers.netapp.dataontap.nfs_cmode return trace(f)(*args, **kwargs)
2017-05-15 18:59:05.631 11360 ERROR cinder.volume.drivers.netapp.dataontap.nfs_cmode File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 924, in trace_logging_wrapper
2017-05-15 18:59:05.631 11360 ERROR cinder.volume.drivers.netapp.dataontap.nfs_cmode result = f(*args, **kwargs)
2017-05-15 18:59:05.631 11360 ERROR cinder.volume.drivers.netapp.dataontap.nfs_cmode File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 871, in trace_method_logging_wrapper
2017-05-15 18:59:05.631 11360 ERROR cinder.volume.drivers.netapp.dataontap.nfs_cmode return trace(f)(*args, **kwargs)
2017-05-15 18:59:05.631 11360 ERROR cinder.volume.drivers.netapp.dataontap.nfs_cmode File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 924, in trace_logging_wrapper
2017-05-15 18:59:05.631 11360 ERROR cinder.volume.drivers.netapp.dataontap.nfs_cmode result = f(*args, **kwargs)
2017-05-15 18:59:05.631 11360 ERROR cinder.volume.drivers.netapp.dataontap.nfs_cmode File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 871, in trace_method_logging_wrapper
2017-05-15 18:59:05.631 11360 ERROR cinder.volume.drivers.netapp.dataontap.nfs_cmode return trace(f)(*args, **kwargs)
2017-05-15 18:59:05.631 11360 ERROR cinder.volume.drivers.netapp.dataontap.nfs_cmode File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 924, in trace_logging_wrapper
2017-05-15 18:59:05.631 11360 ERROR cinder.volume.drivers.netapp.dataontap.nfs_cmode result = f(*args, **kwargs)
2017-05-15 18:59:05.631 11360 ERROR cinder.volume.drivers.netapp.dataontap.nfs_cmode File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/netapp/dataontap/client/client_cmode.py", line 638, in delete_file
2017-05-15 18:59:05.631 11360 ERROR cinder.volume.drivers.netapp.dataontap.nfs_cmode self.send_request('file-delete-file', api_args, True)
2017-05-15 18:59:05.631 11360 ERROR cinder.volume.drivers.netapp.dataontap.nfs_cmode File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 871, in trace_method_logging_wrapper
2017-05-15 18:59:05.631 11360 ERROR cinder.volume.drivers.netapp.dataontap.nfs_cmode return trace(f)(*args, **kwargs)
2017-05-15 18:59:05.631 11360 ERROR cinder.volume.drivers.netapp.dataontap.nfs_cmode File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 924, in trace_logging_wrapper
2017-05-15 18:59:05.631 11360 ERROR cinder.volume.drivers.netapp.dataontap.nfs_cmode result = f(*args, **kwargs)
2017-05-15 18:59:05.631 11360 ERROR cinder.volume.drivers.netapp.dataontap.nfs_cmode File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/netapp/dataontap/client/client_base.py", line 76, in send_request
2017-05-15 18:59:05.631 11360 ERROR cinder.volume.drivers.netapp.dataontap.nfs_cmode return self.connection.invoke_successfully(request, enable_tunneling)
2017-05-15 18:59:05.631 11360 ERROR cinder.volume.drivers.netapp.dataontap.nfs_cmode File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/netapp/dataontap/client/api.py", line 247, in invoke_successfully
2017-05-15 18:59:05.631 11360 ERROR cinder.volume.drivers.netapp.dataontap.nfs_cmode raise NaApiError(code, msg)
2017-05-15 18:59:05.631 11360 ERROR cinder.volume.drivers.netapp.dataontap.nfs_cmode NaApiError: NetApp API failed. Reason - 13040:Volume "gouthamr_cinder_vol_2" does not exist in Vserver "gouthamr". Reason: entry doesn't exist.
2017-05-15 18:59:05.631 11360 ERROR cinder.volume.drivers.netapp.dataontap.nfs_cmode
2017-05-15 18:59:05.633 11360 DEBUG cinder.volume.drivers.netapp.dataontap.nfs_cmode [req-20cd71c9-3a8e-48ed-b55a-bbcb5b8a523c c348f53ccf6e46c8b4a9b472350c5b49 ab610d234daf4ed2a2a87f71e27c8f04 - - -] ==> delete_snapshot: call {'self': <cinder.volume.drivers.netapp.dataontap.nfs_cmode.NetAppCmodeNfsDriver object at 0x36a6c10>, 'snapshot': Snapshot(cgsnapshot_id=None,created_at=2017-05-15T22:59:03Z,deleted=False,deleted_at=None,display_description=None,display_name=None,encryption_key_id=None,id=0de08672-d231-4871-b3c1-4c5cbe75fc2e,metadata={},progress='100%',project_id=ab610d234daf4ed2a2a87f71e27c8f04,provider_auth=None,provider_id=None,provider_location=None,status='deleting',updated_at=2017-05-15T22:59:04Z,user_id=c348f53ccf6e46c8b4a9b472350c5b49,volume=Volume(c97a71f9-ce2d-4ed7-b0b6-3bc5a8d1a791),volume_id=c97a71f9-ce2d-4ed7-b0b6-3bc5a8d1a791,volume_size=1,volume_type_id=93613042-bb7c-4971-884f-9a29d463592e)} trace_logging_wrapper /usr/lib/python2.7/site-packages/cinder/utils.py:920

RCA:
The bug seems because the driver assumes the flexvol name is the name of the junction path: https://github.com/openstack/cinder/blob/a55a6b5/cinder/volume/drivers/netapp/dataontap/nfs_cmode.py#L439
https://github.com/openstack/cinder/blob/987476c/cinder/volume/drivers/netapp/dataontap/nfs_base.py#L857

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/559400

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/559400
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=4d75cbf3c35a8aa917d3970beac99d612f13eed3
Submitter: Zuul
Branch: master

commit 4d75cbf3c35a8aa917d3970beac99d612f13eed3
Author: tpsilva <email address hidden>
Date: Wed Apr 4 13:24:33 2018 -0300

    NetApp ONTAP: Fix export path used as volume name

    On ONTAP NFS driver, the export path was being used as the volume
    name. If the export path is different than the volume name, the
    API call to delete files would fail and the driver would invoke
    a fallback method that deletes the files manually. This patch fixes
    that by finding the correct volume name.

    Change-Id: Ice78889573a36ff5e8873a0d316ddcf180d0263f
    Closes-bug: #1690954

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/cinder 13.0.0.0b1 development milestone.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/563150

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/563150
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=64df0693991bd3815acc8e445da912a499198f7e
Submitter: Zuul
Branch: stable/queens

commit 64df0693991bd3815acc8e445da912a499198f7e
Author: tpsilva <email address hidden>
Date: Wed Apr 4 13:24:33 2018 -0300

    NetApp ONTAP: Fix export path used as volume name

    On ONTAP NFS driver, the export path was being used as the volume
    name. If the export path is different than the volume name, the
    API call to delete files would fail and the driver would invoke
    a fallback method that deletes the files manually. This patch fixes
    that by finding the correct volume name.

    (cherry-picked from commit 4d75cbf3c35a8aa917d3970beac99d612f13eed3)
    Change-Id: Ice78889573a36ff5e8873a0d316ddcf180d0263f
    Closes-bug: #1690954

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/564626

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/564626
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=fd2c17edc0332e20149e1aed652a613f2f90de61
Submitter: Zuul
Branch: stable/pike

commit fd2c17edc0332e20149e1aed652a613f2f90de61
Author: tpsilva <email address hidden>
Date: Wed Apr 4 13:24:33 2018 -0300

    NetApp ONTAP: Fix export path used as volume name

    On ONTAP NFS driver, the export path was being used as the volume
    name. If the export path is different than the volume name, the
    API call to delete files would fail and the driver would invoke
    a fallback method that deletes the files manually. This patch fixes
    that by finding the correct volume name.

    (cherry-picked from commit 64df0693991bd3815acc8e445da912a499198f7e)
    Change-Id: Ice78889573a36ff5e8873a0d316ddcf180d0263f
    Closes-bug: #1690954

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/567893

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/cinder 12.0.2 release.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/567893
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=dbca62c207d63bbc192acf75ae39c6b56702295a
Submitter: Zuul
Branch: stable/ocata

commit dbca62c207d63bbc192acf75ae39c6b56702295a
Author: tpsilva <email address hidden>
Date: Wed Apr 4 13:24:33 2018 -0300

    NetApp ONTAP: Fix export path used as volume name

    On ONTAP NFS driver, the export path was being used as the volume
    name. If the export path is different than the volume name, the
    API call to delete files would fail and the driver would invoke
    a fallback method that deletes the files manually. This patch fixes
    that by finding the correct volume name.

    (cherry-picked from commit fd2c17edc0332e20149e1aed652a613f2f90de61)
    Change-Id: Ice78889573a36ff5e8873a0d316ddcf180d0263f
    Closes-bug: #1690954

[Purandhar Sairam Mannidi (sai438)]

Can we backport the patch to newton too??

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: driverfixes/newton
Review: https://review.openstack.org/572754

[Tiago Pasqualini da Silva (tiago.pasqualini)]

Backported to driverfixes/newton. Waiting for community review.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/572754
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=21b39d24dbd6fd496f6ba3e2116b5e705f64a76f
Submitter: Zuul
Branch: driverfixes/newton

commit 21b39d24dbd6fd496f6ba3e2116b5e705f64a76f
Author: tpsilva <email address hidden>
Date: Wed Apr 4 13:24:33 2018 -0300

    NetApp ONTAP: Fix export path used as volume name

    On ONTAP NFS driver, the export path was being used as the volume
    name. If the export path is different than the volume name, the
    API call to delete files would fail and the driver would invoke
    a fallback method that deletes the files manually. This patch fixes
    that by finding the correct volume name.

    (cherry-picked from commit dbca62c207d63bbc192acf75ae39c6b56702295a)
    Change-Id: Ice78889573a36ff5e8873a0d316ddcf180d0263f
    Closes-bug: #1690954

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/cinder 11.1.1 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/cinder 10.0.8 release.