Fail to retype attached volume from NetApp OnTAP to SolidFire

[Fernando Ferraz]

Cinder fails to retype a attached volume from NetApp OnTAP backend to SolidFire. However, all tests I've performed shows that Cinder can successfully retype attached volumes from Solidfire to OnTAP.

Cinder also successfully retyped detached volumes on both directions.

The error message from the volume service:

Jan 21 21:05:29.821928 51-sfernand-development cinder-volume[11780]: ERROR cinder.volume.drivers.solidfire [req-2ae12450-aa3b-4eb5-be4b-18585178dcc5 req-ef5917f3-fb4a-41ad-b118-3fbedcb6c569 service None] Volume 3b29da6b-8808-417b-bc2b-924e45f24aab, not found on SF Cluster.
Jan 21 21:05:29.822830 51-sfernand-development cinder-volume[11780]: ERROR cinder.volume.drivers.solidfire [req-2ae12450-aa3b-4eb5-be4b-18585178dcc5 req-ef5917f3-fb4a-41ad-b118-3fbedcb6c569 service None] Volume ID 3b29da6b-8808-417b-bc2b-924e45f24aab was not found on the SolidFire Cluster while attempting attach_volume operation!
Jan 21 21:05:29.823776 51-sfernand-development cinder-volume[11780]: INFO cinder.message.api [req-2ae12450-aa3b-4eb5-be4b-18585178dcc5 req-ef5917f3-fb4a-41ad-b118-3fbedcb6c569 service None] Creating message record for request_id = req-ef5917f3-fb4a-41ad-b118-3fbedcb6c569
Jan 21 21:05:29.838113 51-sfernand-development cinder-volume[11780]: ERROR oslo_messaging.rpc.server [req-2ae12450-aa3b-4eb5-be4b-18585178dcc5 req-ef5917f3-fb4a-41ad-b118-3fbedcb6c569 service None] Exception during message handling: cinder.exception.VolumeNotFound: Volume 3b29da6b-8808-417b-bc2b-924e45f24aab could not be found.
Jan 21 21:05:29.838113 51-sfernand-development cinder-volume[11780]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
Jan 21 21:05:29.838113 51-sfernand-development cinder-volume[11780]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.7/dist-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
Jan 21 21:05:29.838113 51-sfernand-development cinder-volume[11780]: ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
Jan 21 21:05:29.838113 51-sfernand-development cinder-volume[11780]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 274, in dispatch
Jan 21 21:05:29.838113 51-sfernand-development cinder-volume[11780]: ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
Jan 21 21:05:29.838113 51-sfernand-development cinder-volume[11780]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 194, in _do_dispatch
Jan 21 21:05:29.838113 51-sfernand-development cinder-volume[11780]: ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
Jan 21 21:05:29.838113 51-sfernand-development cinder-volume[11780]: ERROR oslo_messaging.rpc.server File "/opt/stack/new/cinder/cinder/volume/manager.py", line 4570, in attachment_update
Jan 21 21:05:29.838113 51-sfernand-development cinder-volume[11780]: ERROR oslo_messaging.rpc.server fields.VolumeAttachStatus.ERROR_ATTACHING})
Jan 21 21:05:29.838113 51-sfernand-development cinder-volume[11780]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
Jan 21 21:05:29.838113 51-sfernand-development cinder-volume[11780]: ERROR oslo_messaging.rpc.server self.force_reraise()
Jan 21 21:05:29.838113 51-sfernand-development cinder-volume[11780]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
Jan 21 21:05:29.838113 51-sfernand-development cinder-volume[11780]: ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb)
Jan 21 21:05:29.838113 51-sfernand-development cinder-volume[11780]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.7/dist-packages/six.py", line 693, in reraise
Jan 21 21:05:29.838113 51-sfernand-development cinder-volume[11780]: ERROR oslo_messaging.rpc.server raise value
Jan 21 21:05:29.838113 51-sfernand-development cinder-volume[11780]: ERROR oslo_messaging.rpc.server File "/opt/stack/new/cinder/cinder/volume/manager.py", line 4560, in attachment_update
Jan 21 21:05:29.838113 51-sfernand-development cinder-volume[11780]: ERROR oslo_messaging.rpc.server connector.get('mountpoint', 'na'))
Jan 21 21:05:29.838113 51-sfernand-development cinder-volume[11780]: ERROR oslo_messaging.rpc.server File "/opt/stack/new/cinder/cinder/volume/drivers/solidfire.py", line 2082, in attach_volume
Jan 21 21:05:29.838113 51-sfernand-development cinder-volume[11780]: ERROR oslo_messaging.rpc.server raise exception.VolumeNotFound(volume_id=volume['id'])
Jan 21 21:05:29.838113 51-sfernand-development cinder-volume[11780]: ERROR oslo_messaging.rpc.server cinder.exception.VolumeNotFound: Volume 3b29da6b-8808-417b-bc2b-924e45f24aab could not be found.
Jan 21 21:05:29.838113 51-sfernand-development cinder-volume[11780]: ERROR oslo_messaging.rpc.server
Jan 21 21:05:30.108045 51-sfernand-development cinder-volume[11780]: ERROR cinder.volume.drivers.solidfire [req-2ae12450-aa3b-4eb5-be4b-18585178dcc5 req-4401c014-5207-4af0-9dd3-2c5e57e94e75 service None] Volume 3b29da6b-8808-417b-bc2b-924e45f24aab, not found on SF Cluster.
Jan 21 21:05:30.109065 51-sfernand-development cinder-volume[11780]: ERROR cinder.volume.drivers.solidfire [req-2ae12450-aa3b-4eb5-be4b-18585178dcc5 req-4401c014-5207-4af0-9dd3-2c5e57e94e75 service None] Volume ID 3b29da6b-8808-417b-bc2b-924e45f24aab was not found on the SolidFire Cluster while attempting detach_volume operation!

[Anung Wihadmadyatama (anungwhd)]

Peform retyping a volume (volume is attached to instance) from ONTAP ISCSI LUN to Solidfire (SF), it always failed – error log attached. From SF to ONTAP, SF to SF, and ONTAP to ONTAP are working fine. Attached the error logs.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/710336

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/710336
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=ca475a3dad993624920d2bcbe65bf98162f32e2f
Submitter: Zuul
Branch: master

commit ca475a3dad993624920d2bcbe65bf98162f32e2f
Author: Thiago Correa <email address hidden>
Date: Thu Feb 20 18:07:23 2020 +0000

    NetApp SolidFire: Fix retype to SolidFire

    Cinder can successfully retype an attached volume from SolidFire
    to different backends, but the other way around was failing. This patch
    fixes this issue, allowing retype operation for attached volumes from
    other backends to SolidFire.

    Change-Id: I75b828198a2af707d59120f29704a81ba9f2b553
    Closes-bug: #1859652

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/726436

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/train
Review: https://review.opendev.org/726438

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/726436
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=0a223ff59cab6cca7491d2e774957f5b9f06a550
Submitter: Zuul
Branch: stable/ussuri

commit 0a223ff59cab6cca7491d2e774957f5b9f06a550
Author: Thiago Correa <email address hidden>
Date: Thu Feb 20 18:07:23 2020 +0000

    NetApp SolidFire: Fix retype to SolidFire

    Cinder can successfully retype an attached volume from SolidFire
    to different backends, but the other way around was failing. This patch
    fixes this issue, allowing retype operation for attached volumes from
    other backends to SolidFire.

    Change-Id: I75b828198a2af707d59120f29704a81ba9f2b553
    Closes-bug: #1859652
    (cherry picked from commit ca475a3dad993624920d2bcbe65bf98162f32e2f)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/726438
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=3563e75687b4d581a37e168817727fb37e7d3e4c
Submitter: Zuul
Branch: stable/train

commit 3563e75687b4d581a37e168817727fb37e7d3e4c
Author: Thiago Correa <email address hidden>
Date: Thu Feb 20 18:07:23 2020 +0000

    NetApp SolidFire: Fix retype to SolidFire

    Cinder can successfully retype an attached volume from SolidFire
    to different backends, but the other way around was failing. This patch
    fixes this issue, allowing retype operation for attached volumes from
    other backends to SolidFire.

    Change-Id: I75b828198a2af707d59120f29704a81ba9f2b553
    Closes-bug: #1859652
    (cherry picked from commit ca475a3dad993624920d2bcbe65bf98162f32e2f)
    (cherry picked from commit 0a223ff59cab6cca7491d2e774957f5b9f06a550)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/cinder 15.4.1 release.