Netapp Cinder: lun mapping behavior is inconsistent when initiator connected to a custom igroup

[chad morgenstern]

In the case where a iSCSI (and one can assume FCP as well) initiator is associated with a non openstack igroup:

1) Upon issuance of the command openstack server add volume, the cinder NetApp driver maps the NetApp LUN with the existing igroup.
2) Upon deletion of the of the nova instance, the cinder NetApp driver is not unmapping the LUN from the pre-existing igroup.
3) The next time cinder tries to map the same LUN, mapping fails as the LUN is already mapped
4) Because the LUN is already mapped, cinder fails to attach the LUN (cinder volume) again.
5) Manual unmapping within ONTAP alleviates the issue (though the same issue is encountered the next time the cinder volume is detached - reattached).

Here is my assesment:
1) As cinder generated igroups are preferable over custom created igroups, cinder should always create its own igroup even if one already exists to which the initiator is connected.
2) In contrast, Cinder should never map to a pre-existing non-openstack generated igroup. If 1) above is deemed unsafe or inappropriate by engineering, then mapping should fail and appropriate error reported in the error log stating the pre-existance of a non-openstack generated igroup.
3) If it is deemed appropriate to use a pre-existing non openstack generated igroup to which the initiator is already assigned, as cinder is presently doing, then an unmap must be issued just as the map is issued.

Here is the error from the volume.log:
2017-06-12 11:31:04.455 16260 ERROR cinder.volume.manager NaApiError: NetApp API failed. Reason - 9023:LUN already mapped to this group
2017-06-12 11:31:04.455 16260 ERROR cinder.volume.manager
2017-06-12 11:31:04.457 16260 ERROR oslo_messaging.rpc.server [req-63d598b2-20d3-478d-b259-6dde127f4a1d 6215333466e94962aed5a39c31d22772 8ff64c0ece054430aabbef286dbf4a69 - default default] Exception during message handling
2017-06-12 11:31:04.457 16260 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2017-06-12 11:31:04.457 16260 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 133, in _process_incoming
2017-06-12 11:31:04.457 16260 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
2017-06-12 11:31:04.457 16260 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 150, in dispatch
2017-06-12 11:31:04.457 16260 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
2017-06-12 11:31:04.457 16260 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 121, in _do_dispatch
2017-06-12 11:31:04.457 16260 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
2017-06-12 11:31:04.457 16260 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 4400, in initialize_connection
2017-06-12 11:31:04.457 16260 ERROR oslo_messaging.rpc.server connector, volume=volume)
2017-06-12 11:31:04.457 16260 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1437, in initialize_connection
2017-06-12 11:31:04.457 16260 ERROR oslo_messaging.rpc.server raise exception.VolumeBackendAPIException(data=err_msg)
2017-06-12 11:31:04.457 16260 ERROR oslo_messaging.rpc.server VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Driver initialize connection failed (error: NetApp API failed. Reason - 9023:LUN already mapped to this group).
2017-06-12 11:31:04.457 16260 ERROR oslo_messaging.rpc.server
2017-06-12 11:31:04.947 16260 WARNING cinder.volume.drivers.netapp.dataontap.client.client_base [req-ab913ded-dd74-4f8d-a342-e40df4eaf706 6215333466e94962aed5a39c31d22772 8ff64c0ece054430aabbef286dbf4a69 - default default] Error unmapping LUN. Code :13115, Message: Invalid value specified for "initiator-group" element within "lun-unmap": "".

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/730936

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/cinder/+/730936
Committed: https://opendev.org/openstack/cinder/commit/1b2742a3d397d67ea989f080d44cc537bdcf1f5a
Submitter: "Zuul (22348)"
Branch: master

commit 1b2742a3d397d67ea989f080d44cc537bdcf1f5a
Author: Gorka Eguileor <email address hidden>
Date: Tue May 26 20:40:08 2020 +0200

    NetApp: Support custom igroups

    This patch adds compatibility for the NetApp driver to allow custom and
    OpenStack igroups to coexist with each other.

    Current NetApp dataontap code for iSCSI and FC fails when detaching a
    volume if we have a custom (as in not created by the Cinder driver)
    igroup on the array.

    This is because when we unmap a volume we look for an igroup created by
    the Cinder driver, which has the "openstack-" prefix, but we won't find
    it when there's a custom igroup because on attach we used the custom
    igroup instead of creating an "openstack-" one.

    Since the NetApp backend supports having multiple igroups with the same
    initiators we will only use the OpenStack one when mapping volumes and
    create one if it doesn't exist, even if there's already a custom one.

    This patch also adds code so we can properly unmap volumes that have
    been atached with old code and custom igroups.

    Closes-Bug: #1697490
    Change-Id: I9490229fb4f2852cd1d5e5c838b6ca59fe946358

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/cinder 21.0.0.0rc1 release candidate.