NetApp driver fails when promoting back a replica using CIFS

Bug #1879368 reported by Carlos Eduardo
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Shared File Systems Service (Manila)
Low
Carlos Eduardo

Bug Description

Description
===========
The promote back of a share replica created from a CIFS share fails using the NetApp driver.

Steps to reproduce
==================

* Create a share network
* Create a security service (I've used an AD sec service)
* Attach the security service into the share network
* Create a share using the CIFS protocol
* Create a share replica of the created share
* Wait for the replica to achieve 'in_sync' state
* Promote the share replica
* Wait for the former active replica to become 'in_sync'
* Promote back the former active replica

Expected result
===============
The share replica should be promoted back successfully.

Actual result
=============
The share replica is not being promoted and the service is setting both share replicas (active and non-active) to error state.

Environment
===========
1. Devstack deployed on Ubuntu 18.04

2. Manila using the master branch (microversion 2.55)

3. NetApp DHSS=True backend using the standalone network plugin

Logs & Configs
==============

Traceback (most recent call last):
   File "/usr/local/lib/python3.7/dist-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
     res = self.dispatcher.dispatch(message)
   File "/usr/local/lib/python3.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 276, in dispatch
     return self._do_dispatch(endpoint, method, ctxt, args)
   File "/usr/local/lib/python3.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 196, in _do_dispatch
     result = func(ctxt, **new_args)
   File "/opt/stack/new/manila/manila/share/manager.py", line 187, in wrapped
     return f(self, *args, **kwargs)
   File "/opt/stack/new/manila/manila/utils.py", line 568, in wrapper
     return func(self, *args, **kwargs)
   File "/opt/stack/new/manila/manila/share/manager.py", line 173, in wrapped
     return locked_replica_operation(*args, **kwargs)
   File "<decorator-gen-52>", line 2, in locked_replica_operation
   File "/opt/stack/new/manila/manila/coordination.py", line 201, in _synchronized
     return f(*a, **k)
   File "/opt/stack/new/manila/manila/share/manager.py", line 172, in locked_replica_operation
     return operation(*_args, **_kwargs)
   File "/opt/stack/new/manila/manila/share/manager.py", line 2224, in promote_share_replica
     exception=excep)
   File "/usr/local/lib/python3.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
     self.force_reraise()
   File "/usr/local/lib/python3.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
     six.reraise(self.type_, self.value, self.tb)
   File "/usr/local/lib/python3.7/dist-packages/six.py", line 703, in reraise
     raise value
   File "/opt/stack/new/manila/manila/share/manager.py", line 2201, in promote_share_replica
     share_server=share_server)
   File "/opt/stack/new/manila/manila/share/drivers/netapp/dataontap/cluster_mode/drv_multi_svm.py", line 150, in promote_replica
     share_server=share_server)
   File "/opt/stack/new/manila/manila/share/drivers/netapp/dataontap/cluster_mode/lib_base.py", line 1938, in promote_replica
     access_rules, share_server=share_server))
   File "/opt/stack/new/manila/manila/share/drivers/netapp/dataontap/cluster_mode/lib_base.py", line 2075, in _convert_destination_replica_to_independent
     new_active_replica, share_server, vserver, vserver_client)
   File "/opt/stack/new/manila/manila/share/drivers/netapp/utils.py", line 96, in trace_wrapper
     result = f(self, *args, **kwargs)
   File "/opt/stack/new/manila/manila/share/drivers/netapp/dataontap/cluster_mode/lib_base.py", line 1136, in _create_export
     clear_current_export_policy=clear_current_export_policy)
   File "/opt/stack/new/manila/manila/share/drivers/netapp/utils.py", line 96, in trace_wrapper
     result = f(self, *args, **kwargs)
   File "/opt/stack/new/manila/manila/share/drivers/netapp/dataontap/protocols/cifs_cmode.py", line 34, in create_share
     self._client.create_cifs_share(share_name)
   File "/opt/stack/new/manila/manila/share/drivers/netapp/utils.py", line 96, in trace_wrapper
     result = f(self, *args, **kwargs)
   File "/opt/stack/new/manila/manila/share/drivers/netapp/dataontap/client/client_cmode.py", line 2688, in create_cifs_share
     self.send_request('cifs-share-create', api_args)
   File "/opt/stack/new/manila/manila/share/drivers/netapp/dataontap/client/client_base.py", line 88, in send_request
     return self.connection.invoke_successfully(request, enable_tunneling)
   File "/opt/stack/new/manila/manila/share/drivers/netapp/dataontap/client/api.py", line 288, in invoke_successfully
     raise NaApiError(code, msg)
 manila.share.drivers.netapp.dataontap.client.api.NaApiError: NetApp API failed. Reason - 13130:duplicate entry

summary: - NetApp driver fails when promoting back a replica created using CIFS
+ NetApp driver fails when promoting back a replica using CIFS
Douglas Viroel (dviroel)
tags: added: cifs netapp replication
Changed in manila:
assignee: nobody → Carlos Eduardo (silvacarlose)
Douglas Viroel (dviroel)
Changed in manila:
status: New → Triaged
importance: Undecided → Low
milestone: none → victoria-1
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (master)

Fix proposed to branch: master
Review: https://review.opendev.org/732757

Changed in manila:
status: Triaged → In Progress
Changed in manila:
milestone: victoria-1 → victoria-2
Changed in manila:
milestone: victoria-2 → victoria-3
Changed in manila:
milestone: victoria-3 → victoria-rc1
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to manila (master)

Reviewed: https://review.opendev.org/732757
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=c3aed22f9454ca684af45c929be19b8593d55609
Submitter: Zuul
Branch: master

commit c3aed22f9454ca684af45c929be19b8593d55609
Author: silvacarloss <email address hidden>
Date: Fri May 29 23:21:44 2020 +0000

    [NetApp] Fix CIFS promote back issue

    This change fixes the NetApp promote back issue when using CIFS
    protocol. When promoting a replica, the NetApp ONTAP driver
    attempts to create a new CIFS share entity (an access point as
    defined in [1]) for the new active replica. This behavior
    causes a failure since the storage identifies that a current
    backend CIFS share with the same name exists, considering
    that the reffered replica was once the active one.

    This issue is addressed by removing the related CIFS share
    entity when the replica gets promoted.

    [1] https://library.netapp.com/ecmdocs/ECMP1401220/html/GUID-1898D717-A510-4B3D-B2E3-CCDDD5BD0089.html

    Closes-Bug: #1879368
    Change-Id: Id9bdd5df0ff05ea08881dd2c83397f0a367d9945

Changed in manila:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (stable/ussuri)

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/753293

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to manila (stable/ussuri)

Reviewed: https://review.opendev.org/753293
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=d2514735141bcf982e4bf043802591b3d017b526
Submitter: Zuul
Branch: stable/ussuri

commit d2514735141bcf982e4bf043802591b3d017b526
Author: silvacarloss <email address hidden>
Date: Fri May 29 23:21:44 2020 +0000

    [NetApp] Fix CIFS promote back issue

    This change fixes the NetApp promote back issue when using CIFS
    protocol. When promoting a replica, the NetApp ONTAP driver
    attempts to create a new CIFS share entity (an access point as
    defined in [1]) for the new active replica. This behavior
    causes a failure since the storage identifies that a current
    backend CIFS share with the same name exists, considering
    that the reffered replica was once the active one.

    This issue is addressed by removing the related CIFS share
    entity when the replica gets promoted.

    [1] https://library.netapp.com/ecmdocs/ECMP1401220/html/GUID-1898D717-A510-4B3D-B2E3-CCDDD5BD0089.html

    Closes-Bug: #1879368
    Change-Id: Id9bdd5df0ff05ea08881dd2c83397f0a367d9945
    (cherry picked from commit c3aed22f9454ca684af45c929be19b8593d55609)

tags: added: in-stable-ussuri
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (stable/train)

Fix proposed to branch: stable/train
Review: https://review.opendev.org/755787

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to manila (stable/train)

Reviewed: https://review.opendev.org/755787
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=226f7e758a0b704e023e6aec0dfd271d9c4d3716
Submitter: Zuul
Branch: stable/train

commit 226f7e758a0b704e023e6aec0dfd271d9c4d3716
Author: silvacarloss <email address hidden>
Date: Fri May 29 23:21:44 2020 +0000

    [NetApp] Fix CIFS promote back issue

    This change fixes the NetApp promote back issue when using CIFS
    protocol. When promoting a replica, the NetApp ONTAP driver
    attempts to create a new CIFS share entity (an access point as
    defined in [1]) for the new active replica. This behavior
    causes a failure since the storage identifies that a current
    backend CIFS share with the same name exists, considering
    that the reffered replica was once the active one.

    This issue is addressed by removing the related CIFS share
    entity when the replica gets promoted.

    [1] https://library.netapp.com/ecmdocs/ECMP1401220/html/GUID-1898D717-A510-4B3D-B2E3-CCDDD5BD0089.html

    Closes-Bug: #1879368
    Change-Id: Id9bdd5df0ff05ea08881dd2c83397f0a367d9945
    (cherry picked from commit c3aed22f9454ca684af45c929be19b8593d55609)
    (cherry picked from commit d2514735141bcf982e4bf043802591b3d017b526)

tags: added: in-stable-train
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (stable/stein)

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/762110

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to manila (stable/stein)

Reviewed: https://review.opendev.org/762110
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=ef8d704c3b08d7de64926549a2f8f14cec3bfd0b
Submitter: Zuul
Branch: stable/stein

commit ef8d704c3b08d7de64926549a2f8f14cec3bfd0b
Author: silvacarloss <email address hidden>
Date: Fri May 29 23:21:44 2020 +0000

    [NetApp] Fix CIFS promote back issue

    This change fixes the NetApp promote back issue when using CIFS
    protocol. When promoting a replica, the NetApp ONTAP driver
    attempts to create a new CIFS share entity (an access point as
    defined in [1]) for the new active replica. This behavior
    causes a failure since the storage identifies that a current
    backend CIFS share with the same name exists, considering
    that the reffered replica was once the active one.

    This issue is addressed by removing the related CIFS share
    entity when the replica gets promoted.

    [1] https://library.netapp.com/ecmdocs/ECMP1401220/html/GUID-1898D717-A510-4B3D-B2E3-CCDDD5BD0089.html

    Closes-Bug: #1879368
    Change-Id: Id9bdd5df0ff05ea08881dd2c83397f0a367d9945
    (cherry picked from commit c3aed22f9454ca684af45c929be19b8593d55609)
    (cherry picked from commit d2514735141bcf982e4bf043802591b3d017b526)
    (cherry picked from commit 226f7e758a0b704e023e6aec0dfd271d9c4d3716)

tags: added: in-stable-stein
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (stable/rocky)

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/762492

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/manila 9.1.5

This issue was fixed in the openstack/manila 9.1.5 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/manila 10.0.2

This issue was fixed in the openstack/manila 10.0.2 release.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers