NetApp replica update check eventually fails with "Another transfer is in progress"

Bug #1848889 reported by Douglas Viroel
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Shared File Systems Service (Manila)
Fix Released
Low
Douglas Viroel

Bug Description

The NetApp driver eventually fails during the share replica update routine, after a snapmirror resync operation, returning "Another transfer is in progress". The share goes to Error right after being created by tempest tests. This issue is often seen when the replica update interval is set to a value around 10 seconds.
The error returned by the NetApp ONTAP is the following:

manila-share[23252]: ERROR manila.share.drivers.netapp.dataontap.cluster_mode.lib_base [None req-139d6c40-76d8-4294-b216-b4c1e5aec83d None None] Could not resync snapmirror.: manila.share.drivers.netapp.dataontap.client.api.NaApiError: NetApp API failed. Reason - 17131:Another transfer is in progress.

The error seems to be related to the snapmirror-status returned by the last query:

manila-share[23252]: DEBUG manila.share.drivers.netapp.dataontap.client.api [None req-139d6c40-76d8-4294-b216-b4c1e5aec83d None None] Response: b'<results xmlns="http://www.netapp.com/filer/admin" status="passed">\n <attributes-list>\n <snapmirror-info>\n <destination-location>os_0c0372e8-228a-4aa6-a865-9c394babeb22:share_2727fcc1_bdc7_416a_9d6f_50fd7bcb21b1</destination-location>\n <destination-volume>share_2727fcc1_bdc7_416a_9d6f_50fd7bcb21b1</destination-volume>\n <destination-vserver>os_0c0372e8-228a-4aa6-a865-9c394babeb22</destination-vserver>\n <mirror-state>uninitialized</mirror-state>\n <relationship-status>finalizing</relationship-status>\n <source-location>os_a7489ba9-55c0-4b3e-a016-e8ac05072f35:share_e3597f01_1b88_4bf3_bca8_53e83f647a15</source-location>\n <source-volume>share_e3597f01_1b88_4bf3_bca8_53e83f647a15</source-volume>\n <source-vserver>os_a7489ba9-55c0-4b3e-a016-e8ac05072f35</source-vserver>\n </snapmirror-info>\n </attributes-list>\n <num-records>1</num-records>\n</results>\n' {{(pid=24223) invoke_elem /opt/stack/new/manila/manila/share/drivers/netapp/dataontap/client/api.py:263}}

Revision history for this message
Douglas Viroel (dviroel) wrote :
Revision history for this message
Douglas Viroel (dviroel) wrote :
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (master)

Fix proposed to branch: master
Review: https://review.opendev.org/693082

Changed in manila:
status: New → In Progress
Changed in manila:
milestone: none → ussuri-1
Tom Barron (tpb)
tags: added: driver
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to manila (master)

Reviewed: https://review.opendev.org/693082
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=572be43bb03a67376b52a11c70e587eb160b68a3
Submitter: Zuul
Branch: master

commit 572be43bb03a67376b52a11c70e587eb160b68a3
Author: Douglas Viroel <email address hidden>
Date: Fri Oct 18 10:47:03 2019 -0300

    [NetApp] Fix share replica failing for 'transfer in progress' error

    This patch fix an issue in NetApp driver share replica periodic
    check that erroneuouly set a share replica to error after receiving
    a 'Another transfer is in progress' error. The driver now skips the
    resync operation for all expected transfer in progress relationship
    status.

    Closes-Bug: #1848889
    Change-Id: If82aaa184db84660389f82654279f3c5b238fd51

Changed in manila:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (stable/train)

Fix proposed to branch: stable/train
Review: https://review.opendev.org/719300

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to manila (stable/train)

Reviewed: https://review.opendev.org/719300
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=cc2dea14a514ad1a7025d904074307817c60e3ce
Submitter: Zuul
Branch: stable/train

commit cc2dea14a514ad1a7025d904074307817c60e3ce
Author: Douglas Viroel <email address hidden>
Date: Fri Oct 18 10:47:03 2019 -0300

    [NetApp] Fix share replica failing for 'transfer in progress' error

    This patch fix an issue in NetApp driver share replica periodic
    check that erroneuouly set a share replica to error after receiving
    a 'Another transfer is in progress' error. The driver now skips the
    resync operation for all expected transfer in progress relationship
    status.

    Closes-Bug: #1848889
    Change-Id: If82aaa184db84660389f82654279f3c5b238fd51
    (cherry picked from commit 572be43bb03a67376b52a11c70e587eb160b68a3)

tags: added: in-stable-train
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (stable/stein)

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/723622

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to manila (stable/stein)

Reviewed: https://review.opendev.org/723622
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=48f3e09d12f38084a16309360fd2fbdddd441c4f
Submitter: Zuul
Branch: stable/stein

commit 48f3e09d12f38084a16309360fd2fbdddd441c4f
Author: Douglas Viroel <email address hidden>
Date: Fri Oct 18 10:47:03 2019 -0300

    [NetApp] Fix share replica failing for 'transfer in progress' error

    This patch fix an issue in NetApp driver share replica periodic
    check that erroneuouly set a share replica to error after receiving
    a 'Another transfer is in progress' error. The driver now skips the
    resync operation for all expected transfer in progress relationship
    status.

    Closes-Bug: #1848889
    Change-Id: If82aaa184db84660389f82654279f3c5b238fd51
    (cherry picked from commit 572be43bb03a67376b52a11c70e587eb160b68a3)
    (cherry picked from commit cc2dea14a514ad1a7025d904074307817c60e3ce)

tags: added: in-stable-stein
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (stable/rocky)

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/724347

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to manila (stable/rocky)

Reviewed: https://review.opendev.org/724347
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=d03dc6ef512f211cc81b83a62499c897dcefd3bc
Submitter: Zuul
Branch: stable/rocky

commit d03dc6ef512f211cc81b83a62499c897dcefd3bc
Author: Douglas Viroel <email address hidden>
Date: Fri Oct 18 10:47:03 2019 -0300

    [NetApp] Fix share replica failing for 'transfer in progress' error

    This patch fix an issue in NetApp driver share replica periodic
    check that erroneuouly set a share replica to error after receiving
    a 'Another transfer is in progress' error. The driver now skips the
    resync operation for all expected transfer in progress relationship
    status.

    Closes-Bug: #1848889
    Change-Id: If82aaa184db84660389f82654279f3c5b238fd51
    (cherry picked from commit 572be43bb03a67376b52a11c70e587eb160b68a3)
    (cherry picked from commit cc2dea14a514ad1a7025d904074307817c60e3ce)
    (cherry picked from commit 48f3e09d12f38084a16309360fd2fbdddd441c4f)

tags: added: in-stable-rocky
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (stable/queens)

Fix proposed to branch: stable/queens
Review: https://review.opendev.org/724853

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to manila (stable/queens)

Reviewed: https://review.opendev.org/724853
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=9065e7c09cc4f7b5ac611e35c61fc7f571cd40a0
Submitter: Zuul
Branch: stable/queens

commit 9065e7c09cc4f7b5ac611e35c61fc7f571cd40a0
Author: Douglas Viroel <email address hidden>
Date: Fri Oct 18 10:47:03 2019 -0300

    [NetApp] Fix share replica failing for 'transfer in progress' error

    This patch fix an issue in NetApp driver share replica periodic
    check that erroneuouly set a share replica to error after receiving
    a 'Another transfer is in progress' error. The driver now skips the
    resync operation for all expected transfer in progress relationship
    status.

    Closes-Bug: #1848889
    Change-Id: If82aaa184db84660389f82654279f3c5b238fd51
    (cherry picked from commit 572be43bb03a67376b52a11c70e587eb160b68a3)
    (cherry picked from commit cc2dea14a514ad1a7025d904074307817c60e3ce)
    (cherry picked from commit 48f3e09d12f38084a16309360fd2fbdddd441c4f)
    (cherry picked from commit d03dc6ef512f211cc81b83a62499c897dcefd3bc)

tags: added: in-stable-queens
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/manila queens-eol

This issue was fixed in the openstack/manila queens-eol release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/manila rocky-eol

This issue was fixed in the openstack/manila rocky-eol release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.