NetApp cDOT - Can't extend a replicated share

[Goutham Pacha Ravi]

Steps to reproduce:
* Create a share, using the ONTAP driver with share type indicating replication_type='dr
* Create a replica
* Promote replica
* Delete secondary
* Extend the share

Failure:

Jun 27 22:46:24.223242 openstack8 manila-share[3370]: 2017-06-27 22:46:24.217 ERROR manila.share.manager [req-b94bdd3d-78c5-4bcb-87cf-9ab833823768 e83153b7fe7a4886a9e45ee4ae9de822 57461a16b8324c3d956cc5d23bbe937a] Extend share failed.: NaApiError: NetApp API failed. Reason - 160:Unable to set volume attribute "size" for volume "share_13663fca_00a6_4e06_b463_f6019b804a5f" on Vserver "gouthamr". Reason: Volume has the fixed filesystem size option set.
Jun 27 22:46:24.223407 openstack8 manila-share[3370]: 2017-06-27 22:46:24.217 TRACE manila.share.manager Traceback (most recent call last):
Jun 27 22:46:24.223566 openstack8 manila-share[3370]: 2017-06-27 22:46:24.217 TRACE manila.share.manager File "/opt/stack/manila/manila/share/manager.py", line 3341, in extend_share
Jun 27 22:46:24.223732 openstack8 manila-share[3370]: 2017-06-27 22:46:24.217 TRACE manila.share.manager share_instance, new_size, share_server=share_server)
Jun 27 22:46:24.223894 openstack8 manila-share[3370]: 2017-06-27 22:46:24.217 TRACE manila.share.manager File "/opt/stack/manila/manila/share/drivers/netapp/dataontap/cluster_mode/drv_single_svm.py", line 68, in extend_share
Jun 27 22:46:24.223894 openstack8 manila-share[3370]: 2017-06-27 22:46:24.217 TRACE manila.share.manager File "/opt/stack/manila/manila/share/drivers/netapp/dataontap/cluster_mode/drv_single_svm.py", line 68, in extend_share
Jun 27 22:46:24.224054 openstack8 manila-share[3370]: 2017-06-27 22:46:24.217 TRACE manila.share.manager self.library.extend_share(share, new_size, **kwargs)
Jun 27 22:46:24.224212 openstack8 manila-share[3370]: 2017-06-27 22:46:24.217 TRACE manila.share.manager File "/opt/stack/manila/manila/share/drivers/netapp/utils.py", line 86, in trace_wrapper
Jun 27 22:46:24.224372 openstack8 manila-share[3370]: 2017-06-27 22:46:24.217 TRACE manila.share.manager result = f(self, *args, **kwargs)
Jun 27 22:46:24.224541 openstack8 manila-share[3370]: 2017-06-27 22:46:24.217 TRACE manila.share.manager File "/opt/stack/manila/manila/share/drivers/netapp/dataontap/cluster_mode/lib_base.py", line 1287, in extend_share
Jun 27 22:46:24.224703 openstack8 manila-share[3370]: 2017-06-27 22:46:24.217 TRACE manila.share.manager vserver_client.set_volume_size(share_name, new_size)
Jun 27 22:46:24.224863 openstack8 manila-share[3370]: 2017-06-27 22:46:24.217 TRACE manila.share.manager File "/opt/stack/manila/manila/share/drivers/netapp/utils.py", line 86, in trace_wrapper
Jun 27 22:46:24.225029 openstack8 manila-share[3370]: 2017-06-27 22:46:24.217 TRACE manila.share.manager result = f(self, *args, **kwargs)
Jun 27 22:46:24.225200 openstack8 manila-share[3370]: 2017-06-27 22:46:24.217 TRACE manila.share.manager File "/opt/stack/manila/manila/share/drivers/netapp/dataontap/client/client_cmode.py", line 1542, in set_volume_size
Jun 27 22:46:24.225359 openstack8 manila-share[3370]: 2017-06-27 22:46:24.217 TRACE manila.share.manager errors[0].get_child_content('error-message'))
Jun 27 22:46:24.225517 openstack8 manila-share[3370]: 2017-06-27 22:46:24.217 TRACE manila.share.manager NaApiError: NetApp API failed. Reason - 160:Unable to set volume attribute "size" for volume "share_13663fca_00a6_4e06_b463_f6019b804a5f" on Vserver "gouthamr". Reason: Volume has the fixed filesystem size option set.
Jun 27 22:46:24.225677 openstack8 manila-share[3370]: 2017-06-27 22:46:24.217 TRACE manila.share.manager
Jun 27 22:46:24.415007 openstack8 manila-share[3370]: 2017-06-27 22:46:24.409 DEBUG manila.quota [req-b94bdd3d-78c5-4bcb-87cf-9ab833823768 e83153b7fe7a4886a9e45ee4ae9de822 57461a16b8324c3d956cc5d23bbe937a] Rolled back reservations [u'9abf15bc-0897-4b74-83bc-ec9a1e4a652a']: NaApiError: NetApp API failed. Reason - 160:Unable to set volume attribute "size" for volume "share_13663fca_00a6_4e06_b463_f6019b804a5f" on Vserver "gouthamr". Reason: Volume has the fixed filesystem size option set. from (pid=3397) rollback /opt/stack/manila/manila/quota.py:1017
Jun 27 22:46:24.420364 openstack8 manila-share[3370]: 2017-06-27 22:46:24.413 ERROR oslo_messaging.rpc.server [req-b94bdd3d-78c5-4bcb-87cf-9ab833823768 e83153b7fe7a4886a9e45ee4ae9de822 57461a16b8324c3d956cc5d23bbe937a] Exception during message handling: ShareExtendingError: Share 63b74bcd-a804-4848-90bd-9df71d5d5a1b could not be extended due to error in the driver: NetApp API failed. Reason - 160:Unable to set volume attribute "size" for volume "share_13663fca_00a6_4e06_b463_f6019b804a5f" on Vserver "gouthamr". Reason: Volume has the fixed filesystem size option set.
Jun 27 22:46:24.421283 openstack8 manila-share[3370]: 2017-06-27 22:46:24.413 TRACE oslo_messaging.rpc.server Traceback (most recent call last):
Jun 27 22:46:24.421799 openstack8 manila-share[3370]: 2017-06-27 22:46:24.413 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 153, in _process_incoming
Jun 27 22:46:24.422217 openstack8 manila-share[3370]: 2017-06-27 22:46:24.413 TRACE oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
Jun 27 22:46:24.422914 openstack8 manila-share[3370]: 2017-06-27 22:46:24.413 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch
Jun 27 22:46:24.423457 openstack8 manila-share[3370]: 2017-06-27 22:46:24.413 TRACE oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
Jun 27 22:46:24.423859 openstack8 manila-share[3370]: 2017-06-27 22:46:24.413 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch
Jun 27 22:46:24.424218 openstack8 manila-share[3370]: 2017-06-27 22:46:24.413 TRACE oslo_messaging.rpc.server result = func(ctxt, **new_args)
Jun 27 22:46:24.424421 openstack8 manila-share[3370]: 2017-06-27 22:46:24.413 TRACE oslo_messaging.rpc.server File "/opt/stack/manila/manila/share/manager.py", line 169, in wrapped
Jun 27 22:46:24.424636 openstack8 manila-share[3370]: 2017-06-27 22:46:24.413 TRACE oslo_messaging.rpc.server return f(self, *args, **kwargs)
Jun 27 22:46:24.424815 openstack8 manila-share[3370]: 2017-06-27 22:46:24.413 TRACE oslo_messaging.rpc.server File "/opt/stack/manila/manila/utils.py", line 552, in wrapper
Jun 27 22:46:24.425006 openstack8 manila-share[3370]: 2017-06-27 22:46:24.413 TRACE oslo_messaging.rpc.server return func(self, *args, **kwargs)
Jun 27 22:46:24.425185 openstack8 manila-share[3370]: 2017-06-27 22:46:24.413 TRACE oslo_messaging.rpc.server File "/opt/stack/manila/manila/share/manager.py", line 3351, in extend_share
Jun 27 22:46:24.425388 openstack8 manila-share[3370]: 2017-06-27 22:46:24.413 TRACE oslo_messaging.rpc.server reason=six.text_type(e), share_id=share_id)
Jun 27 22:46:24.425600 openstack8 manila-share[3370]: 2017-06-27 22:46:24.413 TRACE oslo_messaging.rpc.server ShareExtendingError: Share 63b74bcd-a804-4848-90bd-9df71d5d5a1b could not be extended due to error in the driver: NetApp API failed. Reason - 160:Unable to set volume attribute "size" for volume "share_13663fca_00a6_4e06_b463_f6019b804a5f" on Vserver "gouthamr". Reason: Volume has the fixed filesystem size option set.

[Douglas Viroel (dviroel)]

Hi,

I can't reproduce this issue in my environment. This may have something to do with the ONTAP version used. My replica is always created with 'Is File System Size Fixed: false' configured, which allows it to be extended after promotion.
The ONTAP version used was 9.4.

Naresh, have you reproduced this issue? If yes, which ONTAP version you were using?

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/686794

[Douglas Viroel (dviroel)]

Hi,

Issue confirmed on older version of ONTAP (8) and can be reproduced on newer versions if the share replica was created before upgrading the ONTAP to a newer version.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/686794
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=75f108b0aee630b54a44d0c27467a85fd8a6af3a
Submitter: Zuul
Branch: master

commit 75f108b0aee630b54a44d0c27467a85fd8a6af3a
Author: Naresh Kumar Gunjalli <email address hidden>
Date: Thu Feb 14 13:21:30 2019 +0000

    [NetApp] Allow extension/shrinking of NetApp replicated share

    The NetApp ONTAP driver is now fixed to allow extension
    and shrinking of share replicas after they get promoted.

    Change-Id: Iea92feaf5894c10674d2ec4c1a4d7b0191e6d5b4
    Closes-Bug: #1700871

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/train
Review: https://review.opendev.org/688176

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/688176
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=e38c8b508ff0cd1f5ea81a0f566fd83fb0b069ce
Submitter: Zuul
Branch: stable/train

commit e38c8b508ff0cd1f5ea81a0f566fd83fb0b069ce
Author: Naresh Kumar Gunjalli <email address hidden>
Date: Thu Feb 14 13:21:30 2019 +0000

    [NetApp] Allow extension/shrinking of NetApp replicated share

    The NetApp ONTAP driver is now fixed to allow extension
    and shrinking of share replicas after they get promoted.

    Change-Id: Iea92feaf5894c10674d2ec4c1a4d7b0191e6d5b4
    Closes-Bug: #1700871
    (cherry picked from commit 75f108b0aee630b54a44d0c27467a85fd8a6af3a)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/689137

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/689607

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/689137
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=5255ca20fb2aff3b2548475d4b7914dc02eb8bbf
Submitter: Zuul
Branch: stable/stein

commit 5255ca20fb2aff3b2548475d4b7914dc02eb8bbf
Author: Naresh Kumar Gunjalli <email address hidden>
Date: Thu Feb 14 13:21:30 2019 +0000

    [NetApp] Allow extension/shrinking of NetApp replicated share

    The NetApp ONTAP driver is now fixed to allow extension
    and shrinking of share replicas after they get promoted.

    Change-Id: Iea92feaf5894c10674d2ec4c1a4d7b0191e6d5b4
    Closes-Bug: #1700871
    (cherry picked from commit 75f108b0aee630b54a44d0c27467a85fd8a6af3a)
    (cherry picked from commit e38c8b508ff0cd1f5ea81a0f566fd83fb0b069ce)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/689607
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=8ae267e4f144e88dda9b513f104b3244c544caea
Submitter: Zuul
Branch: stable/rocky

commit 8ae267e4f144e88dda9b513f104b3244c544caea
Author: Naresh Kumar Gunjalli <email address hidden>
Date: Thu Feb 14 13:21:30 2019 +0000

    [NetApp] Allow extension/shrinking of NetApp replicated share

    The NetApp ONTAP driver is now fixed to allow extension
    and shrinking of share replicas after they get promoted.

    Change-Id: Iea92feaf5894c10674d2ec4c1a4d7b0191e6d5b4
    Closes-Bug: #1700871
    (cherry picked from commit 75f108b0aee630b54a44d0c27467a85fd8a6af3a)
    (cherry picked from commit e38c8b508ff0cd1f5ea81a0f566fd83fb0b069ce)
    (cherry picked from commit 5255ca20fb2aff3b2548475d4b7914dc02eb8bbf)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.opendev.org/689638

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/689638
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=71cff225b079b8d7c8f9e5dfce18bfa8e179a75b
Submitter: Zuul
Branch: stable/queens

commit 71cff225b079b8d7c8f9e5dfce18bfa8e179a75b
Author: Naresh Kumar Gunjalli <email address hidden>
Date: Thu Feb 14 13:21:30 2019 +0000

    [NetApp] Allow extension/shrinking of NetApp replicated share

    The NetApp ONTAP driver is now fixed to allow extension
    and shrinking of share replicas after they get promoted.

    Change-Id: Iea92feaf5894c10674d2ec4c1a4d7b0191e6d5b4
    Closes-Bug: #1700871
    (cherry picked from commit 75f108b0aee630b54a44d0c27467a85fd8a6af3a)
    (cherry picked from commit e38c8b508ff0cd1f5ea81a0f566fd83fb0b069ce)
    (cherry picked from commit 5255ca20fb2aff3b2548475d4b7914dc02eb8bbf)
    (cherry picked from commit 8ae267e4f144e88dda9b513f104b3244c544caea)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/manila 6.3.2 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/manila 9.1.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/manila 8.1.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/manila 7.4.0 release.