Manila share does not get into "shrinking_possible_data_loss_error" status when shrinking a share

Bug #1858328 reported by Liron Kuchlani
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack Shared File Systems Service (Manila)
Fix Released
Low
Daniel Tapia

Bug Description

Description of problem:
Manila share does not get into "shrinking_possible_data_loss_error"
status when shrinking a share which will cause to lose a data.

Version-Release number of selected component (if applicable):
puppet-manila-12.5.1-1.el7ost.noarch
python2-manilaclient-1.21.2-1.el7ost.noarch
python2-manila-tests-tempest-0.3.0-1.el7ost.noarch

How reproducible:
100%

Steps to Reproduce:
1. Launch an instance
2. Create share (Configured size + 1)
3. Configure RW access to the share
4. Perform ssh to instance
5. Mount share
6. Write data in share (in excess of 1GB)
7. Shrink share to 1GB

Actual results:
Manila share get into "shrinking_error" status.

Expected results:
Manila share should get into "shrinking_possible_data_loss_error" status and not "shrinking_error".

Additional info:

2020-01-05 06:27:14.166 43 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/manila/share/manager.py", line 3707, in shrink_share
2020-01-05 06:27:14.166 43 ERROR oslo_messaging.rpc.server error_occurred(e, **error_params)
2020-01-05 06:27:14.166 43 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/manila/share/manager.py", line 3672, in error_occurred
2020-01-05 06:27:14.166 43 ERROR oslo_messaging.rpc.server reason=six.text_type(exc), share_id=share_id)
2020-01-05 06:27:14.166 43 ERROR oslo_messaging.rpc.server ShareShrinkingError: Share 15888b1a-a320-45b3-a647-78fc8e4da24f could not be shrunk due to error in the driver: NetApp API failed. Reason - 160:Unabl
e to set volume attribute "size" for volume "share_401cac30_41d3_4330_9039_f3e01eba66ae" on Vserver "rhos_ci". Reason: Selected volume size is too small to hold the current volume data.
2020-01-05 06:27:14.166 43 ERROR oslo_messaging.rpc.server

Traceback (most recent call last):
  File "/home/stack/tempest-auto/manila-tempest-plugin/manila_tempest_tests/tests/scenario/test_share_shrink.py", line 84, in test_create_shrink_and_write
    LOG.debug('Step 8 - try update size, shrink and wait')
  File "/home/stack/tempest-auto/manila-tempest-plugin/manila_tempest_tests/services/share/v2/json/shares_client.py", line 457, in shrink_share
    "shares/%s/action" % share_id, body, version=version)
  File "/home/stack/tempest-auto/manila-tempest-plugin/manila_tempest_tests/services/share/v2/json/shares_client.py", line 70, in post
    headers=headers)
  File "tempest/lib/common/rest_client.py", line 283, in post
    return self.request('POST', url, extra_headers, headers, body, chunked)
  File "tempest/lib/common/rest_client.py", line 687, in request
    self._error_checker(resp, resp_body)
  File "tempest/lib/common/rest_client.py", line 798, in _error_checker
    raise exceptions.BadRequest(resp_body, resp=resp)
tempest.lib.exceptions.BadRequest: Bad request
Details: {u'message': u'Invalid share: Share 15888b1a-a320-45b3-a647-78fc8e4da24f status must in (available, shrinking_possible_data_loss_error) to shrink, but current status is: shrinking_error.', u'code': 400}

Tom Barron (tpb)
tags: added: drivers netapp
Douglas Viroel (dviroel)
Changed in manila:
importance: Undecided → Low
assignee: nobody → Douglas Viroel (dviroel)
status: New → Triaged
Revision history for this message
Liron Kuchlani (lkuchlan) wrote :

I think the status of the share should remain "available" and not stuck on "shrinking_error".
I suggest throwing a message to the user that indicate the shrinking failed, something like:
"The shrinking attempt was rejected due to insufficient size"

Douglas Viroel (dviroel)
Changed in manila:
milestone: none → ussuri-3
status: Triaged → Confirmed
Douglas Viroel (dviroel)
Changed in manila:
assignee: Douglas Viroel (dviroel) → Daniel Tapia (danielarthurt)
Revision history for this message
Douglas Viroel (dviroel) wrote :

@Liron, as was discussed in weekly meeting[0] and in the manila channel[1], letting the share with status "shrinking_possible_data_loss_error" will block users to manage their shares even if it is healthy. So from our point of view we agree with you that the share should remain as available and the user to get an appropriated error message warning that the shrink operation was not completed.
Even so, the NetApp driver doesn't raises the correct exception expected by the share-manager and a fix will need to be provided.
We shall fix the share-manager behavior in a new change/bug.

[0] http://eavesdrop.openstack.org/irclogs/%23openstack-meeting-alt/%23openstack-meeting-alt.2020-02-27.log.html#t2020-02-27T15:27:03
[1] http://eavesdrop.openstack.org/irclogs/%23openstack-manila/%23openstack-manila.2020-02-28.log.html#t2020-02-28T17:06:09

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (master)

Fix proposed to branch: master
Review: https://review.opendev.org/713268

Changed in manila:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to manila (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/713867

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to manila-tempest-plugin (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/715758

Changed in manila:
milestone: ussuri-3 → ussuri-rc1
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to manila (master)

Reviewed: https://review.opendev.org/713268
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=c169902d41ef2b70ce1c1181408bc0916a7ea319
Submitter: Zuul
Branch: master

commit c169902d41ef2b70ce1c1181408bc0916a7ea319
Author: danielarthurt <email address hidden>
Date: Fri Feb 14 13:28:19 2020 +0000

    [NetApp] Fix share shrink error status

    This patch fixes the error raised by NetApp driver when the user try
    to shrink a share to a size smaller than the current used space. Now
    the driver raises "shrinking_possible_data_loss" instead of
    "shrinking_error".

    Change-Id: I01a549a0e299ae0d0d9b603a9d0797f16f4b68b3
    Closes-Bug: #1858328

Changed in manila:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (stable/train)

Fix proposed to branch: stable/train
Review: https://review.opendev.org/722356

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to manila (stable/train)

Reviewed: https://review.opendev.org/722356
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=97a913209825f975ccfc2d76c91f4215d02622ad
Submitter: Zuul
Branch: stable/train

commit 97a913209825f975ccfc2d76c91f4215d02622ad
Author: danielarthurt <email address hidden>
Date: Fri Feb 14 13:28:19 2020 +0000

    [NetApp] Fix share shrink error status

    This patch fixes the error raised by NetApp driver when the user try
    to shrink a share to a size smaller than the current used space. Now
    the driver raises "shrinking_possible_data_loss" instead of
    "shrinking_error".

    Change-Id: I01a549a0e299ae0d0d9b603a9d0797f16f4b68b3
    Closes-Bug: #1858328
    (cherry picked from commit c169902d41ef2b70ce1c1181408bc0916a7ea319)

tags: added: in-stable-train
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (stable/stein)

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/723632

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to manila (stable/stein)

Reviewed: https://review.opendev.org/723632
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=d7853ae6ed258077eab90fb29055a055e085a5fd
Submitter: Zuul
Branch: stable/stein

commit d7853ae6ed258077eab90fb29055a055e085a5fd
Author: danielarthurt <email address hidden>
Date: Fri Feb 14 13:28:19 2020 +0000

    [NetApp] Fix share shrink error status

    This patch fixes the error raised by NetApp driver when the user try
    to shrink a share to a size smaller than the current used space. Now
    the driver raises "shrinking_possible_data_loss" instead of
    "shrinking_error".

    Change-Id: I01a549a0e299ae0d0d9b603a9d0797f16f4b68b3
    Closes-Bug: #1858328
    (cherry picked from commit c169902d41ef2b70ce1c1181408bc0916a7ea319)
    (cherry picked from commit 97a913209825f975ccfc2d76c91f4215d02622ad)

tags: added: in-stable-stein
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to manila-tempest-plugin (master)

Reviewed: https://review.opendev.org/715758
Committed: https://git.openstack.org/cgit/openstack/manila-tempest-plugin/commit/?id=247b0b69791154ea51a84c036168e6bf9cb1208e
Submitter: Zuul
Branch: master

commit 247b0b69791154ea51a84c036168e6bf9cb1208e
Author: danielarthurt <email address hidden>
Date: Wed Mar 18 14:31:39 2020 +0000

    Update shrink scenario tests

    This patch updates the shrink scenario tests to handle with the
    "available" or "shrinking_possible_data_loss_error" state received
    from the manila share manager after a failed shrink operation when
    attempt to shrink a share with a size smaller than thecurrent used
    space.

    Change-Id: Idf34a149e5a975d5fbced0ec78d102c5d90b87c6
    Related-Bug: #1858328

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to manila (master)

Reviewed: https://review.opendev.org/713867
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=7aeed39ba940b5108a2937bc37a4fab1475f8f50
Submitter: Zuul
Branch: master

commit 7aeed39ba940b5108a2937bc37a4fab1475f8f50
Author: danielarthurt <email address hidden>
Date: Tue Mar 3 16:41:05 2020 +0000

    Update share-manager behavior for shrink share operation

    This patch updates the share-manager to handle with the
    "ShareShrinkingPossibleDataLoss" driver exception in an appropriate way
    by setting the status to "available", and sending to the user an error
    message warning that the shrink operation was not completed.

    Change-Id: Ie1e7fcbf116d3fd831e550aa70f02a3aa60291b8
    Related-Bug: #1858328
    Depends-On: Idf34a149e5a975d5fbced0ec78d102c5d90b87c6

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to manila (stable/ussuri)

Related fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/724128

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (stable/rocky)

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/724342

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to manila (stable/rocky)

Reviewed: https://review.opendev.org/724342
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=421cd52014a860815dd4df06cc0100a59a6d2f81
Submitter: Zuul
Branch: stable/rocky

commit 421cd52014a860815dd4df06cc0100a59a6d2f81
Author: danielarthurt <email address hidden>
Date: Fri Feb 14 13:28:19 2020 +0000

    [NetApp] Fix share shrink error status

    This patch fixes the error raised by NetApp driver when the user try
    to shrink a share to a size smaller than the current used space. Now
    the driver raises "shrinking_possible_data_loss" instead of
    "shrinking_error".

    Change-Id: I01a549a0e299ae0d0d9b603a9d0797f16f4b68b3
    Closes-Bug: #1858328
    (cherry picked from commit c169902d41ef2b70ce1c1181408bc0916a7ea319)
    (cherry picked from commit 97a913209825f975ccfc2d76c91f4215d02622ad)
    (cherry picked from commit d7853ae6ed258077eab90fb29055a055e085a5fd)

tags: added: in-stable-rocky
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (stable/queens)

Fix proposed to branch: stable/queens
Review: https://review.opendev.org/724854

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to manila (stable/ussuri)

Reviewed: https://review.opendev.org/724128
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=3ea582e1f9ff6398a084df9cd07908dfb9f9d76c
Submitter: Zuul
Branch: stable/ussuri

commit 3ea582e1f9ff6398a084df9cd07908dfb9f9d76c
Author: danielarthurt <email address hidden>
Date: Tue Mar 3 16:41:05 2020 +0000

    Update share-manager behavior for shrink share operation

    This patch updates the share-manager to handle with the
    "ShareShrinkingPossibleDataLoss" driver exception in an appropriate way
    by setting the status to "available", and sending to the user an error
    message warning that the shrink operation was not completed.

    Change-Id: Ie1e7fcbf116d3fd831e550aa70f02a3aa60291b8
    Related-Bug: #1858328
    Depends-On: Idf34a149e5a975d5fbced0ec78d102c5d90b87c6
    (cherry picked from commit 7aeed39ba940b5108a2937bc37a4fab1475f8f50)

tags: added: in-stable-ussuri
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to manila (stable/queens)

Reviewed: https://review.opendev.org/724854
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=d5a16f96a4160f55e43d93fa9f225ddef395d1d1
Submitter: Zuul
Branch: stable/queens

commit d5a16f96a4160f55e43d93fa9f225ddef395d1d1
Author: danielarthurt <email address hidden>
Date: Fri Feb 14 13:28:19 2020 +0000

    [NetApp] Fix share shrink error status

    This patch fixes the error raised by NetApp driver when the user try
    to shrink a share to a size smaller than the current used space. Now
    the driver raises "shrinking_possible_data_loss" instead of
    "shrinking_error".

    Change-Id: I01a549a0e299ae0d0d9b603a9d0797f16f4b68b3
    Closes-Bug: #1858328
    (cherry picked from commit c169902d41ef2b70ce1c1181408bc0916a7ea319)
    (cherry picked from commit 97a913209825f975ccfc2d76c91f4215d02622ad)
    (cherry picked from commit d7853ae6ed258077eab90fb29055a055e085a5fd)
    (cherry picked from commit 421cd52014a860815dd4df06cc0100a59a6d2f81)

tags: added: in-stable-queens
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to manila (stable/train)

Related fix proposed to branch: stable/train
Review: https://review.opendev.org/c/openstack/manila/+/794057

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/manila queens-eol

This issue was fixed in the openstack/manila queens-eol release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/manila rocky-eol

This issue was fixed in the openstack/manila rocky-eol release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on manila (stable/train)

Change abandoned by "Goutham Pacha Ravi <email address hidden>" on branch: stable/train
Review: https://review.opendev.org/c/openstack/manila/+/794057
Reason: Not an appropriate backport to this branch anymore

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.