OpenStack Shared File Systems Service (Manila)

When deleting the share instance, I get an error'Couldn't find the share in used list'

Bug #1894362 reported by Lin PeiWen
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Shared File Systems Service (Manila)
Fix Released
Undecided
Lin PeiWen

Bug Description

Description
===========
There are three manila share driver services in my environment, and I am configuring'host = glusterfs' to improve high availability and load balancing capabilities, but an error occurs when deleting the share instance

Steps to reproduce
==================

* Deploying Manila using glusterfs driver
* Create a shared instance
* delete the shared instance

Expected result
===============

The manila shared instance was successfully deleted

Actual result
=============
vi /var/log/manila/share.log
2020-09-05 15:14:36.018 19 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/manila/share/manager.py", line 3070, in delete_share_instance
2020-09-05 15:14:36.018 19 ERROR oslo_messaging.rpc.server share_server=share_server)
2020-09-05 15:14:36.018 19 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/manila/share/drivers/glusterfs/layout.py", line 177, in delete_share
2020-09-05 15:14:36.018 19 ERROR oslo_messaging.rpc.server return self.layout.delete_share(*a, **kw)
2020-09-05 15:14:36.018 19 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/manila/share/drivers/glusterfs/layout_volume.py", line 437, in delete_share
2020-09-05 15:14:36.018 19 ERROR oslo_messaging.rpc.server self._push_gluster_vol(gmgr.qualified)
2020-09-05 15:14:36.018 19 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py", line 359, in inner
2020-09-05 15:14:36.018 19 ERROR oslo_messaging.rpc.server return f(*args, **kwargs)
2020-09-05 15:14:36.018 19 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/manila/share/drivers/glusterfs/layout_volume.py", line 334, in _push_gluster_vol
2020-09-05 15:14:36.018 19 ERROR oslo_messaging.rpc.server raise exception.GlusterfsException(msg)
2020-09-05 15:14:36.018 19 ERROR oslo_messaging.rpc.server manila.exception.GlusterfsException: Couldn't find the share in used list.

Environment
===========

1.OpenStack master/rocky

2.Which storage backend did you use? GlusterFS driver

3.glusterfs_nfs_server_type = Gluster

4.glusterfs_share_layout = layout_volume.GlusterfsVolumeMappedLayout

Logs & Configs
==============

# vim /etc/manila/manila.conf
[DEFAULT]
host = glusterfs

Logs error: http://paste.openstack.org/show/797491/

Lin PeiWen (linpeiwen)
Changed in manila:
assignee: nobody → Lin PeiWen (linpeiwen)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (master)

Fix proposed to branch: master
Review: https://review.opendev.org/750067

Changed in manila:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to manila (master)

Reviewed: https://review.opendev.org/750067
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=41b0b95ef67e6c65df3125f94d2fe56b5a3630c6
Submitter: Zuul
Branch: master

commit 41b0b95ef67e6c65df3125f94d2fe56b5a3630c6
Author: linpeiwen <email address hidden>
Date: Sat Sep 5 09:42:31 2020 +0000

    [Glusterfs] Fix delete share, Couldn't find the 'gluster_used_vols'

    When we have multiple share driver backends, create a shared instance,
    and the list of'self.gluster_used_vols' will only be updated on the
    current node. If the RPC request to delete the share instance is sent
    to other nodes,'self.gluster_used_vols' will be Cannot find the
    information of the glusterfs volume we want to delete, so we need to
    update'self.gluster_used_vols' when deleting the instance

    Change-Id: I14835f6c54376737b41cbf78c94908ea1befde15
    Closes-Bug: #1894362

Changed in manila:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (stable/ussuri)

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/752193

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to manila (stable/ussuri)

Reviewed: https://review.opendev.org/752193
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=18623a0eebe223c0208d70310a8d0308460bd355
Submitter: Zuul
Branch: stable/ussuri

commit 18623a0eebe223c0208d70310a8d0308460bd355
Author: linpeiwen <email address hidden>
Date: Sat Sep 5 09:42:31 2020 +0000

    [Glusterfs] Fix delete share, Couldn't find the 'gluster_used_vols'

    When we have multiple share driver backends, create a shared instance,
    and the list of'self.gluster_used_vols' will only be updated on the
    current node. If the RPC request to delete the share instance is sent
    to other nodes,'self.gluster_used_vols' will be Cannot find the
    information of the glusterfs volume we want to delete, so we need to
    update'self.gluster_used_vols' when deleting the instance

    Change-Id: I14835f6c54376737b41cbf78c94908ea1befde15
    Closes-Bug: #1894362
    (cherry picked from commit 41b0b95ef67e6c65df3125f94d2fe56b5a3630c6)

tags: added: in-stable-ussuri
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (stable/train)

Fix proposed to branch: stable/train
Review: https://review.opendev.org/752600

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to manila (stable/train)

Reviewed: https://review.opendev.org/752600
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=807b10f8dbd45af7ea5d11204b1a1bb2e5a2a9a3
Submitter: Zuul
Branch: stable/train

commit 807b10f8dbd45af7ea5d11204b1a1bb2e5a2a9a3
Author: linpeiwen <email address hidden>
Date: Sat Sep 5 09:42:31 2020 +0000

    [Glusterfs] Fix delete share, Couldn't find the 'gluster_used_vols'

    When we have multiple share driver backends, create a shared instance,
    and the list of'self.gluster_used_vols' will only be updated on the
    current node. If the RPC request to delete the share instance is sent
    to other nodes,'self.gluster_used_vols' will be Cannot find the
    information of the glusterfs volume we want to delete, so we need to
    update'self.gluster_used_vols' when deleting the instance

    Change-Id: I14835f6c54376737b41cbf78c94908ea1befde15
    Closes-Bug: #1894362
    (cherry picked from commit 41b0b95ef67e6c65df3125f94d2fe56b5a3630c6)
    (cherry picked from commit 18623a0eebe223c0208d70310a8d0308460bd355)

tags: added: in-stable-train
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (stable/stein)

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/752780

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to manila (stable/stein)

Reviewed: https://review.opendev.org/752780
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=017c0c42d330ad870187f48a7f469599746051f9
Submitter: Zuul
Branch: stable/stein

commit 017c0c42d330ad870187f48a7f469599746051f9
Author: linpeiwen <email address hidden>
Date: Sat Sep 5 09:42:31 2020 +0000

    [Glusterfs] Fix delete share, Couldn't find the 'gluster_used_vols'

    When we have multiple share driver backends, create a shared instance,
    and the list of'self.gluster_used_vols' will only be updated on the
    current node. If the RPC request to delete the share instance is sent
    to other nodes,'self.gluster_used_vols' will be Cannot find the
    information of the glusterfs volume we want to delete, so we need to
    update'self.gluster_used_vols' when deleting the instance

    Change-Id: I14835f6c54376737b41cbf78c94908ea1befde15
    Closes-Bug: #1894362
    (cherry picked from commit 41b0b95ef67e6c65df3125f94d2fe56b5a3630c6)
    (cherry picked from commit 18623a0eebe223c0208d70310a8d0308460bd355)
    (cherry picked from commit 807b10f8dbd45af7ea5d11204b1a1bb2e5a2a9a3)

tags: added: in-stable-stein
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (stable/rocky)

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/753228

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to manila (stable/rocky)

Reviewed: https://review.opendev.org/753228
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=79b837ad6c0b761d6069da299c4c39ecd44dabf2
Submitter: Zuul
Branch: stable/rocky

commit 79b837ad6c0b761d6069da299c4c39ecd44dabf2
Author: linpeiwen <email address hidden>
Date: Sat Sep 5 09:42:31 2020 +0000

    [Glusterfs] Fix delete share, Couldn't find the 'gluster_used_vols'

    When we have multiple share driver backends, create a shared instance,
    and the list of'self.gluster_used_vols' will only be updated on the
    current node. If the RPC request to delete the share instance is sent
    to other nodes,'self.gluster_used_vols' will be Cannot find the
    information of the glusterfs volume we want to delete, so we need to
    update'self.gluster_used_vols' when deleting the instance

    Change-Id: I14835f6c54376737b41cbf78c94908ea1befde15
    Closes-Bug: #1894362
    (cherry picked from commit 41b0b95ef67e6c65df3125f94d2fe56b5a3630c6)
    (cherry picked from commit 18623a0eebe223c0208d70310a8d0308460bd355)
    (cherry picked from commit 807b10f8dbd45af7ea5d11204b1a1bb2e5a2a9a3)
    (cherry picked from commit 017c0c42d330ad870187f48a7f469599746051f9)

tags: added: in-stable-rocky
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (stable/queens)

Fix proposed to branch: stable/queens
Review: https://review.opendev.org/753479

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to manila (stable/queens)

Reviewed: https://review.opendev.org/753479
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=c5eb4b9aa189fd849b2116686be732d17522525e
Submitter: Zuul
Branch: stable/queens

commit c5eb4b9aa189fd849b2116686be732d17522525e
Author: linpeiwen <email address hidden>
Date: Sat Sep 5 09:42:31 2020 +0000

    [Glusterfs] Fix delete share, Couldn't find the 'gluster_used_vols'

    When we have multiple share driver backends, create a shared instance,
    and the list of'self.gluster_used_vols' will only be updated on the
    current node. If the RPC request to delete the share instance is sent
    to other nodes,'self.gluster_used_vols' will be Cannot find the
    information of the glusterfs volume we want to delete, so we need to
    update'self.gluster_used_vols' when deleting the instance

    Change-Id: I14835f6c54376737b41cbf78c94908ea1befde15
    Closes-Bug: #1894362
    (cherry picked from commit 41b0b95ef67e6c65df3125f94d2fe56b5a3630c6)
    (cherry picked from commit 18623a0eebe223c0208d70310a8d0308460bd355)
    (cherry picked from commit 807b10f8dbd45af7ea5d11204b1a1bb2e5a2a9a3)
    (cherry picked from commit 017c0c42d330ad870187f48a7f469599746051f9)
    (cherry picked from commit 79b837ad6c0b761d6069da299c4c39ecd44dabf2)

tags: added: in-stable-queens
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/manila 9.1.5

This issue was fixed in the openstack/manila 9.1.5 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/manila 10.0.2

This issue was fixed in the openstack/manila 10.0.2 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to manila (stable/train)

Reviewed: https://review.opendev.org/c/openstack/manila/+/770593
Committed: https://opendev.org/openstack/manila/commit/8f5a3f4ffbe1350ebebaa46319f29e75c2375da8
Submitter: "Zuul (22348)"
Branch: stable/train

commit 8f5a3f4ffbe1350ebebaa46319f29e75c2375da8
Author: Goutham Pacha Ravi <email address hidden>
Date: Tue Nov 24 15:32:11 2020 -0800

    [glusterfs] don't reinit volume list on deletion

    We don't need to re-initialize the volumes list
    on deletion, it still makes sense to add a missing
    volume to the list, going by the reasoning defined
    in I14835f6c54376737b41cbf78c94908ea1befde15

    Related-Bug: #1894362
    Change-Id: I96d49f84122a34701328909c929ede4d66746911
    (cherry picked from commit 914d87377479ab3951641d658fc461ad2f338ba0)
    (cherry picked from commit 2ad9bcc1b42819f7a8c7aa7200e6f8d128294205)
    (cherry picked from commit 312e4d16652383e77d3764777f531ecba4f3f0c8)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/manila queens-eol

This issue was fixed in the openstack/manila queens-eol release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/manila rocky-eol

This issue was fixed in the openstack/manila rocky-eol release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.