When deleting the share instance, I get an error'Couldn't find the share in used list'

Bug #1894362 reported by Lin PeiWen
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Shared File Systems Service (Manila)
Fix Released
Undecided
Lin PeiWen

Bug Description

Description
===========
There are three manila share driver services in my environment, and I am configuring'host = glusterfs' to improve high availability and load balancing capabilities, but an error occurs when deleting the share instance

Steps to reproduce
==================

* Deploying Manila using glusterfs driver
* Create a shared instance
* delete the shared instance

Expected result
===============

The manila shared instance was successfully deleted

Actual result
=============
vi /var/log/manila/share.log
2020-09-05 15:14:36.018 19 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/manila/share/manager.py", line 3070, in delete_share_instance
2020-09-05 15:14:36.018 19 ERROR oslo_messaging.rpc.server share_server=share_server)
2020-09-05 15:14:36.018 19 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/manila/share/drivers/glusterfs/layout.py", line 177, in delete_share
2020-09-05 15:14:36.018 19 ERROR oslo_messaging.rpc.server return self.layout.delete_share(*a, **kw)
2020-09-05 15:14:36.018 19 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/manila/share/drivers/glusterfs/layout_volume.py", line 437, in delete_share
2020-09-05 15:14:36.018 19 ERROR oslo_messaging.rpc.server self._push_gluster_vol(gmgr.qualified)
2020-09-05 15:14:36.018 19 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py", line 359, in inner
2020-09-05 15:14:36.018 19 ERROR oslo_messaging.rpc.server return f(*args, **kwargs)
2020-09-05 15:14:36.018 19 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/manila/share/drivers/glusterfs/layout_volume.py", line 334, in _push_gluster_vol
2020-09-05 15:14:36.018 19 ERROR oslo_messaging.rpc.server raise exception.GlusterfsException(msg)
2020-09-05 15:14:36.018 19 ERROR oslo_messaging.rpc.server manila.exception.GlusterfsException: Couldn't find the share in used list.

Environment
===========

1.OpenStack master/rocky

2.Which storage backend did you use? GlusterFS driver

3.glusterfs_nfs_server_type = Gluster

4.glusterfs_share_layout = layout_volume.GlusterfsVolumeMappedLayout

Logs & Configs
==============

# vim /etc/manila/manila.conf
[DEFAULT]
host = glusterfs

Logs error: http://paste.openstack.org/show/797491/

Lin PeiWen (linpeiwen)
Changed in manila:
assignee: nobody → Lin PeiWen (linpeiwen)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (master)

Fix proposed to branch: master
Review: https://review.opendev.org/750067

Changed in manila:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to manila (master)

Reviewed: https://review.opendev.org/750067
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=41b0b95ef67e6c65df3125f94d2fe56b5a3630c6
Submitter: Zuul
Branch: master

commit 41b0b95ef67e6c65df3125f94d2fe56b5a3630c6
Author: linpeiwen <email address hidden>
Date: Sat Sep 5 09:42:31 2020 +0000

    [Glusterfs] Fix delete share, Couldn't find the 'gluster_used_vols'

    When we have multiple share driver backends, create a shared instance,
    and the list of'self.gluster_used_vols' will only be updated on the
    current node. If the RPC request to delete the share instance is sent
    to other nodes,'self.gluster_used_vols' will be Cannot find the
    information of the glusterfs volume we want to delete, so we need to
    update'self.gluster_used_vols' when deleting the instance

    Change-Id: I14835f6c54376737b41cbf78c94908ea1befde15
    Closes-Bug: #1894362

Changed in manila:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (stable/ussuri)

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/752193

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to manila (stable/ussuri)

Reviewed: https://review.opendev.org/752193
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=18623a0eebe223c0208d70310a8d0308460bd355
Submitter: Zuul
Branch: stable/ussuri

commit 18623a0eebe223c0208d70310a8d0308460bd355
Author: linpeiwen <email address hidden>
Date: Sat Sep 5 09:42:31 2020 +0000

    [Glusterfs] Fix delete share, Couldn't find the 'gluster_used_vols'

    When we have multiple share driver backends, create a shared instance,
    and the list of'self.gluster_used_vols' will only be updated on the
    current node. If the RPC request to delete the share instance is sent
    to other nodes,'self.gluster_used_vols' will be Cannot find the
    information of the glusterfs volume we want to delete, so we need to
    update'self.gluster_used_vols' when deleting the instance

    Change-Id: I14835f6c54376737b41cbf78c94908ea1befde15
    Closes-Bug: #1894362
    (cherry picked from commit 41b0b95ef67e6c65df3125f94d2fe56b5a3630c6)

tags: added: in-stable-ussuri
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (stable/train)

Fix proposed to branch: stable/train
Review: https://review.opendev.org/752600

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to manila (stable/train)

Reviewed: https://review.opendev.org/752600
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=807b10f8dbd45af7ea5d11204b1a1bb2e5a2a9a3
Submitter: Zuul
Branch: stable/train

commit 807b10f8dbd45af7ea5d11204b1a1bb2e5a2a9a3
Author: linpeiwen <email address hidden>
Date: Sat Sep 5 09:42:31 2020 +0000

    [Glusterfs] Fix delete share, Couldn't find the 'gluster_used_vols'

    When we have multiple share driver backends, create a shared instance,
    and the list of'self.gluster_used_vols' will only be updated on the
    current node. If the RPC request to delete the share instance is sent
    to other nodes,'self.gluster_used_vols' will be Cannot find the
    information of the glusterfs volume we want to delete, so we need to
    update'self.gluster_used_vols' when deleting the instance

    Change-Id: I14835f6c54376737b41cbf78c94908ea1befde15
    Closes-Bug: #1894362
    (cherry picked from commit 41b0b95ef67e6c65df3125f94d2fe56b5a3630c6)
    (cherry picked from commit 18623a0eebe223c0208d70310a8d0308460bd355)

tags: added: in-stable-train
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (stable/stein)

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/752780

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to manila (stable/stein)

Reviewed: https://review.opendev.org/752780
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=017c0c42d330ad870187f48a7f469599746051f9
Submitter: Zuul
Branch: stable/stein

commit 017c0c42d330ad870187f48a7f469599746051f9
Author: linpeiwen <email address hidden>
Date: Sat Sep 5 09:42:31 2020 +0000

    [Glusterfs] Fix delete share, Couldn't find the 'gluster_used_vols'

    When we have multiple share driver backends, create a shared instance,
    and the list of'self.gluster_used_vols' will only be updated on the
    current node. If the RPC request to delete the share instance is sent
    to other nodes,'self.gluster_used_vols' will be Cannot find the
    information of the glusterfs volume we want to delete, so we need to
    update'self.gluster_used_vols' when deleting the instance

    Change-Id: I14835f6c54376737b41cbf78c94908ea1befde15
    Closes-Bug: #1894362
    (cherry picked from commit 41b0b95ef67e6c65df3125f94d2fe56b5a3630c6)
    (cherry picked from commit 18623a0eebe223c0208d70310a8d0308460bd355)
    (cherry picked from commit 807b10f8dbd45af7ea5d11204b1a1bb2e5a2a9a3)

tags: added: in-stable-stein
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (stable/rocky)

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/753228

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to manila (stable/rocky)

Reviewed: https://review.opendev.org/753228
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=79b837ad6c0b761d6069da299c4c39ecd44dabf2
Submitter: Zuul
Branch: stable/rocky

commit 79b837ad6c0b761d6069da299c4c39ecd44dabf2
Author: linpeiwen <email address hidden>
Date: Sat Sep 5 09:42:31 2020 +0000

    [Glusterfs] Fix delete share, Couldn't find the 'gluster_used_vols'

    When we have multiple share driver backends, create a shared instance,
    and the list of'self.gluster_used_vols' will only be updated on the
    current node. If the RPC request to delete the share instance is sent
    to other nodes,'self.gluster_used_vols' will be Cannot find the
    information of the glusterfs volume we want to delete, so we need to
    update'self.gluster_used_vols' when deleting the instance

    Change-Id: I14835f6c54376737b41cbf78c94908ea1befde15
    Closes-Bug: #1894362
    (cherry picked from commit 41b0b95ef67e6c65df3125f94d2fe56b5a3630c6)
    (cherry picked from commit 18623a0eebe223c0208d70310a8d0308460bd355)
    (cherry picked from commit 807b10f8dbd45af7ea5d11204b1a1bb2e5a2a9a3)
    (cherry picked from commit 017c0c42d330ad870187f48a7f469599746051f9)

tags: added: in-stable-rocky
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (stable/queens)

Fix proposed to branch: stable/queens
Review: https://review.opendev.org/753479

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to manila (stable/queens)

Reviewed: https://review.opendev.org/753479
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=c5eb4b9aa189fd849b2116686be732d17522525e
Submitter: Zuul
Branch: stable/queens

commit c5eb4b9aa189fd849b2116686be732d17522525e
Author: linpeiwen <email address hidden>
Date: Sat Sep 5 09:42:31 2020 +0000

    [Glusterfs] Fix delete share, Couldn't find the 'gluster_used_vols'

    When we have multiple share driver backends, create a shared instance,
    and the list of'self.gluster_used_vols' will only be updated on the
    current node. If the RPC request to delete the share instance is sent
    to other nodes,'self.gluster_used_vols' will be Cannot find the
    information of the glusterfs volume we want to delete, so we need to
    update'self.gluster_used_vols' when deleting the instance

    Change-Id: I14835f6c54376737b41cbf78c94908ea1befde15
    Closes-Bug: #1894362
    (cherry picked from commit 41b0b95ef67e6c65df3125f94d2fe56b5a3630c6)
    (cherry picked from commit 18623a0eebe223c0208d70310a8d0308460bd355)
    (cherry picked from commit 807b10f8dbd45af7ea5d11204b1a1bb2e5a2a9a3)
    (cherry picked from commit 017c0c42d330ad870187f48a7f469599746051f9)
    (cherry picked from commit 79b837ad6c0b761d6069da299c4c39ecd44dabf2)

tags: added: in-stable-queens
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/manila 9.1.5

This issue was fixed in the openstack/manila 9.1.5 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/manila 10.0.2

This issue was fixed in the openstack/manila 10.0.2 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to manila (stable/train)

Reviewed: https://review.opendev.org/c/openstack/manila/+/770593
Committed: https://opendev.org/openstack/manila/commit/8f5a3f4ffbe1350ebebaa46319f29e75c2375da8
Submitter: "Zuul (22348)"
Branch: stable/train

commit 8f5a3f4ffbe1350ebebaa46319f29e75c2375da8
Author: Goutham Pacha Ravi <email address hidden>
Date: Tue Nov 24 15:32:11 2020 -0800

    [glusterfs] don't reinit volume list on deletion

    We don't need to re-initialize the volumes list
    on deletion, it still makes sense to add a missing
    volume to the list, going by the reasoning defined
    in I14835f6c54376737b41cbf78c94908ea1befde15

    Related-Bug: #1894362
    Change-Id: I96d49f84122a34701328909c929ede4d66746911
    (cherry picked from commit 914d87377479ab3951641d658fc461ad2f338ba0)
    (cherry picked from commit 2ad9bcc1b42819f7a8c7aa7200e6f8d128294205)
    (cherry picked from commit 312e4d16652383e77d3764777f531ecba4f3f0c8)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/manila queens-eol

This issue was fixed in the openstack/manila queens-eol release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/manila rocky-eol

This issue was fixed in the openstack/manila rocky-eol release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers