OpenStack Shared File Systems Service (Manila)

[Generic driver] CIFS extension failing because of volume in use thus unable to unmount

Bug #1703581 reported by Jan Vondra
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack Shared File Systems Service (Manila)
Fix Released
Medium
Goutham Pacha Ravi
OpenStack Shared File Systems Service (Manila) victoria-1

Bug Description

Unmouting of the volume from service instance sometimes fails because Windows clients tends to check the availability of shares even if not mounted and the volume is in use then. The solution is to find out the processes via smbstatus and kill them before unmounting.

Revision history for this message
Jan Vondra (janvondra) wrote :
Jan Vondra (janvondra)
Changed in manila:
assignee: nobody → Jan Vondra (janvondra)
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (master)

Fix proposed to branch: master
Review: https://review.openstack.org/498445

Revision history for this message
Thomas Bechtold (toabctl) wrote :

Would it be possible to get a stacktrace of the error?

Changed in manila:
status: In Progress → Incomplete
Revision history for this message
Jan Vondra (janvondra) wrote :

Well, the stacktrace is pretty useless, because it's wrapped into timeout exception:

2017-09-21 15:12:46.684 30033 ERROR oslo_messaging.rpc.server [req-f9d5ed41-1dd5-4757-9341-d4243d05385c 73979cf70cac4f8995e65bdd833765ac d1c044d0731d42af942c165fc4c801a8 - - -] Exception during message handling: ShareExtendingError: Share 51aeabaa-6471-465e-8e51-0d7f108beaae could not be extended due to error in the driver: Volume has not been extended in 180s. Giving up
2017-09-21 15:12:46.684 30033 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2017-09-21 15:12:46.684 30033 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming
2017-09-21 15:12:46.684 30033 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
2017-09-21 15:12:46.684 30033 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch
2017-09-21 15:12:46.684 30033 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
2017-09-21 15:12:46.684 30033 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch
2017-09-21 15:12:46.684 30033 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
2017-09-21 15:12:46.684 30033 ERROR oslo_messaging.rpc.server File "/opt/stack/manila/manila/share/manager.py", line 186, in wrapped
2017-09-21 15:12:46.684 30033 ERROR oslo_messaging.rpc.server return f(self, *args, **kwargs)
2017-09-21 15:12:46.684 30033 ERROR oslo_messaging.rpc.server File "/opt/stack/manila/manila/utils.py", line 560, in wrapper
2017-09-21 15:12:46.684 30033 ERROR oslo_messaging.rpc.server return func(self, *args, **kwargs)
2017-09-21 15:12:46.684 30033 ERROR oslo_messaging.rpc.server File "/opt/stack/manila/manila/share/manager.py", line 3544, in extend_share
2017-09-21 15:12:46.684 30033 ERROR oslo_messaging.rpc.server reason=six.text_type(e), share_id=share_id)
2017-09-21 15:12:46.684 30033 ERROR oslo_messaging.rpc.server ShareExtendingError: Share 51aeabaa-6471-465e-8e51-0d7f108beaae could not be extended due to error in the driver: Volume has not been extended in 180s. Giving up
2017-09-21 15:12:46.684 30033 ERROR oslo_messaging.rpc.server

Further investigation reveals the reason - the samba processes holding volume making it unavailable to unmount.

OpenStack Infra (hudson-openstack)
Changed in manila:
assignee: Jan Vondra (janvondra) → Tom Barron (tpb)
status: Incomplete → In Progress
Revision history for this message
Jason Grosso (jgrosso) wrote :

Jan are you still working on this issue?

OpenStack Infra (hudson-openstack)
Changed in manila:
assignee: Tom Barron (tpb) → Goutham Pacha Ravi (gouthamr)
Goutham Pacha Ravi (gouthamr)
Changed in manila:
importance: Undecided → Medium
assignee: Goutham Pacha Ravi (gouthamr) → Jan Vondra (janvondra)
milestone: none → victoria-1
OpenStack Infra (hudson-openstack)
Changed in manila:
assignee: Jan Vondra (janvondra) → Goutham Pacha Ravi (gouthamr)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to manila (master)

Reviewed: https://review.opendev.org/498445
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=2c9fbc75059ac724cd845317473b7df4a8dcdc35
Submitter: Zuul
Branch: master

commit 2c9fbc75059ac724cd845317473b7df4a8dcdc35
Author: Jan Vondra <email address hidden>
Date: Mon Aug 28 14:57:35 2017 +0200

    CIFS extension failing because of volume in use

    Unmouting of the volume from service instance sometimes fails because
    Windows clients tends to check the availability of shares even if not
    mounted and the volume is in use then.

    Fix finds out processes via smbstatus and kill them before unmounting.

    Closes-Bug: 1703581
    Change-Id: I2cbf959d7cb849d3ead1655b1a0932fd428a9e9f
    Signed-off-by: Goutham Pacha Ravi <email address hidden>

Changed in manila:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (stable/ussuri)

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/732059

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to manila (stable/ussuri)

Reviewed: https://review.opendev.org/732059
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=95d251e38b4b4526c862a428da93f568fe7a23ad
Submitter: Zuul
Branch: stable/ussuri

commit 95d251e38b4b4526c862a428da93f568fe7a23ad
Author: Jan Vondra <email address hidden>
Date: Mon Aug 28 14:57:35 2017 +0200

    CIFS extension failing because of volume in use

    Unmouting of the volume from service instance sometimes fails because
    Windows clients tends to check the availability of shares even if not
    mounted and the volume is in use then.

    Fix finds out processes via smbstatus and kill them before unmounting.

    Closes-Bug: 1703581
    Change-Id: I2cbf959d7cb849d3ead1655b1a0932fd428a9e9f
    Signed-off-by: Goutham Pacha Ravi <email address hidden>
    (cherry picked from commit 2c9fbc75059ac724cd845317473b7df4a8dcdc35)

tags: added: in-stable-ussuri
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (stable/train)

Fix proposed to branch: stable/train
Review: https://review.opendev.org/734394

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to manila (stable/train)

Reviewed: https://review.opendev.org/734394
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=b07c2484b6f0d5165d867651fbc123280e7b012d
Submitter: Zuul
Branch: stable/train

commit b07c2484b6f0d5165d867651fbc123280e7b012d
Author: Jan Vondra <email address hidden>
Date: Mon Aug 28 14:57:35 2017 +0200

    CIFS extension failing because of volume in use

    Unmouting of the volume from service instance sometimes fails because
    Windows clients tends to check the availability of shares even if not
    mounted and the volume is in use then.

    Fix finds out processes via smbstatus and kill them before unmounting.

    Closes-Bug: 1703581
    Change-Id: I2cbf959d7cb849d3ead1655b1a0932fd428a9e9f
    Signed-off-by: Goutham Pacha Ravi <email address hidden>
    (cherry picked from commit 2c9fbc75059ac724cd845317473b7df4a8dcdc35)
    (cherry picked from commit 95d251e38b4b4526c862a428da93f568fe7a23ad)

tags: added: in-stable-train
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (stable/stein)

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/737324

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (stable/rocky)

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/738890

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to manila (stable/stein)

Reviewed: https://review.opendev.org/737324
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=a26b027b73669efb2a99530574051d284595b264
Submitter: Zuul
Branch: stable/stein

commit a26b027b73669efb2a99530574051d284595b264
Author: Jan Vondra <email address hidden>
Date: Mon Aug 28 14:57:35 2017 +0200

    CIFS extension failing because of volume in use

    Unmouting of the volume from service instance sometimes fails because
    Windows clients tends to check the availability of shares even if not
    mounted and the volume is in use then.

    Fix finds out processes via smbstatus and kill them before unmounting.

    Closes-Bug: 1703581
    Change-Id: I2cbf959d7cb849d3ead1655b1a0932fd428a9e9f
    Signed-off-by: Goutham Pacha Ravi <email address hidden>
    (cherry picked from commit 2c9fbc75059ac724cd845317473b7df4a8dcdc35)
    (cherry picked from commit 95d251e38b4b4526c862a428da93f568fe7a23ad)
    (cherry picked from commit b07c2484b6f0d5165d867651fbc123280e7b012d)

tags: added: in-stable-stein
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to manila (stable/rocky)

Reviewed: https://review.opendev.org/738890
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=a627a29e7225202475a8cdc927911b3cd80b487c
Submitter: Zuul
Branch: stable/rocky

commit a627a29e7225202475a8cdc927911b3cd80b487c
Author: Jan Vondra <email address hidden>
Date: Mon Aug 28 14:57:35 2017 +0200

    CIFS extension failing because of volume in use

    Unmouting of the volume from service instance sometimes fails because
    Windows clients tends to check the availability of shares even if not
    mounted and the volume is in use then.

    Fix finds out processes via smbstatus and kill them before unmounting.

    Closes-Bug: 1703581
    Change-Id: I2cbf959d7cb849d3ead1655b1a0932fd428a9e9f
    Signed-off-by: Goutham Pacha Ravi <email address hidden>
    (cherry picked from commit 2c9fbc75059ac724cd845317473b7df4a8dcdc35)
    (cherry picked from commit 95d251e38b4b4526c862a428da93f568fe7a23ad)
    (cherry picked from commit b07c2484b6f0d5165d867651fbc123280e7b012d)
    (cherry picked from commit a26b027b73669efb2a99530574051d284595b264)

tags: added: in-stable-rocky
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (stable/queens)

Fix proposed to branch: stable/queens
Review: https://review.opendev.org/743028

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to manila (stable/queens)

Reviewed: https://review.opendev.org/743028
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=48b2c835bba0346df6d06090a12d492196f9c19f
Submitter: Zuul
Branch: stable/queens

commit 48b2c835bba0346df6d06090a12d492196f9c19f
Author: Jan Vondra <email address hidden>
Date: Mon Aug 28 14:57:35 2017 +0200

    CIFS extension failing because of volume in use

    Unmouting of the volume from service instance sometimes fails because
    Windows clients tends to check the availability of shares even if not
    mounted and the volume is in use then.

    Fix finds out processes via smbstatus and kill them before unmounting.

    Closes-Bug: 1703581
    Change-Id: I2cbf959d7cb849d3ead1655b1a0932fd428a9e9f
    Signed-off-by: Goutham Pacha Ravi <email address hidden>
    (cherry picked from commit 2c9fbc75059ac724cd845317473b7df4a8dcdc35)
    (cherry picked from commit 95d251e38b4b4526c862a428da93f568fe7a23ad)
    (cherry picked from commit b07c2484b6f0d5165d867651fbc123280e7b012d)
    (cherry picked from commit a26b027b73669efb2a99530574051d284595b264)
    (cherry picked from commit a627a29e7225202475a8cdc927911b3cd80b487c)

tags: added: in-stable-queens
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/manila queens-eol

This issue was fixed in the openstack/manila queens-eol release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/manila rocky-eol

This issue was fixed in the openstack/manila rocky-eol release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.