remotefs mount can fail when multiple threads race

Bug #1780813 reported by Eric Harney
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
os-brick
Fix Released
Undecided
Unassigned

Bug Description

If multiple threads try to mount the same share, one can fail with an "already mounted" error.

This should be ignored and considered a success instead.

Revision history for this message
Eric Harney (eharney) wrote :
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to os-brick (master)

Reviewed: https://review.openstack.org/575160
Committed: https://git.openstack.org/cgit/openstack/os-brick/commit/?id=2f32c98a650910fd837fd592daa6c5e0e5458fcd
Submitter: Zuul
Branch: master

commit 2f32c98a650910fd837fd592daa6c5e0e5458fcd
Author: Eric Harney <email address hidden>
Date: Wed Jun 13 12:08:00 2018 -0400

    RemoteFS: don't fail in do_mount if already mounted

    This fixes a problem where two different callers are
    mounting a share at the same time:
        A: check if share is mounted
        B: check if share is mounted
        A: mount share
        B: mount share (fails because it is already mounted)

    Closes-Bug: 1780813
    Change-Id: I6eca9c9e9eceb7aef357b0ed77bf157ac0010bfe

Changed in os-brick:
status: New → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/os-brick 2.5.3

This issue was fixed in the openstack/os-brick 2.5.3 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to os-brick (stable/queens)

Reviewed: https://review.openstack.org/583237
Committed: https://git.openstack.org/cgit/openstack/os-brick/commit/?id=dc5935df1088d199ce987608bd866b50a0cdb5e7
Submitter: Zuul
Branch: stable/queens

commit dc5935df1088d199ce987608bd866b50a0cdb5e7
Author: Eric Harney <email address hidden>
Date: Wed Jun 13 12:08:00 2018 -0400

    RemoteFS: don't fail in do_mount if already mounted

    This fixes a problem where two different callers are
    mounting a share at the same time:
        A: check if share is mounted
        B: check if share is mounted
        A: mount share
        B: mount share (fails because it is already mounted)

    Closes-Bug: 1780813
    Change-Id: I6eca9c9e9eceb7aef357b0ed77bf157ac0010bfe
    (cherry picked from commit 2f32c98a650910fd837fd592daa6c5e0e5458fcd)

tags: added: in-stable-queens
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to os-brick (stable/pike)

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/589509

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/os-brick 2.3.3

This issue was fixed in the openstack/os-brick 2.3.3 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to os-brick (stable/pike)

Reviewed: https://review.openstack.org/589509
Committed: https://git.openstack.org/cgit/openstack/os-brick/commit/?id=544c26adcca167aea4e0dfd8193a62b23d1eb7c1
Submitter: Zuul
Branch: stable/pike

commit 544c26adcca167aea4e0dfd8193a62b23d1eb7c1
Author: Eric Harney <email address hidden>
Date: Wed Jun 13 12:08:00 2018 -0400

    RemoteFS: don't fail in do_mount if already mounted

    This fixes a problem where two different callers are
    mounting a share at the same time:
        A: check if share is mounted
        B: check if share is mounted
        A: mount share
        B: mount share (fails because it is already mounted)

    Closes-Bug: 1780813
    Change-Id: I6eca9c9e9eceb7aef357b0ed77bf157ac0010bfe
    (cherry picked from commit 2f32c98a650910fd837fd592daa6c5e0e5458fcd)
    (cherry picked from commit dc5935df1088d199ce987608bd866b50a0cdb5e7)

tags: added: in-stable-pike
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/os-brick 1.15.6

This issue was fixed in the openstack/os-brick 1.15.6 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to os-brick (stable/rocky)

Related fix proposed to branch: stable/rocky
Review: https://review.openstack.org/623035

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to os-brick (master)

Reviewed: https://review.openstack.org/609543
Committed: https://git.openstack.org/cgit/openstack/os-brick/commit/?id=360a20c7c7b4c385255200a574edc0ca1ac8d87c
Submitter: Zuul
Branch: master

commit 360a20c7c7b4c385255200a574edc0ca1ac8d87c
Author: Eric Harney <email address hidden>
Date: Wed Oct 10 16:46:01 2018 -0400

    Fix NFS "already mounted" detection

    Previous fix

    2f32c98a RemoteFS: don't fail in do_mount if already mounted

    was made with the assumption that "already mounted" appearing
    in stderr from an NFS mount means that the NFS share was already
    mounted.

    However, the NFS client can fail with "busy or already mounted"
    in cases where it fails to mount the share as well.

    This results in the Cinder NFS backup driver believing that the
    NFS mount has succeeded when it didn't, and as a result, data
    can be written to the local disk instead of the NFS target.

    Fail if the share isn't actually mounted rather than succeeding.

    Related-Bug: #1780813
    Closes-Bug: #1797233
    Change-Id: Iebd1afb3340fcaeb1969784966c4f9be35a28417

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to os-brick (stable/queens)

Related fix proposed to branch: stable/queens
Review: https://review.openstack.org/623238

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to os-brick (stable/rocky)

Reviewed: https://review.openstack.org/623035
Committed: https://git.openstack.org/cgit/openstack/os-brick/commit/?id=4c94477cb525b25f178d52ff1b01bfe1fc3d3dd3
Submitter: Zuul
Branch: stable/rocky

commit 4c94477cb525b25f178d52ff1b01bfe1fc3d3dd3
Author: Eric Harney <email address hidden>
Date: Wed Oct 10 16:46:01 2018 -0400

    Fix NFS "already mounted" detection

    Previous fix

    2f32c98a RemoteFS: don't fail in do_mount if already mounted

    was made with the assumption that "already mounted" appearing
    in stderr from an NFS mount means that the NFS share was already
    mounted.

    However, the NFS client can fail with "busy or already mounted"
    in cases where it fails to mount the share as well.

    This results in the Cinder NFS backup driver believing that the
    NFS mount has succeeded when it didn't, and as a result, data
    can be written to the local disk instead of the NFS target.

    Fail if the share isn't actually mounted rather than succeeding.

    Related-Bug: #1780813
    Closes-Bug: #1797233
    Change-Id: Iebd1afb3340fcaeb1969784966c4f9be35a28417
    (cherry picked from commit 360a20c7c7b4c385255200a574edc0ca1ac8d87c)

tags: added: in-stable-rocky
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to os-brick (stable/queens)

Reviewed: https://review.openstack.org/623238
Committed: https://git.openstack.org/cgit/openstack/os-brick/commit/?id=30d7bc124e564d4a2c288f3666d8d14dc96fb9f5
Submitter: Zuul
Branch: stable/queens

commit 30d7bc124e564d4a2c288f3666d8d14dc96fb9f5
Author: Eric Harney <email address hidden>
Date: Wed Oct 10 16:46:01 2018 -0400

    Fix NFS "already mounted" detection

    Previous fix

    2f32c98a RemoteFS: don't fail in do_mount if already mounted

    was made with the assumption that "already mounted" appearing
    in stderr from an NFS mount means that the NFS share was already
    mounted.

    However, the NFS client can fail with "busy or already mounted"
    in cases where it fails to mount the share as well.

    This results in the Cinder NFS backup driver believing that the
    NFS mount has succeeded when it didn't, and as a result, data
    can be written to the local disk instead of the NFS target.

    Fail if the share isn't actually mounted rather than succeeding.

    Related-Bug: #1780813
    Closes-Bug: #1797233
    Change-Id: Iebd1afb3340fcaeb1969784966c4f9be35a28417
    (cherry picked from commit 360a20c7c7b4c385255200a574edc0ca1ac8d87c)
    (cherry picked from commit 4c94477cb525b25f178d52ff1b01bfe1fc3d3dd3)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to os-brick (stable/pike)

Related fix proposed to branch: stable/pike
Review: https://review.openstack.org/647766

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to os-brick (stable/pike)

Reviewed: https://review.openstack.org/647766
Committed: https://git.openstack.org/cgit/openstack/os-brick/commit/?id=f64ad529bfd836a6e18199c3b7d22ff4e07be2c5
Submitter: Zuul
Branch: stable/pike

commit f64ad529bfd836a6e18199c3b7d22ff4e07be2c5
Author: Eric Harney <email address hidden>
Date: Wed Oct 10 16:46:01 2018 -0400

    Fix NFS "already mounted" detection

    Previous fix

    2f32c98a RemoteFS: don't fail in do_mount if already mounted

    was made with the assumption that "already mounted" appearing
    in stderr from an NFS mount means that the NFS share was already
    mounted.

    However, the NFS client can fail with "busy or already mounted"
    in cases where it fails to mount the share as well.

    This results in the Cinder NFS backup driver believing that the
    NFS mount has succeeded when it didn't, and as a result, data
    can be written to the local disk instead of the NFS target.

    Fail if the share isn't actually mounted rather than succeeding.

    Related-Bug: #1780813
    Closes-Bug: #1797233
    Change-Id: Iebd1afb3340fcaeb1969784966c4f9be35a28417
    (cherry picked from commit 360a20c7c7b4c385255200a574edc0ca1ac8d87c)
    (cherry picked from commit 4c94477cb525b25f178d52ff1b01bfe1fc3d3dd3)
    (cherry picked from commit 30d7bc124e564d4a2c288f3666d8d14dc96fb9f5)

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.