sharder fails when drive is unmounted

Bug #1806500 reported by John Dickinson
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Object Storage (swift)
Fix Released
High
Unassigned

Bug Description

The container sharder process fails if a drive is unmounted on the current node. This is because the internal `swift.container.sharder._check_node()` method is not properly handling the response from `check_drive()`. `check_drive()` returns a string if successful and raises a ValueError otherwise. `_check_node()` is simply checking if the response is Falsy and not handling the ValueError at all.

Changed in swift:
importance: Critical → High
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to swift (master)

Reviewed: https://review.openstack.org/621727
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=c26d67efcf44c813034f59b1df78d59fdb4a50e7
Submitter: Zuul
Branch: master

commit c26d67efcf44c813034f59b1df78d59fdb4a50e7
Author: John Dickinson <email address hidden>
Date: Mon Dec 3 14:22:59 2018 -0800

    fixed _check_node() in the container sharder

    Previously, _check_node() wouldn't catch the raise ValueError when
    a drive was unmounted. Therefore the error would bubble up, uncaught,
    and stop the shard cycle. The practical effect is that an unmounted
    drive on a node would prevent sharding for happening.

    This patch updates _check_node() to properly use the check_drive()
    method. Furthermore, the _check_node() return value has been modified
    to be more similar to what check_drive() actually returns. This
    should help prevent similar errors from being introduced in the future.

    Closes-Bug: #1806500

    Change-Id: I3da9b5b120a5980e77ef5c4dc8fa1697e462ce0d

Changed in swift:
status: Confirmed → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to swift (stable/rocky)

Fix proposed to branch: stable/rocky
Review: https://review.openstack.org/624498

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/swift 2.20.0

This issue was fixed in the openstack/swift 2.20.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to swift (stable/rocky)

Reviewed: https://review.openstack.org/624498
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=e237328f651e709e907e2fa4b98d9de4bc4f6e20
Submitter: Zuul
Branch: stable/rocky

commit e237328f651e709e907e2fa4b98d9de4bc4f6e20
Author: John Dickinson <email address hidden>
Date: Mon Dec 3 14:22:59 2018 -0800

    fixed _check_node() in the container sharder

    Previously, _check_node() wouldn't catch the raise ValueError when
    a drive was unmounted. Therefore the error would bubble up, uncaught,
    and stop the shard cycle. The practical effect is that an unmounted
    drive on a node would prevent sharding for happening.

    This patch updates _check_node() to properly use the check_drive()
    method. Furthermore, the _check_node() return value has been modified
    to be more similar to what check_drive() actually returns. This
    should help prevent similar errors from being introduced in the future.

    Closes-Bug: #1806500

    Change-Id: I3da9b5b120a5980e77ef5c4dc8fa1697e462ce0d
    (cherry picked from commit c26d67efcf44c813034f59b1df78d59fdb4a50e7)

tags: added: in-stable-rocky
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/swift 2.19.1

This issue was fixed in the openstack/swift 2.19.1 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.