empty db hash/suffix directories can pile up

[clayg]

It seems that if a db gets quarantined (or possibly by some failure in delete_db locktimeout or OSError) we may fail to cleanup an empty suffix dir (or *maybe* a hashdir).

I've only observed empty suffixes, and was able to reproduce it by creating a bogus db entry that the reqlicator quarantined:

May 19 10:39:50 swift-test-02 container-replicator: Quarantining DB /srv/node/d4/containers/181181/c5d/912ec803b2ce49e4a541068d495abc5d/912ec803b2ce49e4a541068d495a
bc5d.db

This results in an empty suffix dir:

swiftstack@swift-test-02:/srv/node/d4$ find containers/
containers/
containers/181181
containers/181181/c5d

The empty suffix dir never get's cleaned up.

FWIW I think it's reasonable for quarantine to stay out of directories above the hashdir - but the replicator needs to clean these up similarly to the solution in lp bug #1396152

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/331601

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/331601
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=321bb9145f9e2e184994e5d0cf78c62b6043112c
Submitter: Jenkins
Branch: master

commit 321bb9145f9e2e184994e5d0cf78c62b6043112c
Author: Mahati Chamarthy <email address hidden>
Date: Mon Jun 20 15:56:15 2016 +0530

    remove empty db hash and suffix directories

    If a db gets quarantined we may fail to cleanup an empty suffix dir or
    a hash dir.

    Change-Id: I721fa5fe9a7ae22eead8d5141f93e116847ca058
    Closes-Bug: #1583719

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/swift 2.12.0 release.