Default concurrency for auditor/replicator/updater is way to high and might overload nodes

[Christian Schwede]

The default concurrency is $::processorcount, and this results in environments where 24 auditors/replicators/reapers processes might all use the same disk, especially if there are only a few disks. This results in a high I/O load, because every thread blocks the others.

These values are 1 by default in Swift. If one has many disks and a powerful machine, a single-digit concurrency for auditors and replicators might work fine; the reaper is most likely only running with a concurrency of 1 in any case.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/383724

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/383724
Committed: https://git.openstack.org/cgit/openstack/puppet-swift/commit/?id=f8d6c083343cd308da1fc68113b22664d276d011
Submitter: Jenkins
Branch: master

commit f8d6c083343cd308da1fc68113b22664d276d011
Author: Christian Schwede <email address hidden>
Date: Fri Oct 7 14:56:37 2016 +0200

    Set concurrency to 1 for auditor/replicator/updater

    Before this the default concurrency was set to $::processorcount, and
    this resulted in environments where 24 auditors/replicators/reapers
    processes were all using the same disk. This results in a high I/O load,
    because every thread blocks the others.

    These values are 1 by default in Swift. If one has many disks and a
    powerful machine, a single-digit concurrency for auditors and
    replicators might work fine; the reaper is most likely only running with
    a concurrency of 1 in any case.

    Let's set this to the default of 1; operators with a need to use a
    higher value can easily override the defaults (and probably want to use
    a value that is appropriate to their setup).

    Closes-Bug: 1631359
    Change-Id: I54a5d01e4bd357973e142274db79d119e9e46457

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/385626

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/385626
Committed: https://git.openstack.org/cgit/openstack/puppet-swift/commit/?id=5b9f67d745970fbe20de1c6938eab3471a112ccb
Submitter: Jenkins
Branch: stable/newton

commit 5b9f67d745970fbe20de1c6938eab3471a112ccb
Author: Christian Schwede <email address hidden>
Date: Fri Oct 7 14:56:37 2016 +0200

    Set concurrency to 1 for auditor/replicator/updater

    Before this the default concurrency was set to $::processorcount, and
    this resulted in environments where 24 auditors/replicators/reapers
    processes were all using the same disk. This results in a high I/O load,
    because every thread blocks the others.

    These values are 1 by default in Swift. If one has many disks and a
    powerful machine, a single-digit concurrency for auditors and
    replicators might work fine; the reaper is most likely only running with
    a concurrency of 1 in any case.

    Let's set this to the default of 1; operators with a need to use a
    higher value can easily override the defaults (and probably want to use
    a value that is appropriate to their setup).

    Closes-Bug: 1631359
    Change-Id: I54a5d01e4bd357973e142274db79d119e9e46457
    (cherry picked from commit f8d6c083343cd308da1fc68113b22664d276d011)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/puppet-swift 9.4.2 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/puppet-swift 10.0.0 release.