Parallel object auditor

[OpenStack Infra]

https://review.openstack.org/59778
commit d317888a7eae276ae9dddf26a9030d01f6ba00fe
Author: Eamonn O'Toole <email address hidden>
Date: Wed Mar 26 16:32:07 2014 +0000

    Parallel object auditor

    We are soon going to put servers with a high ratio of disk to CPU
    into production as object servers. One of our concerns with this
    configuration is that the object auditor would take too long to
    complete its audit cycle. Therefore we decided to parallelise
    the auditor.

    The auditor already uses fork(), so we decided to use the parallel
    model from the replicator. Concurrency is set by the concurrency
    parameter in the auditor stanza, which sets the number of parallel
    checksum auditors. The actual number of parallel auditing processes
    is concurrency + 1 if zero_byte_fps is non-zero.

    Only one ZBF process is forked, and a new ZBF process is forked as
    soon as the current ZBF process finishes. Thus the last process
    running will always be a ZBF process.

    Both forever and once modes are parallelised.

    Each checksum auditor process submits a nested dictionary with keys
    {'object_auditor_stats_ALL': {'diskn': {..}}} to dump_recon_cache
    so that the object_auditor_stats_ALL dict in recon cache consists
    of individual sub-dicts for each of the object disks on the server.
    The recon cache is no different to before when the checksum auditor
    is run in serial mode. When swift-recon is run, it sums the stats
    for the individual disks.

    DocImpact

    Change-Id: I0ce3db57a43e482d4be351cc522fc9060af6e2d3

[Alexandra Settle (alexandra-settle)]

This has been documented in the Swift dev docs and currently there is no real need for this to be documented in the OS docs.