Comment 6 for bug 1554332

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/mitaka)

Reviewed: https://review.openstack.org/314317
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=3ab2ada748a5f66c3f1dd717223a2d180f686c89
Submitter: Jenkins
Branch: stable/mitaka

commit 3ab2ada748a5f66c3f1dd717223a2d180f686c89
Author: Kevin Benton <email address hidden>
Date: Tue Feb 16 01:50:23 2016 -0800

    Add exponential back-off RPC client

    This adds an exponential backoff mechanism for timeout values
    on any RPC calls in Neutron that don't explicitly request a timeout
    value. This will prevent the clients from DDoSing the server by
    giving up on requests and retrying them before they are fulfilled.

    Each RPC call method in each namespace gets its own timeout value since
    some calls are expected to be much more expensive than others and we
    don't want to modify the timeouts of cheap calls.

    The backoff currently has no reduction mechanism under the assumption
    that timeouts not legitimately caused by heavy system load
    (i.e. messages completely dropped by AMQP) are rare enough that the
    cost of shrinking the timeout back down and potentially causing
    another server timeout isn't worth it. The timeout does have a ceiling
    of 10 times the configured default timeout value.

    Whenever a timeout exception occurs, the client will also sleep for a
    random value between 0 and the configured default timeout value to
    introduce a splay across all of the agents that may be trying to
    communicate with the server.

    This patch is intended to be uninvasive for candidacy to be
    back-ported. A larger refactor of delivering data to the agents
    is being discussed in I3af200ad84483e6e1fe619d516ff20bc87041f7c.

    Closes-Bug: #1554332
    Change-Id: I923e415c1b8e9a431be89221c78c14f39c42c80f
    (cherry picked from commit 3e668b6a3720c1509ffef4ad5b91b4242dfd47b3)