Comment 10 for bug 1846467

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to kolla-ansible (stable/victoria)

Reviewed: https://review.opendev.org/c/openstack/kolla-ansible/+/799238
Committed: https://opendev.org/openstack/kolla-ansible/commit/21cef390e7e5e152f05c311fc5df38030c8027fa
Submitter: "Zuul (22348)"
Branch: stable/victoria

commit 21cef390e7e5e152f05c311fc5df38030c8027fa
Author: John Garbutt <email address hidden>
Date: Mon Apr 27 10:59:06 2020 +0100

    Reduce RabbitMQ busy waiting, lowering CPU load

    On machines with many cores, we were seeing excessive CPU load on systems
    that were not very busy. With the following Erlang VM argument we saw
    RabbitMQ CPU usage drop from about 150% to around 20%, on a system with
    40 hyperthreads.

        +S 2:2

    By default RabbitMQ starts N schedulers where N is the number of CPU
    cores, including hyper-threaded cores. This is fine when you assume all
    your CPUs are dedicated to RabbitMQ. Its not a good idea in a typical
    Kolla Ansible setup. Here we go for two scheduler threads.
    More details can be found here:
    https://www.rabbitmq.com/runtime.html#scheduling
    and here:
    https://erlang.org/doc/man/erl.html#emulator-flags

        +sbwt none

    This stops busy waiting of the scheduler, for more details see:
    https://www.rabbitmq.com/runtime.html#busy-waiting
    Newer versions of rabbit may need additional flags:
    "+sbwt none +sbwtdcpu none +sbwtdio none"
    But this patch should be back portable to older versions of RabbitMQ
    used in Train and Stein.

    Note that information on this tuning was found by looking at data from:
    rabbitmq-diagnostics runtime_thread_stats
    More details on that can be found here:
    https://www.rabbitmq.com/runtime.html#thread-stats

    Related-Bug: #1846467

    Change-Id: Iced014acee7e590c10848e73feca166f48b622dc
    (cherry picked from commit 70f6f8e4c02ed6a8687d2bd714d3fe0b9d04d84a)