Fix calculation of erlang thread pool size for rabbitmq

[Alexey Lebedeff]

Detailed bug description:
  https://bugs.launchpad.net/fuel/+bug/1472793 introduced automated guessing of erlang thread pool size for rabbit. But rabbitmq upstream is currently using another algorithm for doing this calculation - https://github.com/rabbitmq/rabbitmq-server/issues/151

  rabbit uses separate param RABBITMQ_IO_THREAD_POOL_SIZE to control this variable, but fuel controls the same value through RABBITMQ_SERVER_ERL_ARGS. As a result erlang runtime receives two contradictory values for pool size, but I'm not sure which of them is actually used.

  We need:
  - Switch fuel to using RABBITMQ_SERVER_ERL_ARGS (it'll prevent duplicate args and override rabbitmq default calculated thread pool size)
  - Maybe update defaults calculation in fuel, so they'll be the same

Steps to reproduce:

Run following command on a controller and observe that erlang runtime was passed 2 '-A' options:

ps ax | grep -- '-sname rabbit' | perl -nE 'for (/-A\s*\d+/g) { say } '
-A 64
-A18

Expected results:
Only one '-A' option for beam.smp, with value calculated according to chosen algorithm

Impact:
  Machine with big number of CPU cores may be underutilized by rabbit

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/332234
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=0c16cbc9a723edfc7a9edd08faac8f5fc1b974bf
Submitter: Jenkins
Branch: master

commit 0c16cbc9a723edfc7a9edd08faac8f5fc1b974bf
Author: Alexey Lebedeff <email address hidden>
Date: Tue Jun 21 18:58:51 2016 +0300

    Remove erlang/rabbit thread pool calculation

    Proper version of this calculation is implemented upstream -
    https://github.com/rabbitmq/rabbitmq-server/issues/151

    And removed version of the code is actually harmful, as it's using
    physicalprocessorcount (i.e. number of CPU sockets) for
    calculations. So on a 2 CPU/48 thread system there it was 30 threads
    instead of 768 calculated upstream.

    I've decided that it's easier to completely remove this code instead
    of duplicating current formula in upstream and providing a way to
    override this value through hiera - just not worth a hassle.

    Change-Id: I415d446af0a822d2a5ce3478fd9db1dd0f13e115
    Closes-Bug: 1573696

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/8.0
Review: https://review.openstack.org/396677

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/7.0
Review: https://review.openstack.org/396686

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/403740

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/403740
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=63d73db1164da00d032759734a2acb49d95e984e
Submitter: Jenkins
Branch: stable/mitaka

commit 63d73db1164da00d032759734a2acb49d95e984e
Author: Alexey Lebedeff <email address hidden>
Date: Tue Jun 21 18:58:51 2016 +0300

    Remove erlang/rabbit thread pool calculation

    Proper version of this calculation is implemented upstream -
    https://github.com/rabbitmq/rabbitmq-server/issues/151

    And removed version of the code is actually harmful, as it's using
    physicalprocessorcount (i.e. number of CPU sockets) for
    calculations. So on a 2 CPU/48 thread system there it was 30 threads
    instead of 768 calculated upstream.

    I've decided that it's easier to completely remove this code instead
    of duplicating current formula in upstream and providing a way to
    override this value through hiera - just not worth a hassle.

    Change-Id: I415d446af0a822d2a5ce3478fd9db1dd0f13e115
    Closes-Bug: 1573696
    (cherry picked from commit 0c16cbc9a723edfc7a9edd08faac8f5fc1b974bf)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/396677
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=7fa632dfdfc53a1381cde2b7af634cee6479483d
Submitter: Jenkins
Branch: stable/8.0

commit 7fa632dfdfc53a1381cde2b7af634cee6479483d
Author: Alexey Lebedeff <email address hidden>
Date: Fri Nov 11 18:41:34 2016 +0300

    Remove erlang/rabbit thread pool calculation

    Proper version of this calculation is implemented upstream -
    https://github.com/rabbitmq/rabbitmq-server/issues/151

    And removed version of the code is actually harmful, as it's using
    physicalprocessorcount (i.e. number of CPU sockets) for
    calculations. So on a 2 CPU/48 thread system there it was 30 threads
    instead of 768 calculated upstream.

    I've decided that it's easier to completely remove this code instead
    of duplicating current formula in upstream and providing a way to
    override this value through hiera - just not worth a hassle.

    Change-Id: I415d446af0a822d2a5ce3478fd9db1dd0f13e115
    Closes-Bug: 1573696

[TatyanaGladysheva (tgladysheva)]

Verified on 8.0 + MU4 updates.

Before the fix:
root@node-1:~# ps ax | grep -- '-sname rabbit' | perl -nE 'for (/-A\s*\d+/g) { say } '
-A 64
-A4

After the fix:
root@node-1:~# ps ax | grep -- '-sname rabbit' | perl -nE 'for (/-A\s*\d+/g) { say } '
-A 64

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/396686
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=2858871ff02b0637e5df9e17ce26f8217cf38071
Submitter: Jenkins
Branch: stable/7.0

commit 2858871ff02b0637e5df9e17ce26f8217cf38071
Author: Alexey Lebedeff <email address hidden>
Date: Fri Nov 11 19:20:23 2016 +0300

    Remove erlang/rabbit thread pool calculation

    Proper version of this calculation is implemented upstream -
    https://github.com/rabbitmq/rabbitmq-server/issues/151

    And removed version of the code is actually harmful, as it's using
    physicalprocessorcount (i.e. number of CPU sockets) for
    calculations. So on a 2 CPU/48 thread system there it was 30 threads
    instead of 768 calculated upstream.

    I've decided that it's easier to completely remove this code instead
    of duplicating current formula in upstream and providing a way to
    override this value through hiera - just not worth a hassle.

    Change-Id: I415d446af0a822d2a5ce3478fd9db1dd0f13e115
    Closes-Bug: 1573696

[TatyanaGladysheva (tgladysheva)]

Verified on 7.0 + MU8 updates.

After the fix:
root@node-4:~# ps ax | grep -- '-sname rabbit' | perl -nE 'for (/-A\s*\d+/g) { say } '
-A 64