control bus sends 3 hb message in one loop

[Masahito Muroi]

Control bus service sends three heartbeat message in one heartbeat loop. It sometimes causes unexpected high-load performance impacts.

How to reproduce:
1. change HEARTBEAT_INTREVAL in control_bus.py[1] to 5 or more than 5 to easily find additional heartbeat rpc log
2. start congress process

1. https://github.com/openstack/congress/blob/master/congress/dse2/control_bus.py#L101

[Eric K (ekcs)]

Wow strange. Is it always 3 or just sometimes?

[Masahito Muroi (muroi-masahito)]

It happens always. There seems to be 3 heartbeat threads in control bus.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/371012

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/371012
Committed: https://git.openstack.org/cgit/openstack/congress/commit/?id=3785f810b5c59ff2ad2666de794c067e3733c90e
Submitter: Jenkins
Branch: master

commit 3785f810b5c59ff2ad2666de794c067e3733c90e
Author: Masahito Muroi <email address hidden>
Date: Fri Sep 16 02:20:35 2016 +0900

    control_bus service has only 1 heartbeat thread

    control_service class expects to have only one heartbeat thread.
    But the service launchs a heartbeat thread everytime when
    control_bus.start() is called. Thus control_bus has 3 methods
    and sends 3 heartbeat messages always.

    This adds a barrier to stop creating extra heartbeat threads in the class.

    Change-Id: I28f06eef3548e07367a3c3757892112a87391ce5
    Closes-Bug: #1622865

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/379059

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/379059
Committed: https://git.openstack.org/cgit/openstack/congress/commit/?id=d32cb5ae2e248c1fb5c461d21b59e5000e8bb690
Submitter: Jenkins
Branch: stable/newton

commit d32cb5ae2e248c1fb5c461d21b59e5000e8bb690
Author: Masahito Muroi <email address hidden>
Date: Fri Sep 16 02:20:35 2016 +0900

    control_bus service has only 1 heartbeat thread

    control_service class expects to have only one heartbeat thread.
    But the service launchs a heartbeat thread everytime when
    control_bus.start() is called. Thus control_bus has 3 methods
    and sends 3 heartbeat messages always.

    This adds a barrier to stop creating extra heartbeat threads in the class.

    Change-Id: I28f06eef3548e07367a3c3757892112a87391ce5
    Closes-Bug: #1622865
    (cherry picked from commit 3785f810b5c59ff2ad2666de794c067e3733c90e)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/congress 4.0.0.0rc2 release candidate.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/congress 5.0.0.0b1 development milestone.