Comment 10 for bug 1563089

Revision history for this message
Jorge Niedbalski (niedbalski) wrote :

Hello,

I ran the verification for the Trusty version.

root@juju-niedbalski-sec-machine-15:/home/ubuntu# dpkg -l|grep corosync
ii corosync 2.3.3-1ubuntu3 amd64 Standards-based cluster framework (daemon and modules)
ii libcorosync-common4 2.3.3-1ubuntu3 amd64 Standards-based cluster framework, common library

I configured a 3 nodes nova-cloud-controller environment related with hacluster.

ubuntu@niedbalski-sec-bastion:~/openstack-charm-testing/bundles/dev$ juju run --service nova-cloud-controller "sudo corosync-quorumtool -s|grep votes"
- MachineId: "15"
  Stdout: |
    Expected votes: 3
    Total votes: 3
  UnitId: nova-cloud-controller/0
- MachineId: "28"
  Stdout: |
    Expected votes: 3
    Total votes: 3
  UnitId: nova-cloud-controller/1
- MachineId: "29"
  Stdout: |
    Expected votes: 3
    Total votes: 3
  UnitId: nova-cloud-controller/2

I changed the transport mode to UDP by setting:

$ juju set hacluster-ncc corosync_transport=udpu

After this, I moved to the primary node (the one that holds the virtual ip address) and I applied the TC
rules, while monitoring the memory usage of the corosync process (multiple times)

root@juju-niedbalski-sec-machine-15:/home/ubuntu# tc qdisc add dev eth0 root netem delay 550ms
root@juju-niedbalski-sec-machine-15:/home/ubuntu# tc qdisc del dev eth0 root netem

Apr 6 17:57:37 juju-niedbalski-sec-machine-15 cib[14387]: warning: cib_process_request: Completed cib_apply_diff operation for section 'all': Application of an update diff failed (rc=-206, origin=local/cibadmin/2, version=0.27.1)
Apr 6 18:04:12 juju-niedbalski-sec-machine-15 corosync[14376]: [MAIN ] Completed service synchronization, ready to provide service.
Apr 6 18:04:13 juju-niedbalski-sec-machine-15 corosync[18645]: [MAIN ] Completed service synchronization, ready to provide service.
Apr 6 18:06:27 juju-niedbalski-sec-machine-15 corosync[18645]: [MAIN ] Completed service synchronization, ready to provide service.
Apr 6 18:06:28 juju-niedbalski-sec-machine-15 corosync[19528]: [MAIN ] Completed service synchronization, ready to provide service.
Apr 6 18:07:48 juju-niedbalski-sec-machine-15 corosync[19985]: [MAIN ] Completed service synchronization, ready to provide service.
Apr 6 18:07:49 juju-niedbalski-sec-machine-15 corosync[19985]: [MAIN ] Completed service synchronization, ready to provide service.
Apr 6 18:08:16 juju-niedbalski-sec-machine-15 corosync[19985]: [MAIN ] Completed service synchronization, ready to provide service.
Apr 6 18:08:59 juju-niedbalski-sec-machine-15 corosync[19985]: [MAIN ] Completed service synchronization, ready to provide service.
Apr 6 18:09:38 juju-niedbalski-sec-machine-15 corosync[19985]: [MAIN ] Completed service synchronization, ready to provide
service.

After 5 minutes of observation on the corosync process by using:

 $ sudo while true; do ps -o vsz,rss -p $(pgrep corosync) 2>&1 | grep -E '.*[0-9]+.*' | tee -a memory-usage.log && sleep 1; done

I don't see any substantial memory usage increase.

root@juju-niedbalski-sec-machine-15:/home/ubuntu# more memory-usage.log
135584 3928
135584 3928
135584 3928
135584 3928
135584 3928
135584 3928
135584 3928
135584 3928
135584 3928
135584 3928
135584 3928
135584 3928
135584 3928
135584 3928
135584 3928
135584 3928
135584 3928
135584 3928
135584 3928
135584 3928