HA for ceilometer agents/notifier not implemented

[Miguel Alejandro Cantu]

When running multiple central-agents without workload partitioning, each agent will poll the same service and will produce duplicate samples[1].
Currently OSA does not implement the HA solution for the central agent nor the notifier(notifier can do workload partitioning as well). OSA also documents the ability to run HA central agents at the moment[2], which is unsupported with the current implementation.
The fix would be to add the bits necessary to deploy ceilometer central agents/notifier using Tooz with a supported backend solution[1].

[1] http://docs.openstack.org/admin-guide-cloud/telemetry-data-collection.html
[2] https://github.com/openstack/openstack-ansible/blob/master/etc/openstack_deploy/conf.d/ceilometer.yml.example#L7-L13

[Miguel Alejandro Cantu (miguel-cantu)]

One way would be to write OSA to deploy a zookeeper ensemble(muli-server zookeeper cluster). It could use external ansible roles to do so, and it would only need to do so when len(groups['ceilometer_agent_central'] is > 1.
Some thoughts arise, however:
* Do we run the zookeeper services in their own containers?
* If so, we should define a new environment in env.yml. That way the zookeeper services can run on their on infrastructure if need be.
* If we do use external ansible roles, is it appropriate to just add them to ansible-role-requirements.yml?

Maybe we can talk about this during the weekly IRC meetings. I hate having to make these decisions by myself. Zookeeper seems like a pretty hefty application to deploy along side ceilometer, so I want to make sure I'm doing it right.

[Miguel Alejandro Cantu (miguel-cantu)]

For now I could document that OSA does not support workload partitioning they way it is, but if a tooz solution has been implemented ceilometer can be configured with HA using the overrides section

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/248905

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/248905
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible/commit/?id=5371a0eb93ff088e8f26fef698ff7fcae95a58fa
Submitter: Jenkins
Branch: master

commit 5371a0eb93ff088e8f26fef698ff7fcae95a58fa
Author: Miguel Alex Cantu <email address hidden>
Date: Mon Nov 23 20:49:05 2015 +0000

    Add documentation for HA ceilometer

    HA ceilometer can work if the proper overrides
    are configured. This patch adds documentation
    regarding running ceilometer in an active/active
    HA configuration, meaning workload paritioning
    can be done across multiple polling agents and
    notification agents.

    Change-Id: I65402a66f465df8dc82a24460531fba3a24a8cf5
    Closes-Bug: #1518005

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: liberty
Review: https://review.openstack.org/252651

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/252651
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible/commit/?id=6b591d593cc2feae3699dd7d8c321085eba961fd
Submitter: Jenkins
Branch: liberty

commit 6b591d593cc2feae3699dd7d8c321085eba961fd
Author: Miguel Alex Cantu <email address hidden>
Date: Mon Nov 23 20:49:05 2015 +0000

    Add documentation for HA ceilometer

    HA ceilometer can work if the proper overrides
    are configured. This patch adds documentation
    regarding running ceilometer in an active/active
    HA configuration, meaning workload paritioning
    can be done across multiple polling agents and
    notification agents.

    Change-Id: I65402a66f465df8dc82a24460531fba3a24a8cf5
    Closes-Bug: #1518005
    (cherry picked from commit 5371a0eb93ff088e8f26fef698ff7fcae95a58fa)

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/openstack-ansible 12.0.8 release.

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/openstack-ansible 12.0.9 release.

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/openstack-ansible 13.0.0 release.

[Davanum Srinivas (DIMS) (dims-v)]

This issue was fixed in the openstack/openstack-ansible 13.0.0 release.

[Davanum Srinivas (DIMS) (dims-v)]

This issue was fixed in the openstack/openstack-ansible 12.0.11 release.