ceilometer compute agent gets into bad state after upgrade

[Pradeep Kilambi]

After upgrading compute nodes during upgrade from RHOS-9 to RHOS-10 ceilometer-compute-agent fails to collect data with next message in ceilometer/compute.log:

2016-11-07 17:07:09.040 17826 WARNING ceilometer.agent.manager [-] Unknown discovery extension: local_instances
2016-11-07 17:07:09.041 17826 INFO ceilometer.agent.manager [-] Skip pollster network.incoming.bytes.rate, no resources found this cycle
2016-11-07 17:07:09.042 17826 WARNING ceilometer.agent.manager [-] Unknown discovery extension: local_instances
2016-11-07 17:07:09.042 17826 INFO ceilometer.agent.manager [-] Skip pollster network.incoming.packets, no resources found this cycle
2016-11-07 17:07:09.042 17826 WARNING ceilometer.agent.manager [-] Unknown discovery extension: local_instances

Version-Release number of selected component (if applicable):
-------------------------------------------------------------
openstack-ceilometer-notification-7.0.0-2.2.el7ost.noarch
openstack-ceilometer-compute-7.0.0-2.2.el7ost.noarch
python-ceilometerclient-2.6.1-2.el7ost.noarch
python-ceilometer-7.0.0-2.2.el7ost.noarch
openstack-ceilometer-collector-7.0.0-2.2.el7ost.noarch
openstack-ceilometer-common-7.0.0-2.2.el7ost.noarch
openstack-ceilometer-polling-7.0.0-2.2.el7ost.noarch
openstack-ceilometer-central-7.0.0-2.2.el7ost.noarch
python-ceilometermiddleware-0.5.0-1.el7ost.noarch
openstack-ceilometer-api-7.0.0-2.2.el7ost.noarch
puppet-ceilometer-9.4.0-2.el7ost.noarch

openstack-tripleo-heat-templates-compat-2.0.0-34.3.el7ost.noarch
openstack-tripleo-0.0.8-0.2.4de13b3git.el7ost.noarch
openstack-tripleo-image-elements-5.0.0-2.el7ost.noarch
openstack-tripleo-ui-1.0.4-2.el7ost.noarch
openstack-tripleo-puppet-elements-5.0.0-1.el7ost.noarch
openstack-tripleo-common-5.3.0-3.el7ost.noarch
openstack-tripleo-validations-5.1.0-1.el7ost.noarch
openstack-tripleo-heat-templates-5.0.0-1.2.el7ost.noarch
puppet-tripleo-5.3.0-4.el7ost.noarch

Steps to Reproduce:
-------------------
1. Run upgrade from RHOS-9 -> RHOS-10
2. After upgrading computes check ceilometer/compute.log

Additional info:
----------------
Restart of openstack-ceilometer-compute.service eliminates the issue.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/394968

[Marios Andreou (marios-b)]

Note this is first discussed at https://bugzilla.redhat.com/show_bug.cgi?id=1392773

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/395693

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/394968
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=094bbefe71b2f780b8597ef624c8e3e7a53c3ef8
Submitter: Jenkins
Branch: master

commit 094bbefe71b2f780b8597ef624c8e3e7a53c3ef8
Author: Pradeep Kilambi <email address hidden>
Date: Tue Nov 8 08:59:10 2016 -0500

    ceilometer compute agent needs restart on compute upgrade

    After compute nodes are upgraded, the ceilometer compute agent
    doesnt poll and throws warnings. Restarting the compute agent
    at this step gets the service back to its normal state.

    Closes-Bug: #1640177

    Change-Id: I7392de43e933b1d16002e12e407748ae289d5e99

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/395693
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=0468b748e7b01f9bc6364b79fb010c20d77b30c7
Submitter: Jenkins
Branch: stable/newton

commit 0468b748e7b01f9bc6364b79fb010c20d77b30c7
Author: Pradeep Kilambi <email address hidden>
Date: Tue Nov 8 08:59:10 2016 -0500

    ceilometer compute agent needs restart on compute upgrade

    After compute nodes are upgraded, the ceilometer compute agent
    doesnt poll and throws warnings. Restarting the compute agent
    at this step gets the service back to its normal state.

    Closes-Bug: #1640177

    Change-Id: I7392de43e933b1d16002e12e407748ae289d5e99
    (cherry picked from commit 094bbefe71b2f780b8597ef624c8e3e7a53c3ef8)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 6.0.0.0b1 development milestone.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 5.2.0 release.