Services not running that should be: ceilometer-agent-compute

[Jeffrey Chang]

From an SolQA deployment - https://solutions.qa.canonical.com/testruns/58139540-ab0e-4dea-9d96-99a96b95669b
Artifacts - https://oil-jenkins.canonical.com/artifacts/58139540-ab0e-4dea-9d96-99a96b95669b/index.html
We found one of the ceilometer-agent blocked with "Services not running that should be: ceilometer-agent-compute".

ceilometer-agent-compute.log show no problem. But journalctl has below logs repeatly,

Feb 17 23:42:44 solqa-lab1-server-33 systemd[1]: Started OpenStack Compute.
Feb 17 23:42:44 solqa-lab1-server-33 systemd[1]: Started Ceilometer Agent Compute.
Feb 17 23:42:45 solqa-lab1-server-33 nova-compute[1814283]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code.
Feb 17 23:42:45 solqa-lab1-server-33 ceilometer-agent-compute[1814284]: Deprecated: Option "logdir" from group "DEFAULT" is deprecated. Use option "log-dir" from group "DEFAULT".
Feb 17 23:42:46 solqa-lab1-server-33 nova-compute[1814283]: Deprecated: Option "logdir" from group "DEFAULT" is deprecated. Use option "log-dir" from group "DEFAULT".
Feb 17 23:42:48 solqa-lab1-server-33 systemd[1]: nova-compute.service: Main process exited, code=exited, status=1/FAILURE
Feb 17 23:42:48 solqa-lab1-server-33 systemd[1]: nova-compute.service: Failed with result 'exit-code'.
Feb 17 23:42:48 solqa-lab1-server-33 systemd[1]: nova-compute.service: Consumed 2.177s CPU time.
Feb 17 23:42:48 solqa-lab1-server-33 systemd[1]: nova-compute.service: Scheduled restart job, restart counter is at 468.
Feb 17 23:42:48 solqa-lab1-server-33 systemd[1]: Stopping Ceilometer Agent Compute...
Feb 17 23:42:48 solqa-lab1-server-33 libvirtd[44845]: End of file while reading data: Input/output error
Feb 17 23:42:48 solqa-lab1-server-33 libvirtd[44845]: End of file while reading data: Input/output error
Feb 17 23:42:49 solqa-lab1-server-33 systemd[1]: ceilometer-agent-compute.service: Deactivated successfully.
Feb 17 23:42:49 solqa-lab1-server-33 systemd[1]: Stopped Ceilometer Agent Compute.
Feb 17 23:42:49 solqa-lab1-server-33 systemd[1]: Stopped OpenStack Compute.
Feb 17 23:42:49 solqa-lab1-server-33 systemd[1]: nova-compute.service: Consumed 2.177s CPU time.

until
Feb 17 23:43:41 solqa-lab1-server-33 systemd[1]: ceilometer-agent-compute.service: Start request repeated too quickly.
Feb 17 23:43:41 solqa-lab1-server-33 systemd[1]: ceilometer-agent-compute.service: Failed with result 'start-limit-hit'.

[Guillaume Boutry (gboutry)]

Why should ceilometer-agent-compute be running ? have you enabled the telemetry plugin ?

[Jeffrey Chang (modern911)]

I believe SolQA has this long time ago, as we saw LP#1947585 from time to time.
The occurrence can be found from https://solutions.qa.canonical.com/bugs/1947585
And since LP#1947585 is fix released, I decided to file a new one.