ceilometer polling agent consumes too much memory on vmware

[Supreeth Shivanand]

ceilometer polling agent compute consumes more than 5gb of memory when configured for a vmware hypervisor.

ps command output:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
 1938 pf9 20 0 5721636 4.391g 1432 S 0.7 57.5 10:26.83 pf9-ceilometerd

Root cause of this issue seems to be because of a change how pollster initializes inspector introduced in pike. Instead of creating only one instance of inspector, it creates one inspector instance per pollster which is a regression and leads to unnecessary memory usage.

In our configuration, we had 50 pollsters in compute namespace and it ended up creating 50 vmware sessions which consumed more than 5 GB memory. I think this issue not specific to vmware and may not be evident on kvm,etc because their session are probably much lighter.

https://github.com/openstack/ceilometer/commit/66179f89a0651168d59cd9c36bac5f9ffb104471#diff-9f6451b59db910e9a91011fe337f9a63R39

[Pooja Ghumre (pooja-9)]

Seeing this since upgrading to Pike release. Probably this change in Pike is related?

https://github.com/openstack/ceilometer/commit/66179f89a0651168d59cd9c36bac5f9ffb104471

[OpenStack Infra (hudson-openstack)]

Change abandoned by Supreeth Shivanand (<email address hidden>) on branch: master
Review: https://review.openstack.org/642897

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/642902

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/642902
Committed: https://git.openstack.org/cgit/openstack/ceilometer/commit/?id=ab73c911dd17400e5dc67a5485fdf8b5d6826283
Submitter: Zuul
Branch: master

commit ab73c911dd17400e5dc67a5485fdf8b5d6826283
Author: Supreeth Shivanand <email address hidden>
Date: Tue Mar 12 14:06:44 2019 -0700

    making inspector object singleton again by assigning to the base pollster class

    Change-Id: I1afc02d595273f7b699afac9c12fcb17a777b4a8
    Closes-Bug: #1819562

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/ceilometer 13.0.0.0rc1 release candidate.