ceilometer agent managers bails out if a libvirt XML is missing a field

Bug #1749655 reported by Junien Fridrick
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
ceilometer (Ubuntu)
Undecided
Unassigned

Bug Description

Hi,

Using python-ceilometer 1:8.1.2-0ubuntu1~cloud0 on Ubuntu 16.04.3 LTS, we just got the traceback below. The problem is that a single libvirt XML was missing the "nova:owner" block (I don't know why), and it looks like this single problem was impacting meter collection for all the other VMs. Once this VM got removed, the meters were properly populated (see below for before and after the VM removal).

A single bad XML should not block collection for all the VMs.

Thanks

Traceback :

2018-02-15 07:26:43.779 1904096 ERROR ceilometer.agent.manager [-] Unable to discover resources: 'NoneType' object has no attribute 'attrib'
2018-02-15 07:26:43.779 1904096 ERROR ceilometer.agent.manager Traceback (most recent call last):
2018-02-15 07:26:43.779 1904096 ERROR ceilometer.agent.manager File "/usr/lib/python2.7/dist-packages/ceilometer/agent/manager.py", line 505, in disco
ver
2018-02-15 07:26:43.779 1904096 ERROR ceilometer.agent.manager discovered = discoverer.discover(self, param)
2018-02-15 07:26:43.779 1904096 ERROR ceilometer.agent.manager File "/usr/lib/python2.7/dist-packages/ceilometer/compute/discovery.py", line 133, in d
iscover
2018-02-15 07:26:43.779 1904096 ERROR ceilometer.agent.manager return self.discover_libvirt_polling(manager, param=None)
2018-02-15 07:26:43.779 1904096 ERROR ceilometer.agent.manager File "/usr/lib/python2.7/dist-packages/ceilometer/compute/virt/libvirt/utils.py", line
90, in decorator
2018-02-15 07:26:43.779 1904096 ERROR ceilometer.agent.manager return function(self, *args, **kwargs)
2018-02-15 07:26:43.779 1904096 ERROR ceilometer.agent.manager File "/usr/lib/python2.7/dist-packages/ceilometer/compute/discovery.py", line 184, in d
iscover_libvirt_polling
2018-02-15 07:26:43.779 1904096 ERROR ceilometer.agent.manager user_id = metadata_xml.find("./owner/user").attrib["uuid"]
2018-02-15 07:26:43.779 1904096 ERROR ceilometer.agent.manager AttributeError: 'NoneType' object has no attribute 'attrib'
2018-02-15 07:26:43.779 1904096 ERROR ceilometer.agent.manager
2018-02-15 07:26:43.780 1904096 INFO ceilometer.agent.manager [-] Skip pollster disk.device.write.bytes.rate, no resources found this cycle

Before the fix :

$ ceilometer meter-list --query resource=3bfc7f11-a0b9-46af-b7cc-733361fe0efc
+-------------------------------+-------+------+--------------------------------------+------------------------------------------------------------------+----------------------------------+
| Name | Type | Unit | Resource ID | User ID | Project ID |
+-------------------------------+-------+------+--------------------------------------+------------------------------------------------------------------+----------------------------------+
| compute.instance.booting.time | gauge | sec | 3bfc7f11-a0b9-46af-b7cc-733361fe0efc | c25152c3f94599f7dbc7f81ddb01dd8922f68b433e745819509d9745d07b6a2e | b18ffb39e718417bb28e806e6c1e802a |
| disk.ephemeral.size | gauge | GB | 3bfc7f11-a0b9-46af-b7cc-733361fe0efc | c25152c3f94599f7dbc7f81ddb01dd8922f68b433e745819509d9745d07b6a2e | b18ffb39e718417bb28e806e6c1e802a |
| disk.root.size | gauge | GB | 3bfc7f11-a0b9-46af-b7cc-733361fe0efc | c25152c3f94599f7dbc7f81ddb01dd8922f68b433e745819509d9745d07b6a2e | b18ffb39e718417bb28e806e6c1e802a |
| memory | gauge | MB | 3bfc7f11-a0b9-46af-b7cc-733361fe0efc | c25152c3f94599f7dbc7f81ddb01dd8922f68b433e745819509d9745d07b6a2e | b18ffb39e718417bb28e806e6c1e802a |
| vcpus | gauge | vcpu | 3bfc7f11-a0b9-46af-b7cc-733361fe0efc | c25152c3f94599f7dbc7f81ddb01dd8922f68b433e745819509d9745d07b6a2e | b18ffb39e718417bb28e806e6c1e802a |
+-------------------------------+-------+------+--------------------------------------+------------------------------------------------------------------+----------------------------------+

After the fix :

$ ceilometer meter-list --query resource=3bfc7f11-a0b9-46af-b7cc-733361fe0efc
+-------------------------------+------------+---------+--------------------------------------+------------------------------------------------------------------+----------------------------------+
| Name | Type | Unit | Resource ID | User ID | Project ID |
+-------------------------------+------------+---------+--------------------------------------+------------------------------------------------------------------+----------------------------------+
| compute.instance.booting.time | gauge | sec | 3bfc7f11-a0b9-46af-b7cc-733361fe0efc | c25152c3f94599f7dbc7f81ddb01dd8922f68b433e745819509d9745d07b6a2e | b18ffb39e718417bb28e806e6c1e802a |
| cpu | cumulative | ns | 3bfc7f11-a0b9-46af-b7cc-733361fe0efc | c25152c3f94599f7dbc7f81ddb01dd8922f68b433e745819509d9745d07b6a2e | b18ffb39e718417bb28e806e6c1e802a |
| disk.allocation | gauge | B | 3bfc7f11-a0b9-46af-b7cc-733361fe0efc | c25152c3f94599f7dbc7f81ddb01dd8922f68b433e745819509d9745d07b6a2e | b18ffb39e718417bb28e806e6c1e802a |
| disk.capacity | gauge | B | 3bfc7f11-a0b9-46af-b7cc-733361fe0efc | c25152c3f94599f7dbc7f81ddb01dd8922f68b433e745819509d9745d07b6a2e | b18ffb39e718417bb28e806e6c1e802a |
| disk.ephemeral.size | gauge | GB | 3bfc7f11-a0b9-46af-b7cc-733361fe0efc | c25152c3f94599f7dbc7f81ddb01dd8922f68b433e745819509d9745d07b6a2e | b18ffb39e718417bb28e806e6c1e802a |
| disk.read.bytes | cumulative | B | 3bfc7f11-a0b9-46af-b7cc-733361fe0efc | c25152c3f94599f7dbc7f81ddb01dd8922f68b433e745819509d9745d07b6a2e | b18ffb39e718417bb28e806e6c1e802a |
| disk.read.requests | cumulative | request | 3bfc7f11-a0b9-46af-b7cc-733361fe0efc | c25152c3f94599f7dbc7f81ddb01dd8922f68b433e745819509d9745d07b6a2e | b18ffb39e718417bb28e806e6c1e802a |
| disk.root.size | gauge | GB | 3bfc7f11-a0b9-46af-b7cc-733361fe0efc | c25152c3f94599f7dbc7f81ddb01dd8922f68b433e745819509d9745d07b6a2e | b18ffb39e718417bb28e806e6c1e802a |
| disk.usage | gauge | B | 3bfc7f11-a0b9-46af-b7cc-733361fe0efc | c25152c3f94599f7dbc7f81ddb01dd8922f68b433e745819509d9745d07b6a2e | b18ffb39e718417bb28e806e6c1e802a |
| disk.write.bytes | cumulative | B | 3bfc7f11-a0b9-46af-b7cc-733361fe0efc | c25152c3f94599f7dbc7f81ddb01dd8922f68b433e745819509d9745d07b6a2e | b18ffb39e718417bb28e806e6c1e802a |
| disk.write.requests | cumulative | request | 3bfc7f11-a0b9-46af-b7cc-733361fe0efc | c25152c3f94599f7dbc7f81ddb01dd8922f68b433e745819509d9745d07b6a2e | b18ffb39e718417bb28e806e6c1e802a |
| memory | gauge | MB | 3bfc7f11-a0b9-46af-b7cc-733361fe0efc | c25152c3f94599f7dbc7f81ddb01dd8922f68b433e745819509d9745d07b6a2e | b18ffb39e718417bb28e806e6c1e802a |
| memory.resident | gauge | MB | 3bfc7f11-a0b9-46af-b7cc-733361fe0efc | c25152c3f94599f7dbc7f81ddb01dd8922f68b433e745819509d9745d07b6a2e | b18ffb39e718417bb28e806e6c1e802a |
| memory.usage | gauge | MB | 3bfc7f11-a0b9-46af-b7cc-733361fe0efc | c25152c3f94599f7dbc7f81ddb01dd8922f68b433e745819509d9745d07b6a2e | b18ffb39e718417bb28e806e6c1e802a |
| vcpus | gauge | vcpu | 3bfc7f11-a0b9-46af-b7cc-733361fe0efc | c25152c3f94599f7dbc7f81ddb01dd8922f68b433e745819509d9745d07b6a2e | b18ffb39e718417bb28e806e6c1e802a |
+-------------------------------+------------+---------+--------------------------------------+------------------------------------------------------------------+----------------------------------+

Revision history for this message
James Page (james-page) wrote :

Might be related to bug 1607313

Revision history for this message
James Page (james-page) wrote :

Actually I'm pretty sure it is that bug

Revision history for this message
Junien Fridrick (axino) wrote :

Certainly looks like it, thanks

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers