resource metadata not properly reported

[fetahi]

As soon as a VM is booted, 'ceilometer resource-show' returns
g@ubuntu:~/trigger$ ceilometer resource-show -r dd4a2a98-07bf-4c05-85dd-adfaa873531b
+-------------+------------------------------------------------------------------------+
| Property | Value |
+-------------+------------------------------------------------------------------------+
| source | |
| project_id | d7030309ef3c47a6ac95a01be6ddbd12 |
| user_id | f45dceb47a35417f813ad7cea8fc60ce |
| metadata | {u'ephemeral_gb': u'0', u'event_type': u'compute.instance.create.end', |
| | u'disk_gb': u'0', u'availability_zone': u'None', u'kernel_id': u'', |
| | u'ramdisk_id': u'', u'host': u'compute.ubuntu', u'memory_mb': u'48', |
| | u'instance_type': u'8', u'vcpus': u'1', u'root_gb': u'0', |
| | u'architecture': u'None', u'os_type': u'None', u'reservation_id': |
| | u'r-a9pg5fd4'} |
| resource_id | dd4a2a98-07bf-4c05-85dd-adfaa873531b |
+-------------+------------------------------------------------------------------------+

But after a minute or so, metadata values that relate to the specific instance become null.
g@ubuntu:~/trigger$ ceilometer resource-show -r dd4a2a98-07bf-4c05-85dd-adfaa873531b
+-------------+--------------------------------------------------------------------------+
| Property | Value |
+-------------+--------------------------------------------------------------------------+
| source | |
| project_id | d7030309ef3c47a6ac95a01be6ddbd12 |
| user_id | f45dceb47a35417f813ad7cea8fc60ce |
| metadata | {u'ephemeral_gb': u'', u'display_name': u'one', u'name': |
| | u'instance-00000023', u'disk_gb': u'', u'availability_zone': u'', |
| | u'kernel_id': u'', u'ramdisk_id': u'', u'vcpus': u'', u'memory_mb': u'', |
| | u'instance_type': u'10', u'host': |
| | u'41378f4bbbb00b48d03cd6b79d98bd0a900f8486201243a78c67e9b0', u'root_gb': |
| | u'', u'image_ref': u'05f878a5-cca3-48cf-94ba-6928e9c02441', |
| | u'architecture': u'', u'os_type': u'', u'reservation_id': u'', |
| | u'image_ref_url': u'http://10.0.0.1:8774/86339ddb59f74dabab0e0e9a87a6f26 |
| | 6/images/05f878a5-cca3-48cf-94ba-6928e9c02441'} |
| resource_id | dd4a2a98-07bf-4c05-85dd-adfaa873531b |
+-------------+--------------------------------------------------------------------------+

I suspect this is the result of values reported by pollsters overwriting the values reported by the notifications.

[John Tran (jtran)]

I ran into this issue today 'stable/grizzy 2013.1.2 June' ubuntu cloud archive release as well as on devstack stable/grizzly. I've looked into the issue, and indeed the notification is sending the correct metadata on instance.create and then subsequent pollsters overwrite the metadata, using novaclient.servers.list(detailed=True, search_opts={'host': 'mynode', 'all_tenants': True}) which doesn't return values in regards to vcpu, memory, etc for complete metadata. In older versions of ceilometer, it used directly libvirt calls instead of novaclient to grab that metadata.

[John Tran (jtran)]

I noticed this has been set to Havana-2. Any reason, if a fix is submitted, to get it into grizzly 2013.1.3 Aug 8? I have a patch that I can submit and will fix these specific attributes: availability_zone, disk_gb, kernel_id, memory_mb, ramdisk_id, vcpus.

Anyone know how or where I can get the other missing metadata: 'architecture', 'os_type', 'root_gb' and 'reservation_id'? The first three appear to be glance related however, I've found they are not a built in for all openstack installations and are custom implemented attributes.

[xingzhou (xingzhou)]

@John, I've removing myself as the defect owner, if you already had a patch for this bug, please go ahead to submit it, thx

[Julien Danjou (jdanjou)]

John, we need to get the patch into master before we can land it into stable/grizzly. But feel free to propose one !

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/34494
Committed: http://github.com/openstack/ceilometer/commit/298e2f7e7b7074f1b6dab8175fa19bc456f4a0c4
Submitter: Jenkins
Branch: master

commit 298e2f7e7b7074f1b6dab8175fa19bc456f4a0c4
Author: John Tran <email address hidden>
Date: Tue Jun 25 18:07:58 2013 +0000

    add metadata to nova_client results

    Fixes bug 1190547

    Change-Id: I90861038f0623e6183dfd43899669aba88b47796

[Julien Danjou (jdanjou)]

John, how would/should we fix that in Grizzly?
It seems to me it's affected too, but that's the patch may be a bit invasive for this version. Is there a simpler version that could be used as a backport?

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/grizzly
Review: https://review.openstack.org/35245

[John Tran (jtran)]

Julien, looks like the only changes needed for stable/grizzly were minor in one of the tests. I've submitted a patch to stable/grizzly branch with the necessary changes.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/grizzly
Review: https://review.openstack.org/38842

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/38842
Committed: http://github.com/openstack/ceilometer/commit/ce16637f448819caf753d51fddae6e78c4dd5d6f
Submitter: Jenkins
Branch: stable/grizzly

commit ce16637f448819caf753d51fddae6e78c4dd5d6f
Author: John Tran <email address hidden>
Date: Sat Jul 27 02:52:31 2013 +0000

    add metadata to nova_client results

    Fixes bug 1190547

    Conflicts:

     tests/compute/test_instance.py

    Change-Id: I90861038f0623e6183dfd43899669aba88b47796
    (Cherry-picked from commit 298e2f7e7b7074f1b6dab8175fa19bc456f4a0c4)