For polled samples related to autoscaled instances, we apply a mapping to the user metadata so as to ensure that there are no embedded periods:
https://github.com/openstack/ceilometer/blob/master/ceilometer/compute/pollsters/util.py
However, the same user metdata will also be present for notifications that related to the same instance, but the mapping is not applied in that case.
This causes the mongo driver to fail with not okForStorage when the sample derived from the notification is dispatched.
For example, when the sample derived from a compute.instance.delete.end is processed:
ceilometer-ceilometer.openstack.common.rpc.amqp DEBUG: received {u'_context_request_id': u'req-1fa1edc3-7cc2-482a-8526-7a308daacbe0', u'args': {u'data': [{u'counter_name': u'disk.root.size', u'user_id': u'ef4e983291ef4ad1b88eb1f776bd52b6', u'resource_id': u'0b900e89-cf10-44a8-a79c-f2f68f55fc07', u'timestamp': u'2014-02-24 11:59:55.295790', u'message_signature': u'49ea1df965fd09b6ce3c84812977e9c21474030006510ef402afd6b2e12c9f90', u'resource_metadata': {u'state_description': u'', u'event_type': u'compute.instance.delete.end', u'availability_zone': None, u'terminated_at': u'2014-02-24T11:59:54.965388', u'ephemeral_gb': 0, u'instance_type_id': 2, u'deleted_at': u'', u'reservation_id': u'r-8pz6nwrl', u'instance_id': u'0b900e89-cf10-44a8-a79c-f2f68f55fc07', u'user_id': u'ef4e983291ef4ad1b88eb1f776bd52b6', u'hostname': u'tyky-group-a-wste7bd63rkl-group-a-1-xzgy6fkgqvhc', u'state': u'deleted', u'launched_at': u'2014-02-24T11:58:16.000000', u'metadata': {u'metering.server_group': u'Group_A', u'AutoScalingGroupName': u'tyky-Group_A-wste7bd63rkl', u'assign_floating_ip': u'true'}, u'node': u'node-17', u'ramdisk_id': u'', u'access_ip_v6': None, u'disk_gb': 1, u'access_ip_v4': None, u'kernel_id': u'', u'host': u'compute.node-17', u'display_name': u'tyky-Group_A-wste7bd63rkl-Group_A-1-xzgy6fkgqvhc', u'image_ref_url': u'http://192.168.100.4:9292/images/11848cbf-a428-4dfb-8818-2f0a981f540b', u'root_gb': 1, u'tenant_id': u'efcca4ba425c4beda73eb31a54df931a', u'created_at': u'2014-02-24 11:58:08+00:00', u'memory_mb': 512, u'instance_type': u'm1.tiny', u'vcpus': 1, u'image_meta': {u'min_disk': u'1', u'container_format': u'bare', u'min_ram': u'0', u'disk_format': u'qcow2', u'base_image_ref': u'11848cbf-a428-4dfb-8818-2f0a981f540b'}, u'architecture': None, u'os_type': None, u'instance_flavor_id': u'1'}, u'source': u'openstack', u'counter_unit': u'GB', u'counter_volume': 1, u'project_id': u'efcca4ba425c4beda73eb31a54df931a', u'message_id': u'2d03ed96-9d4b-11e3-b0e0-080027e519cb', u'counter_type': u'gauge'}]}, u'_context_auth_token': '<SANITIZED>', u'_context_show_deleted': False, u'_context_tenant': None, u'_unique_id': u'6bf474db7f9f4bc3a3a4342508c54218', u'_context_is_admin': True, u'version': u'1.0', u'_context_read_only': False, u'_context_user': None, u'method': u'record_metering_data'}
<47>Feb 24 11:59:55 node-16 <U+FEFF>ceilometer-ceilometer.openstack.common.rpc.amqp DEBUG: unpacked context: {'read_only': False, 'show_deleted': False, 'auth_token': '<SANITIZED>', 'is_admin': True, 'user': None, 'request_id': u'req-1fa1edc3-7cc2-482a-8526-7a308daacbe0', 'tenant': None}
<47>Feb 24 11:59:55 node-16 <U+FEFF>ceilometer-ceilometer.collector.dispatcher.database DEBUG: metering data disk.root.size for 0b900e89-cf10-44a8-a79c-f2f68f55fc07 @ 2014-02-24 11:59:55.295790: 1
<47>Feb 24 11:59:55 node-16 ceilometer-amqp DEBUG: Channel open
<43>Feb 24 11:59:55 node-16 ceilometer-ceilometer.collector.dispatcher.database ERROR: Failed to record metering data: not okForStorage
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/ceilometer/collector/dispatcher/database.py", line 65, in record_metering_data
self.storage_conn.record_metering_data(meter)
File "/usr/lib/python2.7/dist-packages/ceilometer/storage/impl_mongodb.py", line 417, in record_metering_data
upsert=True,
File "/usr/lib/python2.7/dist-packages/pymongo/collection.py", line 487, in update
check_keys, self.__uuid_subtype), safe)
File "/usr/lib/python2.7/dist-packages/pymongo/mongo_client.py", line 969, in _send_message
rv = self.__check_response_to_last_error(response)
File "/usr/lib/python2.7/dist-packages/pymongo/mongo_client.py", line 911, in __check_response_to_last_error
raise OperationFailure(details["err"], details["code"])
OperationFailure: not okForStorage
Fix proposed to branch: master /review. openstack. org/77959
Review: https:/