The recent change to ensure the oslo.messaging context is a dict as opposed to a RequestContext:
https://github.com/openstack/ceilometer/commit/5df4cb53
in order to allow the alarm-evaluator to message the alarm-notifier successfully, appears to have had the unintended side-effect of causing a similar issue in the notification agent.
For example, handling the 'compute.instance.delete.end' event is seen to fail as:
2014-05-07 22:25:46.934 14285 DEBUG oslo.messaging._drivers.amqpdriver [-] received {u'_context_request_id': u'req-0b9f4725-2d13-445a-b04f-40470a50f05e', u'_context_quota_class': None, u'event_type': u'compute.instance.delete.end', u'_context_service_catalog': [{u'endpoints': [{u'adminURL': u'http://172.16.12.73:8776/v1/7ce581ebbe99483190479d8444fa2e8c', u'region': u'RegionOne', u'id': u'614644194cc644b98fe5422e4ccfa2e8', u'internalURL': u'http://172.16.12.73:8776/v1/7ce581ebbe99483190479d8444fa2e8c', u'publicURL': u'http://172.16.12.73:8776/v1/7ce581ebbe99483190479d8444fa2e8c'}], u'endpoints_links': [], u'type': u'volume', u'name': u'cinder'}], u'_context_auth_token': '<SANITIZED>', u'_context_user_id': u'0291be3e6d204481bf7ae9921cd292d2', u'payload': {u'state_description': u'', u'availability_zone': None, u'terminated_at': u'2014-05-07T22:25:46.481284', u'ephemeral_gb': 0, u'instance_type_id': 2, u'deleted_at': u'2014-05-07T22:25:46.000000', u'reservation_id': u'r-00z8fz63', u'instance_id': u'a45f52c0-1fb2-4da4-b0c9-17859ec228a1', u'user_id': u'0291be3e6d204481bf7ae9921cd292d2', u'hostname': u's2', u'state': u'deleted', u'launched_at': u'2014-05-07T22:25:17.000000', u'metadata': {}, u'node': u'eglynn-f20-devstack3', u'ramdisk_id': u'4e4fd861-a6e6-4587-8097-ce6b497dfd73', u'access_ip_v6': None, u'disk_gb': 1, u'access_ip_v4': None, u'kernel_id': u'1043414b-d093-416d-97ed-1606044b70f3', u'host': u'eglynn-f20-devstack3', u'display_name': u's2', u'image_ref_url': u'http://172.16.12.73:9292/images/892dd525-eb7e-4188-96b6-d07bcab2b1d3', u'root_gb': 1, u'tenant_id': u'7ce581ebbe99483190479d8444fa2e8c', u'created_at': u'2014-05-07 22:24:29+00:00', u'memory_mb': 512, u'instance_type': u'm1.tiny', u'vcpus': 1, u'image_meta': {u'kernel_id': u'1043414b-d093-416d-97ed-1606044b70f3', u'container_format': u'ami', u'min_ram': u'0', u'ramdisk_id': u'4e4fd861-a6e6-4587-8097-ce6b497dfd73', u'disk_format': u'ami', u'min_disk': u'1', u'base_image_ref': u'892dd525-eb7e-4188-96b6-d07bcab2b1d3'}, u'architecture': None, u'os_type': None, u'instance_flavor_id': u'1'}, u'priority': u'INFO', u'_context_is_admin': True, u'_context_user': u'0291be3e6d204481bf7ae9921cd292d2', u'publisher_id': u'compute.eglynn-f20-devstack3', u'message_id': u'be52211a-94dd-4e8a-bcc7-0cb8aaa62dc8', u'_context_remote_address': u'172.16.12.73', u'_context_roles': [u'heat_stack_owner', u'_member_', u'admin'], u'timestamp': u'2014-05-07 22:25:46.928319', u'_context_timestamp': u'2014-05-07T22:25:43.902013', u'_unique_id': u'6bf58db7c33649b0aa6240ed38d5fcee', u'_context_project_name': u'admin', u'_context_read_deleted': u'no', u'_context_tenant': u'7ce581ebbe99483190479d8444fa2e8c', u'_context_instance_lock_checked': False, u'_context_project_id': u'7ce581ebbe99483190479d8444fa2e8c', u'_context_user_name': u'admin'} _safe_log /usr/lib/python2.7/site-packages/oslo/messaging/_drivers/common.py:193
2014-05-07 22:25:46.936 14285 DEBUG oslo.messaging._drivers.amqp [-] unpacked context: {'tenant': u'7ce581ebbe99483190479d8444fa2e8c', 'project_name': u'admin', 'user_id': u'0291be3e6d204481bf7ae9921cd292d2', 'roles': [u'heat_stack_owner', u'_member_', u'admin'], 'timestamp': u'2014-05-07T22:25:43.902013', 'auth_token': '<SANITIZED>', 'remote_address': u'172.16.12.73', 'quota_class': None, 'is_admin': True, 'user': u'0291be3e6d204481bf7ae9921cd292d2', 'service_catalog': [{u'endpoints_links': [], u'endpoints': [{u'adminURL': u'http://172.16.12.73:8776/v1/7ce581ebbe99483190479d8444fa2e8c', u'region': u'RegionOne', u'publicURL': u'http://172.16.12.73:8776/v1/7ce581ebbe99483190479d8444fa2e8c', u'id': u'614644194cc644b98fe5422e4ccfa2e8', u'internalURL': u'http://172.16.12.73:8776/v1/7ce581ebbe99483190479d8444fa2e8c'}], u'type': u'volume', u'name': u'cinder'}], 'request_id': u'req-0b9f4725-2d13-445a-b04f-40470a50f05e', 'instance_lock_checked': False, 'project_id': u'7ce581ebbe99483190479d8444fa2e8c', 'user_name': u'admin', 'read_deleted': u'no'} _safe_log /usr/lib/python2.7/site-packages/oslo/messaging/_drivers/common.py:193
2014-05-07 22:25:46.937 14285 ERROR oslo.messaging.notify.dispatcher [-] Exception during message handling
2014-05-07 22:25:46.937 14285 TRACE oslo.messaging.notify.dispatcher Traceback (most recent call last):
2014-05-07 22:25:46.937 14285 TRACE oslo.messaging.notify.dispatcher File "/usr/lib/python2.7/site-packages/oslo/messaging/notify/dispatcher.py", line 85, in _dispatch_and_handle_error
2014-05-07 22:25:46.937 14285 TRACE oslo.messaging.notify.dispatcher return self._dispatch(incoming.ctxt, incoming.message)
2014-05-07 22:25:46.937 14285 TRACE oslo.messaging.notify.dispatcher File "/usr/lib/python2.7/site-packages/oslo/messaging/notify/dispatcher.py", line 121, in _dispatch
2014-05-07 22:25:46.937 14285 TRACE oslo.messaging.notify.dispatcher metadata)
2014-05-07 22:25:46.937 14285 TRACE oslo.messaging.notify.dispatcher File "/opt/stack/ceilometer/ceilometer/plugin.py", line 106, in info
2014-05-07 22:25:46.937 14285 TRACE oslo.messaging.notify.dispatcher 'info', ctxt, publisher_id, event_type, payload, metadata)
2014-05-07 22:25:46.937 14285 TRACE oslo.messaging.notify.dispatcher File "/opt/stack/ceilometer/ceilometer/messaging.py", line 124, in convert_to_old_notification_format
2014-05-07 22:25:46.937 14285 TRACE oslo.messaging.notify.dispatcher for k in ctxt:
2014-05-07 22:25:46.937 14285 TRACE oslo.messaging.notify.dispatcher TypeError: 'RequestContext' object is not iterable
2014-05-07 22:25:46.937 14285 TRACE oslo.messaging.notify.dispatcher
The revert of the patch that caused this regression has landed:
https:/