TypeError: unsupported operand type(s) for %: 'dict' and 'unicode'

[Andreas Merk]

After moving to gnocchi via the ocata release update I get following error with the collector:

2017-04-03 19:15:46.055 2325 ERROR ceilometer.collector [-] Dispatcher failed to handle the notification, re-queuing it.
2017-04-03 19:15:46.055 2325 ERROR ceilometer.collector Traceback (most recent call last):
2017-04-03 19:15:46.055 2325 ERROR ceilometer.collector File "/usr/lib/python2.7/dist-packages/ceilometer/collector.py", line 178, in sample
2017-04-03 19:15:46.055 2325 ERROR ceilometer.collector self.dispatcher_manager.map_method(self.method, goods)
2017-04-03 19:15:46.055 2325 ERROR ceilometer.collector File "/usr/lib/python2.7/dist-packages/stevedore/extension.py", line 282, in map_method
2017-04-03 19:15:46.055 2325 ERROR ceilometer.collector method_name, *args, **kwds)
2017-04-03 19:15:46.055 2325 ERROR ceilometer.collector File "/usr/lib/python2.7/dist-packages/stevedore/extension.py", line 255, in map
2017-04-03 19:15:46.055 2325 ERROR ceilometer.collector self._invoke_one_plugin(response.append, func, e, args, kwds)
2017-04-03 19:15:46.055 2325 ERROR ceilometer.collector File "/usr/lib/python2.7/dist-packages/stevedore/extension.py", line 286, in _invoke_one_plugin
2017-04-03 19:15:46.055 2325 ERROR ceilometer.collector response_callback(func(e, *args, **kwds))
2017-04-03 19:15:46.055 2325 ERROR ceilometer.collector File "/usr/lib/python2.7/dist-packages/stevedore/extension.py", line 260, in _call_extension_method
2017-04-03 19:15:46.055 2325 ERROR ceilometer.collector return getattr(extension.obj, method_name)(*args, **kwds)
2017-04-03 19:15:46.055 2325 ERROR ceilometer.collector File "/usr/lib/python2.7/dist-packages/ceilometer/dispatcher/gnocchi.py", line 493, in record_events
2017-04-03 19:15:46.055 2325 ERROR ceilometer.collector self._delete_event(rd, event)
2017-04-03 19:15:46.055 2325 ERROR ceilometer.collector File "/usr/lib/python2.7/dist-packages/ceilometer/dispatcher/gnocchi.py", line 504, in _delete_event
2017-04-03 19:15:46.055 2325 ERROR ceilometer.collector for resource_type, query in associated_resources.items()
2017-04-03 19:15:46.055 2325 ERROR ceilometer.collector TypeError: unsupported operand type(s) for %: 'dict' and 'unicode'
2017-04-03 19:15:46.055 2325 ERROR ceilometer.collector
2017-04-03 19:15:46.027 2325 DEBUG ceilometer.dispatcher.gnocchi [-] 4 measures posted against 4 metrics through 4 resources batch_measures /usr/lib/python2.7/dist-packages/ceilometer/dispatcher/gnocchi.py:433

What further information is required to understand the problem?

[gordon chung (chungg)]

please add the gnocchi and gnocchiclient versions you are using.

[Andreas Merk (amerk)]

All original Debian packages:
gnocchi-3.1.2
gnocchiclient-3.1.1
ceilometer-8.0.1

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/455626

[Hanxi Liu (hanxi-liu)]

@Andreas, Hi, could you tell how to reproduce this?

[Andreas Merk (amerk)]

Hanxi Liu I'll contact you directly - you can have access to the installation or inject debug code if you want.

I can also try to redo the installation on a second controller and document it, but I think it fails on the type of messages it gets. I put some debug code myself in and I think I see old VM delete messages.

~Andreas

[gordon chung (chungg)]

i think it's how the loader is reading in the gnocchi_resources.yaml file. we have a string as the field but it's in reality a stringified dictionary

https://github.com/openstack/ceilometer/blob/master/ceilometer/dispatcher/data/gnocchi_resources.yaml#L78

[OpenStack Infra (hudson-openstack)]

Change abandoned by Hanxi Liu (<email address hidden>) on branch: master
Review: https://review.openstack.org/455626
Reason: no more clue from bug reporter