A new change in ceilometer<https://review.openstack.org/#/c/185503/5> likely broke Heat's functional testing gate.
From the console log[1], we see the stack failing consistently.
2015-07-09 04:22:58.234 | 2015-07-09 04:22:58.205 | {7} heat_integrationtests.scenario.test_ceilometer_alarm.CeilometerAlarmTest.test_alarm [9.439944s] ... FAILED
2015-07-09 04:22:58.235 | 2015-07-09 04:22:58.208 |
2015-07-09 04:22:58.241 | 2015-07-09 04:22:58.213 | Captured traceback:
2015-07-09 04:22:58.244 | 2015-07-09 04:22:58.216 | ~~~~~~~~~~~~~~~~~~~
2015-07-09 04:22:58.246 | 2015-07-09 04:22:58.219 | Traceback (most recent call last):
2015-07-09 04:22:58.247 | 2015-07-09 04:22:58.220 | File "heat_integrationtests/scenario/test_ceilometer_alarm.py", line 41, in test_alarm
2015-07-09 04:22:58.249 | 2015-07-09 04:22:58.222 | stack_identifier = self.stack_create(template=self.template)
2015-07-09 04:22:58.274 | 2015-07-09 04:22:58.227 | File "heat_integrationtests/common/test.py", line 432, in stack_create
2015-07-09 04:22:58.275 | 2015-07-09 04:22:58.232 | self._wait_for_stack_status(**kwargs)
2015-07-09 04:22:58.275 | 2015-07-09 04:22:58.234 | File "heat_integrationtests/common/test.py", line 299, in _wait_for_stack_status
2015-07-09 04:22:58.276 | 2015-07-09 04:22:58.236 | fail_regexp):
2015-07-09 04:22:58.277 | 2015-07-09 04:22:58.239 | File "heat_integrationtests/common/test.py", line 271, in _verify_status
2015-07-09 04:22:58.277 | 2015-07-09 04:22:58.242 | stack_status_reason=stack.stack_status_reason)
2015-07-09 04:22:58.278 | 2015-07-09 04:22:58.243 | heat_integrationtests.common.exceptions.StackBuildErrorException: Stack CeilometerAlarmTest-135953970/b5d5dcdc-49cb-4162-9cc3-d72e2bcd31b0 is in CREATE_FAILED status due to 'Resource CREATE failed: InternalServerError: resources.alarm: 'NoneType' object has no attribute '__getitem__' (HTTP 500) (Request-ID: req-72bf6022-b95b-4f2e-84b1-1603fb44bdda)'
The log from heat-engine[2]
2015-07-09 04:22:54.128 9135 ERROR heat.engine.resource Traceback (most recent call last):
2015-07-09 04:22:54.128 9135 ERROR heat.engine.resource File "/opt/stack/new/heat/heat/engine/resource.py", line 598, in _action_recorder
2015-07-09 04:22:54.128 9135 ERROR heat.engine.resource yield
2015-07-09 04:22:54.128 9135 ERROR heat.engine.resource File "/opt/stack/new/heat/heat/engine/resource.py", line 669, in _do_action
2015-07-09 04:22:54.128 9135 ERROR heat.engine.resource yield self.action_handler_task(action, args=handler_args)
2015-07-09 04:22:54.128 9135 ERROR heat.engine.resource File "/opt/stack/new/heat/heat/engine/scheduler.py", line 319, in wrapper
2015-07-09 04:22:54.128 9135 ERROR heat.engine.resource step = next(subtask)
2015-07-09 04:22:54.128 9135 ERROR heat.engine.resource File "/opt/stack/new/heat/heat/engine/resource.py", line 640, in action_handler_task
2015-07-09 04:22:54.128 9135 ERROR heat.engine.resource handler_data = handler(*args)
2015-07-09 04:22:54.128 9135 ERROR heat.engine.resource File "/opt/stack/new/heat/heat/engine/resources/openstack/ceilometer/alarm.py", line 264, in handle_create
2015-07-09 04:22:54.128 9135 ERROR heat.engine.resource alarm = self.ceilometer().alarms.create(**props)
2015-07-09 04:22:54.128 9135 ERROR heat.engine.resource File "/usr/local/lib/python2.7/dist-packages/ceilometerclient/v2/alarms.py", line 156, in create
2015-07-09 04:22:54.128 9135 ERROR heat.engine.resource return self._create(self._path(), new)
2015-07-09 04:22:54.128 9135 ERROR heat.engine.resource File "/usr/local/lib/python2.7/dist-packages/ceilometerclient/common/base.py", line 63, in _create
2015-07-09 04:22:54.128 9135 ERROR heat.engine.resource body = self.api.post(url, json=body).json()
2015-07-09 04:22:54.128 9135 ERROR heat.engine.resource File "/usr/local/lib/python2.7/dist-packages/ceilometerclient/openstack/common/apiclient/client.py", line 358, in post
2015-07-09 04:22:54.128 9135 ERROR heat.engine.resource return self.client_request("POST", url, **kwargs)
2015-07-09 04:22:54.128 9135 ERROR heat.engine.resource File "/usr/local/lib/python2.7/dist-packages/ceilometerclient/openstack/common/apiclient/client.py", line 345, in client_request
2015-07-09 04:22:54.128 9135 ERROR heat.engine.resource self, method, url, **kwargs)
2015-07-09 04:22:54.128 9135 ERROR heat.engine.resource File "/usr/local/lib/python2.7/dist-packages/ceilometerclient/openstack/common/apiclient/client.py", line 261, in client_request
2015-07-09 04:22:54.128 9135 ERROR heat.engine.resource method, self.concat_url(endpoint, url), **kwargs)
2015-07-09 04:22:54.128 9135 ERROR heat.engine.resource File "/usr/local/lib/python2.7/dist-packages/ceilometerclient/openstack/common/apiclient/client.py", line 201, in request
2015-07-09 04:22:54.128 9135 ERROR heat.engine.resource raise exceptions.from_response(resp, method, url)
2015-07-09 04:22:54.128 9135 ERROR heat.engine.resource InternalServerError: 'NoneType' object has no attribute '__getitem__' (HTTP 500) (Request-ID: req-72bf6022-b95b-4f2e-84b1-1603fb44bdda)
And notice that the error is actually happening in ceilometer[2],
2015-07-09 03:23:28.110 9144 ERROR wsme.api [-] Server-side error: "'NoneType' object has no attribute '__getitem__'". Detail:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/wsmeext/pecan.py", line 84, in callfunction
result = f(self, *args, **kwargs)
File "/opt/stack/new/ceilometer/ceilometer/api/controllers/v2/alarms.py", line 749, in post
data.update_actions()
File "/opt/stack/new/ceilometer/ceilometer/api/controllers/v2/alarms.py", line 399, in update_actions
for index, action in enumerate(actions[:]):
TypeError: 'NoneType' object has no attribute '__getitem__'
[1] http://logs.openstack.org/07/198607/5/check/check-heat-dsvm-functional-orig-mysql/7de758d/console.html
[2] http://logs.openstack.org/07/198607/5/check/check-heat-dsvm-functional-orig-mysql/7de758d/logs/screen-h-eng.txt.gz?level=ERROR#_2015-07-09_04_22_54_128
[3] http://logs.openstack.org/97/195697/3/check/check-heat-dsvm-functional-orig-mysql/1b74166/logs/screen-ceilometer-api.txt.gz?level=TRACE
Fix proposed to branch: master
Review: https:/