Mistral workflow stuck in RUNNING state after error "list index of range" encountered

[jui chandwaskar]

A task was marked as RUNNING despite the error reported in mistral engine and executor logs:

2018-03-16 14:01:54.000 5715 INFO workflow_trace [-] Action 'nova.floating_ips_findall' (2f13d182-13ef-4eb2-9586-08e859a0e9f1)(task=search_for_ip) [RUNNING -> SUCCESS, result = []]
2018-03-16 14:01:54.029 5715 INFO workflow_trace [-] Task 'search_for_ip' (903e328e-5fad-4e5f-8cf5-a2d189f7e443) [RUNNING -> SUCCESS, msg=None] (execution_id=8bcedf9b-3891-4a5c-bb4b-d7829a6f8908)
2018-03-16 14:01:54.064 5715 ERROR oslo_messaging.rpc.server [-] Exception during message handling
2018-03-16 14:01:54.064 5715 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2018-03-16 14:01:54.064 5715 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 133, in _process_incoming
2018-03-16 14:01:54.064 5715 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
2018-03-16 14:01:54.064 5715 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 150, in dispatch
2018-03-16 14:01:54.064 5715 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
2018-03-16 14:01:54.064 5715 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 121, in _do_dispatch
2018-03-16 14:01:54.064 5715 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
2018-03-16 14:01:54.064 5715 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/mistral/engine/rpc_backend/rpc.py", line 203, in on_action_complete
2018-03-16 14:01:54.064 5715 ERROR oslo_messaging.rpc.server return self._engine.on_action_complete(action_ex_id, result, wf_action)
2018-03-16 14:01:54.064 5715 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/mistral/utils/__init__.py", line 113, in _logged
2018-03-16 14:01:54.064 5715 ERROR oslo_messaging.rpc.server return func(*args, **kw)
...
...
2018-03-16 14:01:54.064 5715 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/yaql/language/runner.py", line 51, in call
2018-03-16 14:01:54.064 5715 ERROR oslo_messaging.rpc.server result = delegate()
2018-03-16 14:01:54.064 5715 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/yaql/language/runner.py", line 142, in <lambda>
2018-03-16 14:01:54.064 5715 ERROR oslo_messaging.rpc.server return lambda: delegate()
2018-03-16 14:01:54.064 5715 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/yaql/language/specs.py", line 341, in func
2018-03-16 14:01:54.064 5715 ERROR oslo_messaging.rpc.server six.iteritems(keyword_args)))
2018-03-16 14:01:54.064 5715 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/yaql/standard_library/collections.py", line 180, in list_indexer
2018-03-16 14:01:54.064 5715 ERROR oslo_messaging.rpc.server return lst[index]
2018-03-16 14:01:54.064 5715 ERROR oslo_messaging.rpc.server IndexError: list index out of range
2018-03-16 14:01:54.064 5715 ERROR oslo_messaging.rpc.server

Still the output shows:
mistral task-list
+--------------------+---------------+--------------------+--------------------+---------+------------+
| ID | Name | Workflow name | Execution ID | State | State info |
+--------------------+---------------+--------------------+--------------------+---------+------------+
| 9efa4482-8205 | create_vm | std.create_instanc | 8bcedf9b-3891 | SUCCESS | None |
| -434e- | | e | -4a5c-bb4b- | | |
| bad8-fb3defa843f1 | | | d7829a6f8908 | | |
| 903e328e-5fad- | search_for_ip | std.create_instanc | 8bcedf9b-3891 | RUNNING | None |
| 4e5f- | | e | -4a5c-bb4b- | | |
| 8cf5-a2d189f7e443 | | | d7829a6f8908 | | |
+--------------------+---------------+--------------------+--------------------+---------+------------+

Is this a known issue?

[Dougal Matthews (d0ugal)]

The action "nova.floating_ips_findall" was recently removed as it was deprecated in novaclient - that may be the cause of this failure. Either way the Mistral engine shouldn't get stuck.