Waiting Tasks (join: n) are never terminated if Workflow execution has Errored

Bug #1849562 reported by Mohammad S Khan
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Mistral
New
Undecided
Unassigned

Bug Description

When workflow execution has reached terminal state (Error) as result of any task running into yaql exceptions,
any task that is waiting for n tasks to complete(e.g. join: 6) is always in this state if task has terminated due to yql exceptions.

wf_ex state_info = "Failed to handle action completion error = Can not evaluate YAQL expression [expression=$.resolved.indexOf("gitHub") = -1, error='resolved', data={}] wf = TestJinjaTemplating2 task = JinjaWithResolve action = nsp.jinja_template"

action execution checker always finds this waiting task but fails to terminate it.
Below is exception from engine logs

2019-10-23 15:57:03.281 3523 ERROR mistral.services.action_execution_checker [req-d32cf572-6c7f-4b89-b18e-7c9c92024341 - - - - -] Action execution checker iteration failed due to unexpected exception.: AttributeError: 'NoneType' object has no attribute 'workflow_execution_id'
2019-10-23 15:57:03.281 3523 ERROR mistral.services.action_execution_checker Traceback (most recent call last):
2019-10-23 15:57:03.281 3523 ERROR mistral.services.action_execution_checker File "/opt/nsp/mistral/lib/python2.7/site-packages/mistral/services/action_execution_checker.py", line 88, in _loop
2019-10-23 15:57:03.281 3523 ERROR mistral.services.action_execution_checker handle_expired_actions()
2019-10-23 15:57:03.281 3523 ERROR mistral.services.action_execution_checker File "/opt/nsp/mistral/lib/python2.7/site-packages/mistral/db/utils.py", line 97, in decorate
2019-10-23 15:57:03.281 3523 ERROR mistral.services.action_execution_checker return retry.call(_with_auth_context, auth_ctx, func, *args, **kw)
2019-10-23 15:57:03.281 3523 ERROR mistral.services.action_execution_checker File "/opt/nsp/mistral/lib/python2.7/site-packages/tenacity/__init__.py", line 358, in call
2019-10-23 15:57:03.281 3523 ERROR mistral.services.action_execution_checker do = self.iter(retry_state=retry_state)
2019-10-23 15:57:03.281 3523 ERROR mistral.services.action_execution_checker File "/opt/nsp/mistral/lib/python2.7/site-packages/tenacity/__init__.py", line 319, in iter
2019-10-23 15:57:03.281 3523 ERROR mistral.services.action_execution_checker return fut.result()
2019-10-23 15:57:03.281 3523 ERROR mistral.services.action_execution_checker File "/opt/nsp/mistral/lib/python2.7/site-packages/concurrent/futures/_base.py", line 455, in result
2019-10-23 15:57:03.281 3523 ERROR mistral.services.action_execution_checker return self.__get_result()
2019-10-23 15:57:03.281 3523 ERROR mistral.services.action_execution_checker File "/opt/nsp/mistral/lib/python2.7/site-packages/tenacity/__init__.py", line 361, in call
2019-10-23 15:57:03.281 3523 ERROR mistral.services.action_execution_checker result = fn(*args, **kwargs)
2019-10-23 15:57:03.281 3523 ERROR mistral.services.action_execution_checker File "/opt/nsp/mistral/lib/python2.7/site-packages/mistral/db/utils.py", line 54, in _with_auth_context
2019-10-23 15:57:03.281 3523 ERROR mistral.services.action_execution_checker return func(*args, **kw)
2019-10-23 15:57:03.281 3523 ERROR mistral.services.action_execution_checker File "/opt/nsp/mistral/lib/python2.7/site-packages/mistral/engine/post_tx_queue.py", line 78, in decorate
2019-10-23 15:57:03.281 3523 ERROR mistral.services.action_execution_checker res = func(*args, **kw)
2019-10-23 15:57:03.281 3523 ERROR mistral.services.action_execution_checker File "/opt/nsp/mistral/lib/python2.7/site-packages/mistral/services/action_execution_checker.py", line 69, in handle_expired_actions
2019-10-23 15:57:03.281 3523 ERROR mistral.services.action_execution_checker action_handler.on_action_complete(action_ex, result)
2019-10-23 15:57:03.281 3523 ERROR mistral.services.action_execution_checker File "/opt/nsp/mistral/lib/python2.7/site-packages/osprofiler/profiler.py", line 159, in wrapper
2019-10-23 15:57:03.281 3523 ERROR mistral.services.action_execution_checker result = f(*args, **kwargs)
2019-10-23 15:57:03.281 3523 ERROR mistral.services.action_execution_checker File "/opt/nsp/mistral/lib/python2.7/site-packages/mistral/engine/action_handler.py", line 34, in on_action_complete
2019-10-23 15:57:03.281 3523 ERROR mistral.services.action_execution_checker action = _build_action(action_ex)
2019-10-23 15:57:03.281 3523 ERROR mistral.services.action_execution_checker File "/opt/nsp/mistral/lib/python2.7/site-packages/osprofiler/profiler.py", line 159, in wrapper
2019-10-23 15:57:03.281 3523 ERROR mistral.services.action_execution_checker result = f(*args, **kwargs)
2019-10-23 15:57:03.281 3523 ERROR mistral.services.action_execution_checker File "/opt/nsp/mistral/lib/python2.7/site-packages/mistral/engine/action_handler.py", line 94, in _build_action
2019-10-23 15:57:03.281 3523 ERROR mistral.services.action_execution_checker action_ex.task_execution.workflow_execution_id
2019-10-23 15:57:03.281 3523 ERROR mistral.services.action_execution_checker AttributeError: 'NoneType' object has no attribute 'workflow_execution_id'

Revision history for this message
Mohammad S Khan (saadkhan4) wrote :
description: updated
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.