Mistral

Mistral not rendering multiple Jinja expression properly

Bug #1712133 reported by Nick Maludy
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Mistral
Fix Released
High
Nick Maludy
Mistral queens-1 "q1"

Bug Description

## Problem

I have a Mistral workflow (below) that has a line of Jinja with multiple expressions in it. This is 100% valid Jinja that renders properly as show in code below. In Mistral i receive the following error:

## Error

```
2017-08-16 13:27:27.700 935 ERROR mistral.engine.task_handler [-] Failed to handle action completion [error=Can not evaluate Jinja expression [expression= _.directory }}/{{ _.filename , error=chunk after expression, data={}], wf=examples.broken_jinja_example, task=create_path_jinja, action=std.noop]:
Traceback (most recent call last):
  File "/opt/stackstorm/mistral/lib/python2.7/site-packages/mistral/engine/task_handler.py", line 105, in _on_action_complete
    task.on_action_complete(action_ex)
  File "/opt/stackstorm/mistral/lib/python2.7/site-packages/osprofiler/profiler.py", line 153, in wrapper
    return f(*args, **kwargs)
  File "/opt/stackstorm/mistral/lib/python2.7/site-packages/mistral/engine/tasks.py", line 264, in on_action_complete
    self.complete(state, state_info)
  File "/opt/stackstorm/mistral/lib/python2.7/site-packages/osprofiler/profiler.py", line 153, in wrapper
    return f(*args, **kwargs)
  File "/opt/stackstorm/mistral/lib/python2.7/site-packages/mistral/engine/tasks.py", line 163, in complete
    data_flow.publish_variables(self.task_ex, self.task_spec)
  File "/opt/stackstorm/mistral/lib/python2.7/site-packages/mistral/workflow/data_flow.py", line 211, in publish_variables
    task_ex.published = expr.evaluate_recursively(data, expr_ctx)
  File "/opt/stackstorm/mistral/lib/python2.7/site-packages/mistral/expressions/__init__.py", line 96, in evaluate_recursively
    data[key] = _evaluate_item(data[key], context)
  File "/opt/stackstorm/mistral/lib/python2.7/site-packages/mistral/expressions/__init__.py", line 79, in _evaluate_item
    return evaluate(item, context)
  File "/opt/stackstorm/mistral/lib/python2.7/site-packages/mistral/expressions/__init__.py", line 71, in evaluate
    return evaluator.evaluate(expression, context)
  File "/opt/stackstorm/mistral/lib/python2.7/site-packages/mistral/expressions/jinja_expression.py", line 140, in evaluate
    result = JinjaEvaluator.evaluate(patterns[0][1], data_context)
  File "/opt/stackstorm/mistral/lib/python2.7/site-packages/mistral/expressions/jinja_expression.py", line 89, in evaluate
    ", data=%s]" % (expression, str(e), data_context)
JinjaEvaluationException: Can not evaluate Jinja expression [expression= _.directory }}/{{ _.filename , error=chunk after expression, data={}]
: JinjaEvaluationException: Can not evaluate Jinja expression [expression= _.directory }}/{{ _.filename , error=chunk after expression, data={}]
2017-08-16 13:27:27.700 935 INFO workflow_trace [-] Task 'create_path_jinja' (b7758c32-e182-4dfa-8f5e-4b0437473372) [SUCCESS -> ERROR, msg=Failed to handle action completion [error=Can not evaluate Jinja expression [expression= _.directory }}/{{ _.filename , error=chunk after expression, data={}], wf=examples.broken_jinja_example, task=create_path_jinja, action=std.noop]:
Traceback (most recent call last):
  File "/opt/stackstorm/mistral/lib/python2.7/site-packages/mistral/engine/task_handler.py", line 105, in _on_action_complete
    task.on_action_complete(action_ex)
  File "/opt/stackstorm/mistral/lib/python2.7/site-packages/osprofiler/profiler.py", line 153, in wrapper
    return f(*args, **kwargs)
  File "/opt/stackstorm/mistral/lib/python2.7/site-packages/mistral/engine/tasks.py", line 264, in on_action_complete
    self.complete(state, state_info)
  File "/opt/stackstorm/mistral/lib/python2.7/site-packages/osprofiler/profiler.py", line 153, in wrapper
    return f(*args, **kwargs)
  File "/opt/stackstorm/mistral/lib/python2.7/site-packages/mistral/engine/tasks.py", line 163, in complete
    data_flow.publish_variables(self.task_ex, self.task_spec)
  File "/opt/stackstorm/mistral/lib/python2.7/site-packages/mistral/workflow/data_flow.py", line 211, in publish_variables
    task_ex.published = expr.evaluate_recursively(data, expr_ctx)
  File "/opt/stackstorm/mistral/lib/python2.7/site-packages/mistral/expressions/__init__.py", line 96, in evaluate_recursively
    data[key] = _evaluate_item(data[key], context)
  File "/opt/stackstorm/mistral/lib/python2.7/site-packages/mistral/expressions/__init__.py", line 79, in _evaluate_item
    return evaluate(item, context)
  File "/opt/stackstorm/mistral/lib/python2.7/site-packages/mistral/expressions/__init__.py", line 71, in evaluate
    return evaluator.evaluate(expression, context)
  File "/opt/stackstorm/mistral/lib/python2.7/site-packages/mistral/expressions/jinja_expression.py", line 140, in evaluate
    result = JinjaEvaluator.evaluate(patterns[0][1], data_context)
  File "/opt/stackstorm/mistral/lib/python2.7/site-packages/mistral/expressions/jinja_expression.py", line 89, in evaluate
    ", data=%s]" % (expression, str(e), data_context)
JinjaEvaluationException: Can not evaluate Jinja expression [expression= _.directory }}/{{ _.filename , error=chunk after expression, data={}]
] (execution_id=bf923a97-430b-4c4d-90ec-b5d67ac3b4a5): JinjaEvaluationException: Can not evaluate Jinja expression [expression= _.directory }}/{{ _.filename , error=chunk after expression, data={}]
2017-08-16 13:27:27.704 935 INFO workflow_trace [-] Workflow 'examples.broken_jinja_example' [RUNNING -> ERROR, msg=Failed to handle action completion [error=Can not evaluate Jinja expression [expression= _.directory }}/{{ _.filename , error=chunk after expression, data={}], wf=examples.broken_jinja_example, task=create_path_jinja, action=std.noop]:
Traceback (most recent call last):
  File "/opt/stackstorm/mistral/lib/python2.7/site-packages/mistral/engine/task_handler.py", line 105, in _on_action_complete
    task.on_action_complete(action_ex)
  File "/opt/stackstorm/mistral/lib/python2.7/site-packages/osprofiler/profiler.py", line 153, in wrapper
    return f(*args, **kwargs)
  File "/opt/stackstorm/mistral/lib/python2.7/site-packages/mistral/engine/tasks.py", line 264, in on_action_complete
    self.complete(state, state_info)
  File "/opt/stackstorm/mistral/lib/python2.7/site-packages/osprofiler/profiler.py", line 153, in wrapper
    return f(*args, **kwargs)
  File "/opt/stackstorm/mistral/lib/python2.7/site-packages/mistral/engine/tasks.py", line 163, in complete
    data_flow.publish_variables(self.task_ex, self.task_spec)
  File "/opt/stackstorm/mistral/lib/python2.7/site-packages/mistral/workflow/data_flow.py", line 211, in publish_variables
    task_ex.published = expr.evaluate_recursively(data, expr_ctx)
  File "/opt/stackstorm/mistral/lib/python2.7/site-packages/mistral/expressions/__init__.py", line 96, in evaluate_recursively
    data[key] = _evaluate_item(data[key], context)
  File "/opt/stackstorm/mistral/lib/python2.7/site-packages/mistral/expressions/__init__.py", line 79, in _evaluate_item
    return evaluate(item, context)
  File "/opt/stackstorm/mistral/lib/python2.7/site-packages/mistral/expressions/__init__.py", line 71, in evaluate
    return evaluator.evaluate(expression, context)
  File "/opt/stackstorm/mistral/lib/python2.7/site-packages/mistral/expressions/jinja_expression.py", line 140, in evaluate
    result = JinjaEvaluator.evaluate(patterns[0][1], data_context)
  File "/opt/stackstorm/mistral/lib/python2.7/site-packages/mistral/expressions/jinja_expression.py", line 89, in evaluate
    ", data=%s]" % (expression, str(e), data_context)
JinjaEvaluationException: Can not evaluate Jinja expression [expression= _.directory }}/{{ _.filename , error=chunk after expression, data={}]
] (execution_id=bf923a97-430b-4c4d-90ec-b5d67ac3b4a5): JinjaEvaluationException: Can not evaluate Jinja expression [expression= _.directory }}/{{ _.filename , error=chunk after expression, data={}]
```

## Workflow

```
version: '2.0'

examples.broken_jinja_example:
  description: Workflow that reproduces a problem in Mistral with Jinja expressions
  type: direct
  input:
    - directory
    - filename

  output:
    yaql_path: "{{ _.yaql_path }}"
    jinja_path: "{{ _.jinja_path }}"

  tasks:

    create_path_yaql:
      action: std.noop
      publish:
        yaql_path: <% $.directory %>/<% $.filename %>
      on-success:
        - create_path_jinja

    create_path_jinja:
      action: std.noop
      publish:
        jinja_path: "{{ _.directory }}/{{ _.filename }}"
```

## Python code that renders the Jinja properly

```
#!/usr/bin/env python
import jinja2
import os

class JinjaUtils:

    @staticmethod
    def render_str(jinja_template_str, context):
        env = jinja2.Environment()
        tmpl = env.from_string(jinja_template_str)
        return tmpl.render(context)

if __name__ == "__main__":

    context = {'_': {'directory': '/opt/stackstorm/configs',
                     'filename': 'test.yaml'} }
    template = "{{ _.directory }}/{{ _.filename }}"
    print JinjaUtils.render_str(template, context)
```

Renat Akhmerov (rakhmerov)
Changed in mistral:
milestone: none → queens-1
assignee: nobody → Nick Maludy (nmaludy)
importance: Undecided → High
Renat Akhmerov (rakhmerov)
Changed in mistral:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to mistral (master)

Reviewed: https://review.openstack.org/502198
Committed: https://git.openstack.org/cgit/openstack/mistral/commit/?id=10b27fcdeae4e0f0e5d1f911b9341e4eff33d4c1
Submitter: Jenkins
Branch: master

commit 10b27fcdeae4e0f0e5d1f911b9341e4eff33d4c1
Author: Nick Maludy <email address hidden>
Date: Fri Sep 8 20:01:39 2017 -0400

    Fixes issue rendering strings containing multiple jinja expressions

    Closes-Bug: #1712133
    Change-Id: I058e9858e4a4a9a94e0948df38758446e68bf5b9

Changed in mistral:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/mistral 6.0.0.0b1

This issue was fixed in the openstack/mistral 6.0.0.0b1 development milestone.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.