Published vars lost for complex parallel join

[Winson Chan]

The sample WF below have two parallel branches. I'm expecting output to be the following where the vars are true. However, the join task named "done" following completion of the parallel branches are not executed. But the WF execution reported SUCCESS.

EXPECTED:
{
    "is_done": true,
    "var1": true,
    "var2": true
}

ACTUAL:
~$ cat ~/tmp/test-complex-join.yaml
version: '2.0'
name: test-complex-join

workflows:

    main:
        type: direct
        output:
            var1: <% $.var1 %>
            var2: <% $.var2 %>
            is_done: <% $.is_done %>
        tasks:
            init:
                publish:
                    var1: false
                    var2: false
                    is_done: false
                policies:
                    wait-after: 1
                on-success:
                    - branch1
                    - branch2
            branch1:
                workflow: work
                publish:
                    var1: true
                policies:
                    retry:
                        count: 10
                        delay: 2
                on-complete:
                    - done
            branch2:
                publish:
                    var2: true
                policies:
                    wait-before: 10
                on-complete:
                    - done
            done:
                join: all
                publish:
                    is_done: true

    work:
        type: direct
        tasks:
            do:
                action: std.echo output="Doing..."
                policies:
                    wait-before: 1
                    wait-after: 1
                on-success:
                    - exit
            exit:
                action: std.echo output="Exiting..."

~$ mistral workbook-create ~/tmp/test-complex-join.yaml
Starting new HTTP connection (1): localhost
+------------+----------------------------+
| Field | Value |
+------------+----------------------------+
| Name | test-complex-join |
| Tags | <none> |
| Created at | 2015-02-22 23:08:19.190381 |
| Updated at | None |
+------------+----------------------------+

~$ mistral execution-create test-complex-join.main
Starting new HTTP connection (1): localhost
+------------+--------------------------------------+
| Field | Value |
+------------+--------------------------------------+
| ID | e96f5f2a-f231-4cef-b52a-549becfd8de5 |
| Workflow | test-complex-join.main |
| State | RUNNING |
| Created at | 2015-02-22T23:08:29.843989 |
| Updated at | 2015-02-22T23:08:29.870009 |
+------------+--------------------------------------+

~$ mistral execution-list
Starting new HTTP connection (1): localhost
+--------------------------------------+------------------------+---------+---------------------+---------------------+
| ID | Workflow | State | Created at | Updated at |
+--------------------------------------+------------------------+---------+---------------------+---------------------+
| e96f5f2a-f231-4cef-b52a-549becfd8de5 | test-complex-join.main | SUCCESS | 2015-02-22 23:08:29 | 2015-02-22 23:08:33 |
| c9517936-0d6b-4791-8f1b-ff7712d18587 | test-complex-join.work | SUCCESS | 2015-02-22 23:08:31 | 2015-02-22 23:08:33 |
+--------------------------------------+------------------------+---------+---------------------+---------------------+

~$ mistral task-list
Starting new HTTP connection (1): localhost
+--------------------------------------+---------+------------------------+--------------------------------------+---------+
| ID | Name | Workflow name | Execution ID | State |
+--------------------------------------+---------+------------------------+--------------------------------------+---------+
| 391dc013-510b-4589-887f-21dbe3d988d0 | init | test-complex-join.main | e96f5f2a-f231-4cef-b52a-549becfd8de5 | SUCCESS |
| 022863d8-46bd-4caa-ae72-baba77ff827d | branch2 | test-complex-join.main | e96f5f2a-f231-4cef-b52a-549becfd8de5 | SUCCESS |
| 0f883ca8-c43c-4d33-930a-571b6ab782df | do | test-complex-join.work | c9517936-0d6b-4791-8f1b-ff7712d18587 | SUCCESS |
| f643284d-039c-4899-a454-8ef584c5373c | branch1 | test-complex-join.main | e96f5f2a-f231-4cef-b52a-549becfd8de5 | SUCCESS |
| cd97ce7e-f029-44a6-8896-8d3c5522060c | exit | test-complex-join.work | c9517936-0d6b-4791-8f1b-ff7712d18587 | SUCCESS |
+--------------------------------------+---------+------------------------+--------------------------------------+---------+

~$ mistral execution-get-output e96f5f2a-f231-4cef-b52a-549becfd8de5
Starting new HTTP connection (1): localhost
{
    "is_done": false,
    "var1": true,
    "var2": false
}

[Winson Chan (winson-c-chan)]

Please note the WF is slightly different. This is another bug. There's a race condition on publishing vars. Whereas the one above failed to execute the join task.

EXPECTED:
{
    "is_done": true,
    "var1": true,
    "var2": true
}

ACTUAL:
{
    "is_done": true,
    "var1": false,
    "var2": true
}

~/st2$ cat ~/tmp/test-complex-join.yaml
version: '2.0'
name: test-complex-join

workflows:

    main:
        type: direct
        output:
            var1: <% $.var1 %>
            var2: <% $.var2 %>
            is_done: <% $.is_done %>
        tasks:
            init:
                publish:
                    var1: false
                    var2: false
                    is_done: false
                policies:
                    wait-after: 1
                on-success:
                    - branch1
                    - branch2
            branch1:
                workflow: work
                publish:
                    var1: true
                policies:
                    retry:
                        count: 10
                        delay: 2
                on-success:
                    - done
            branch2:
                publish:
                    var2: true
                on-success:
                    - done
            done:
                join: all
                publish:
                    is_done: true

    work:
        type: direct
        tasks:
            do:
                action: std.echo output="Doing..."
                on-success:
                    - exit
            exit:
                action: std.echo output="Exiting..."

~$ mistral workbook-create ~/tmp/test-complex-join.yaml
Starting new HTTP connection (1): localhost
+------------+----------------------------+
| Field | Value |
+------------+----------------------------+
| Name | test-complex-join |
| Tags | <none> |
| Created at | 2015-02-22 23:15:42.201216 |
| Updated at | None |
+------------+----------------------------+

~$ mistral execution-create test-complex-join.main
Starting new HTTP connection (1): localhost
+------------+--------------------------------------+
| Field | Value |
+------------+--------------------------------------+
| ID | 95b37c87-02c3-4991-903e-0a2619419a2d |
| Workflow | test-complex-join.main |
| State | RUNNING |
| Created at | 2015-02-22T23:15:52.054730 |
| Updated at | 2015-02-22T23:15:52.081809 |
+------------+--------------------------------------+

~$ mistral execution-list
Starting new HTTP connection (1): localhost
+--------------------------------------+------------------------+---------+---------------------+---------------------+
| ID | Workflow | State | Created at | Updated at |
+--------------------------------------+------------------------+---------+---------------------+---------------------+
| 95b37c87-02c3-4991-903e-0a2619419a2d | test-complex-join.main | SUCCESS | 2015-02-22 23:15:52 | 2015-02-22 23:15:54 |
| 1f...

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/159860

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/159860
Committed: https://git.openstack.org/cgit/stackforge/mistral/commit/?id=ceff43b1cdeacfb5a140dc75c8811d0bb92604ee
Submitter: Jenkins
Branch: master

commit ceff43b1cdeacfb5a140dc75c8811d0bb92604ee
Author: Nikolay Mahotkin <email address hidden>
Date: Fri Feb 27 14:49:07 2015 +0300

    Fixing published vars for parallel tasks (and join)

     * Added one more complex failing test
     * This is temp fix

    Partial-Bug: #1424461

    Change-Id: I55e714d95e38c5803d758101a8bf4d7a9a04b2b1