The needs to treat subflows as standalone entity

[Sherif Abdelwahab]

Version: 1.16.0

The current implementation of taskflow, which flatten out all tasks in a flow even if they belong to different suflows behaves strangely in my opinion.

Please check the attached file (ex01.py). Assume that you have 4 tasks: task1, task2, task3, task4 such as

1. task2 depends on task1 with decider1
2. task3 depends on task1 with decider2
3. decider1 and decider2 are always mutually exclusive (decider1 = not decider2).
4. task4 shall run if either task2 "or" task3 transition to SUCCESS.

The current taskflow implementation always link task4 to task2 and task3 so that it depends on "both tasks". Both tasks must run so that task4 run. (See first figure in attached file)

Current: task4 never runs!

If we put task1, task2, and task3 in a subflow, it is supposed that this subflow becomes a separate entity. In this case if task4 is linked to this subflow, it should depends on SUCCESS of the whole subflow not individual tasks within this subflow. Unfortunately, this is not the case and taskflow flatten out the tasks (see second figure in attachments). The result is: task4 never runs also!

Expected behavior is illustrated in Third figure in attachment. I think if we use a subflow1, other linked subflows or tasks to subflow1 should depend on the status of subflow1, and not on "all" tasks in the last stage of subflow1.

[Joshua Harlow (harlowja)]

So looking at the example, the reason that it operates the way it does is due to deciders affecting all successors, which is required if there is a data dependency between the associated tasks (for example in your first diagram task 2 -> task 4 could have a data dependency, so task 4 really can't run without that).

As for the subflow idea u had, treating the whole subflow result as a single entity, yes that was considered, and I'm still fine with considering it if we want to do this, the decomposing of subflows into a single graph was an early decision that the team went back and forth on for a while and we ended up with where we are now (where subflows are just a structural component, and have no data, result or status ...); now maybe this could change, do u want to work on that? (the change would require figuring out what exactly is the result of a subflow, where that information is stored...)

https://bugs.launchpad.net/taskflow/+bug/1480907 is adjusting some of this as well, so that may help u this idea to...

[Joshua Harlow (harlowja)]

If u are so desiring to work on this (which I'm fine with); a blueprint or spec would be great.

https://github.com/openstack/oslo-specs/tree/master/specs

A spec I think would be the better way, since blueprints sort of suck for discussion and/or commenting on and this type of change would probably require that kind of dicussion to be able to make sure we do it right.

[Joshua Harlow (harlowja)]

Another review that was similar to this question:

https://review.openstack.org/#/c/136171/

I think the docs could be made better to denote that subflows really are just 'structural' entities (in that they enforce structure instead of having an actual result, having status...); that is of course unless we change that to be something else...

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/246051
Committed: https://git.openstack.org/cgit/openstack/taskflow/commit/?id=8e8156c488dea8ae876b112c30e41e60da4f5be7
Submitter: Jenkins
Branch: master

commit 8e8156c488dea8ae876b112c30e41e60da4f5be7
Author: Joshua Harlow <email address hidden>
Date: Mon Nov 16 16:27:42 2015 -0800

    Allow for alterations in decider 'area of influence'

    Christmas came early.

    Closes-Bug: #1479466

    Change-Id: I931d826690c925f022dbfffe9afb7bf41345b1d0

[Davanum Srinivas (DIMS) (dims-v)]

This issue was fixed in the openstack/taskflow 1.26.0 release.