Taskflow not working with networkx > 2.0

[Michal Arbet]

Taskflow is not working with networkx > 2.0 as it is noted in requirements.txt ( networkx>=1.10 # BSD )
Networkx > 2.0 is not compatible with 1.X versions.

There is migration guide from 1.X to 2.0 :

https://networkx.github.io/documentation/stable/release/migration_guide_from_1.x_to_2.0.html
https://networkx.github.io/documentation/networkx-2.0/release/release_2.0.html

I started to work on rework taskflow code, but still failing tests.... ( now failing 55, before failing 1000+ )

Thanks,
Michal Arbet

[Michal Arbet (michalarbet)]

This is partial patch which is fixing code, and also tests, but still failing 57 ( before failing 1000+ ) tests for me.

Tests failing with patch applied :

taskflow.tests.unit.patterns.test_graph_flow.TargetedGraphFlowTest.test_targeted_flow_one_node ... FAIL
taskflow.tests.unit.patterns.test_graph_flow.TargetedGraphFlowTest.test_recache_on_add ... FAIL
taskflow.tests.unit.patterns.test_graph_flow.TargetedGraphFlowTest.test_targeted_flow_restricts ... FAIL
taskflow.tests.unit.patterns.test_graph_flow.TargetedGraphFlowTest.test_recache_on_add_no_deps ... FAIL
taskflow.tests.unit.patterns.test_graph_flow.TargetedGraphFlowTest.test_recache_on_link ... FAIL
taskflow.tests.unit.test_engines.ParallelEngineWithEventletTest.test_graph_flow_conditional_ignore_reset ... FAIL
taskflow.tests.unit.test_engines.ParallelEngineWithEventletTest.test_graph_flow_conditional_jumps_across ... FAIL
taskflow.tests.unit.test_engines.ParallelEngineWithEventletTest.test_run_graph_flow_decider_jump_over_atom ... FAIL
taskflow.tests.unit.test_engines.ParallelEngineWithEventletTest.test_graph_flow_conditional_jumps_across_2 ... FAIL
taskflow.tests.unit.test_engines.ParallelEngineWithEventletTest.test_graph_flow_diamond_ignored ... FAIL
taskflow.tests.unit.test_engines.ParallelEngineWithEventletTest.test_run_graph_flow_decider_jump_over_bad_atom ... FAIL
taskflow.tests.unit.test_engines.ParallelEngineWithEventletTest.test_run_graph_flow_decider_various_depths ... FAIL
taskflow.tests.unit.test_engines.ParallelEngineWithProcessTest.test_graph_flow_conditional_jumps_across_2 ... FAIL
taskflow.tests.unit.test_engines.SerialEngineTest.test_graph_flow_conditional ... FAIL
taskflow.tests.unit.test_engines.SerialEngineTest.test_run_graph_flow_decider_various_depths ... FAIL
taskflow.tests.unit.test_engines.SerialEngineTest.test_graph_flow_conditional_ignore_reset ... FAIL
taskflow.tests.unit.test_engines.SerialEngineTest.test_graph_flow_diamond_ignored ... FAIL
taskflow.tests.unit.test_engines.ParallelEngineWithProcessTest.test_run_graph_flow_decider_various_depths ... FAIL
taskflow.tests.unit.test_engines.ParallelEngineWithProcessTest.test_graph_flow_conditional_jumps_across ... FAIL
taskflow.tests.unit.test_engines.ParallelEngineWithEventletTest.test_graph_flow_conditional ... FAIL
taskflow.tests.unit.test_engines.ParallelEngineWithEventletTest.test_graph_flow_conditional_history ... FAIL
taskflow.tests.unit.test_engines.ParallelEngineWithProcessTest.test_run_graph_flow_decider_revert ... FAIL
taskflow.tests.unit.test_engines.ParallelEngineWithThreadsTest.test_run_graph_flow_decider_various_depths ... FAIL
taskflow.tests.unit.test_engines.ParallelEngineWithThreadsTest.test_graph_flow_conditional ... FAIL
taskflow.tests.unit.test_engines.ParallelEngineWithProcessTest.test_run_graph_flow_decider_jump_over_atom ... FAIL
taskflow.tests.unit.test_engines.ParallelEngineWithProcessTest.test_graph_flow_conditional ... FAIL
taskflow.tests.unit.test_engines.SerialEngineTest.test_graph_flow_conditional_jumps_across_2 ... FAIL
taskflow.tests.unit.test_engines.ParallelEngineWithProcessTest.test_run_graph_flow_decider_jump_over_bad_atom ... FAIL
taskflow.tests.unit.test_engines.SerialEngin...

[Michal Arbet (michalarbet)]

Full log of failing tests

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/577833

[Gianfranco Costamagna (costamagnagianfranco)]

I'm uploading the change in cosmic soon, I merged networkx and this is the last blocker for it going in release

[Gianfranco Costamagna (costamagnagianfranco)]

thanks for the patch!

[Gianfranco Costamagna (costamagnagianfranco)]

It seems to be still not enough...
https://launchpad.net/ubuntu/+source/python-taskflow/3.1.0-0ubuntu5/+build/15064887

[Michal Arbet (michalarbet)]

Code was rewritten after discussion on irc with openstack-global-requirements guys to support also old version of networkx < 2.0 . Reason is that we needs to release taskflow supporting both versions , to pass requirements tests with old networkx ..and in the future remove support for networkx < 2.0

Just check the review

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/577833
Committed: https://git.openstack.org/cgit/openstack/taskflow/commit/?id=d985c5a256bcf8f67e80235ba387599f448f2c49
Submitter: Zuul
Branch: master

commit d985c5a256bcf8f67e80235ba387599f448f2c49
Author: Michal Arbet <email address hidden>
Date: Mon Jun 25 16:06:18 2018 +0200

    Fix code to support networkx > 1.0

    With the release of NetworkX 2.0 the reporting API was
    moved to view/iterator model. Many methods were moved from
    reporting lists or dicts to iterating over the information.
    Methods that used to return containers now return views and
    methods that returned iterators have been removed in networkx.
    Because of this change in NetworkX 2.0 , taskflow code
    have to be changed also to support networkx > 2.0

    Change-Id: I23c226f37bd85c1e38039fbcb302a2d0de49f333
    Closes-Bug: #1778115

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/taskflow 3.2.0 release.