networkx upgrade breaks DIB

[German Eichberger]

DIB uses a dependency on networkx (see https://pypi.python.org/pypi/networkx/2.0b1) - the latest version 2.0b1 is incompatible with DIB. Fortunately, upper constraints force it to 1.11 but if that ever gets changed adjustments in DIB are needed.

[Ian Wienand (iwienand)]

Yes, we will be restrained by global requirements on this; pip won't select a "b" release anyway.

But, we don't do anything very funky and so it's just the changes that remove 'node' and iterator changes. For reference, to make sure we weren't totally off-base, the following works with 2.0b1 in unit tests

---
--- a/diskimage_builder/block_device/config.py
+++ b/diskimage_builder/block_device/config.py
@@ -190,7 +190,7 @@ def create_graph(config, default_config, state):
             # ensure node names are unique. networkx by default
             # just appends the attribute to the node dict for
             # existing nodes, which is not what we want.
- if node.name in dg.node:
+ if node.name in dg.nodes:
                 raise BlockDeviceSetupException(
                     "Duplicate node name: %s" % (node.name))
             logger.debug("Adding %s : %s", node.name, node)
@@ -210,12 +210,12 @@ def create_graph(config, default_config, state):
         logger.debug("Edges for %s: f:%s t:%s", name,
                      edges_from, edges_to)
         for edge_from in edges_from:
- if edge_from not in dg.node:
+ if edge_from not in dg.nodes:
                 raise BlockDeviceSetupException(
                     "Edge not defined: %s->%s" % (edge_from, name))
             dg.add_edge(edge_from, name)
         for edge_to in edges_to:
- if edge_to not in dg.node:
+ if edge_to not in dg.nodes:
                 raise BlockDeviceSetupException(
                     "Edge not defined: %s->%s" % (name, edge_to))
             dg.add_edge(name, edge_to)
@@ -231,10 +231,9 @@ def create_graph(config, default_config, state):

     # Topological sort (i.e. create a linear array that satisfies
     # dependencies) and return the object list
- call_order_nodes = nx.topological_sort(dg)
- logger.debug("Call order: %s", list(call_order_nodes))
- call_order = [dg.node[n]['obj'] for n in call_order_nodes]
-
+ call_order_nodes = list(nx.topological_sort(dg))
+ logger.debug("Call order: %s", call_order_nodes)
+ call_order = [dg.nodes[n]['obj'] for n in call_order_nodes]
     return dg, call_order
---

I don't think it's really possible to support both versions at the same time ATM, as "nodes" has changed from an instance method in 1.x to a iterator in 2.x. If someone wants to keep fiddling and make something that does, I'm open to it. I think inclusion of [1] would mean we don't have to do anything at all, which seems like a good idea too.

[1] https://github.com/networkx/networkx/issues/2630

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/505908

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/505963

[Ian Wienand (iwienand)]

I'm marking this as a duplicate of the openstack-requirements bug

[OpenStack Infra (hudson-openstack)]

Change abandoned by Ian Wienand (<email address hidden>) on branch: master
Review: https://review.openstack.org/505908
Reason: this was never going to work, but hit a bunch of mirror errors. anyway, we're working on requirements pin

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/505963
Committed: https://git.openstack.org/cgit/openstack/diskimage-builder/commit/?id=a47ff0dd4abbceae786d9a5a4d1ce8042a9eb4f7
Submitter: yolanda.robla (<email address hidden>)
Branch: master

commit a47ff0dd4abbceae786d9a5a4d1ce8042a9eb4f7
Author: Ian Wienand <email address hidden>
Date: Thu Sep 21 14:07:20 2017 +1000

    Support networkx 2.0

    Networkx 2.0 released recently. The main difference for us is that
    "node" is no longer a dictionary and should be accessed via "nodes",
    and the topological_sort returns an interator

    Closes-Bug: 1712693
    Change-Id: I78e89f2261b8b8d28c68b517c1e61691ab40016c

[OpenStack Infra (hudson-openstack)]

Change abandoned by Ian Wienand (<email address hidden>) on branch: master
Review: https://review.openstack.org/506524
Reason: we had to squash this into I82dc61fac6c156a4f0d574290c7632077aa53195