Composable networks imposes order-of-operations of environment includes

Bug #1717322 reported by Dan Sneddon
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
Medium
Dan Sneddon

Bug Description

Composable networks generates the environments/network-isolation.yaml and environments/network-isolation-v6.yaml files on the fly according to the network_data.yaml. If the Management network is enabled, but the management network is not enabled on a particular role (it is not enabled on any role by default), then the ports will point to network/ports/noop.yaml.

The problem is that this introduces a regression where the Management network ports won't be created if network-management.yaml is included before network-isolation.yaml (or the same with the IPv6 versions of those files). The same thing will happen with custom environment files if network-isolation[-v6].yaml is included after the custom environment.

In order to avoid this regression, the network-isolation[-v6].yaml files should not include noop.yaml references for networks that are not enabled on a role. The noop.yaml was only really required before we generated the network config, since we hard-coded the network ports in each role. The port had to exist, so noop.yaml would populate that port by reusing the control plane port.

Example:

network-isolation.yaml:
  OS::TripleO::Controller::Ports::ManagementPort: ../network/ports/noop.yaml

(the above line was previously not included in the network-isolation.yaml)

network-management.yaml:

  OS::TripleO::Controller::Ports::ManagementPort: ../network/ports/management.yaml

If the network-management.yaml is included before the network-isolation.yaml, this assignment will be overwritten with noop.yaml, and the management network will not work.

We shouldn't need to use noop.yaml at all in the network environment files. We already set all ports to noop.yaml in the role definitions and overcloud-resource-registry.yaml that are generated on the fly. Doing the same thing in the environment files introduces the order-of-operation issue when including environment files.

Changed in tripleo:
status: New → Triaged
milestone: none → queens-1
importance: Undecided → High
Dan Sneddon (dsneddon)
Changed in tripleo:
importance: High → Medium
assignee: nobody → Dan Sneddon (dsneddon)
Changed in tripleo:
status: Triaged → In Progress
Revision history for this message
Steven Hardy (shardy) wrote :

To clarify, wasn't this always order dependent, as we previously had hard-coded noops in the network-isolation.yaml?

Revision history for this message
Steven Hardy (shardy) wrote :

Ok having looked closer I see we defined the noops in the base environment template, e.g:

https://github.com/openstack/tripleo-heat-templates/blob/stable/ocata/overcloud-resource-registry-puppet.j2.yaml#L32

We still do that:

https://github.com/openstack/tripleo-heat-templates/blob/master/overcloud-resource-registry-puppet.j2.yaml#L29

So as Dan says we need to remove generating the ports in network-isolation.yaml, or validate the ordering

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (master)

Reviewed: https://review.openstack.org/504180
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=9b08df3733257ac0fbc150a4071aec051e073ef7
Submitter: Jenkins
Branch: master

commit 9b08df3733257ac0fbc150a4071aec051e073ef7
Author: Dan Sneddon <email address hidden>
Date: Thu Sep 14 13:26:53 2017 -0600

    Remove extra noop.yaml ports from network-isolation files.

    The environments/network-isolation[-v6].yaml files have an
    unneeded reference to network/ports/noop.yaml for unused
    networks.

    This introduces a regression where environment files that
    define the networks and ports on a per-role basis can
    cancel out other environment files. See bug # 1717322.

    The overcloud-resource-registry.j2.yaml already uses noop.yaml
    for every network on every role (whether or not the networks
    are enabled, or whether the particular network is supposed
    to be on a role. So having noop.yaml specified for every
    role in network-isolation[-v6].yaml is not needed and can
    cause issues with upgrades if the environments are not
    included in a specific order.

    Change-Id: If06407e5235587af090ede44674bf9c7e08e340e
    Closes-bug: 1717322

Changed in tripleo:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-heat-templates (stable/pike)

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/505735

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (stable/pike)

Reviewed: https://review.openstack.org/505735
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=4e700bce608ba711486c8f6a6b2bcf72c4423794
Submitter: Jenkins
Branch: stable/pike

commit 4e700bce608ba711486c8f6a6b2bcf72c4423794
Author: Dan Sneddon <email address hidden>
Date: Thu Sep 14 13:26:53 2017 -0600

    Remove extra noop.yaml ports from network-isolation files.

    The environments/network-isolation[-v6].yaml files have an
    unneeded reference to network/ports/noop.yaml for unused
    networks.

    This introduces a regression where environment files that
    define the networks and ports on a per-role basis can
    cancel out other environment files. See bug # 1717322.

    The overcloud-resource-registry.j2.yaml already uses noop.yaml
    for every network on every role (whether or not the networks
    are enabled, or whether the particular network is supposed
    to be on a role. So having noop.yaml specified for every
    role in network-isolation[-v6].yaml is not needed and can
    cause issues with upgrades if the environments are not
    included in a specific order.

    Change-Id: If06407e5235587af090ede44674bf9c7e08e340e
    Closes-bug: 1717322
    (cherry picked from commit 9b08df3733257ac0fbc150a4071aec051e073ef7)

tags: added: in-stable-pike
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-heat-templates 7.0.2

This issue was fixed in the openstack/tripleo-heat-templates 7.0.2 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-heat-templates 8.0.0.0b1

This issue was fixed in the openstack/tripleo-heat-templates 8.0.0.0b1 development milestone.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.