Tripleo custom network config templates ignored in overcloud deploy

[Adam Ratcliff]

My Rocky tripleo deploy fails - because the network deployment is not correct resulting in a ping failure
(in function ping_default_gateways() from openstack-tripleo-heat-templates/validation-scripts/all-nodes.sh)

It seems like my config is being ignored particularly the vlan config and subnets from my custom template file: network-environment-overrides.yaml passed on the deploy command line.

openstack overcloud deploy --templates -r /home/stack/templates/roles_data.yaml -e /home/stack/templates/network-environment-overrides.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/network-environment.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml --ntp-server 172.26.1.60 --timeout 190 --control-scale 1 --compute-flavor compute --compute-scale 2 --ceph-storage-flavor ceph-storage --ceph-storage-scale 1 -e /home/stack/templates/ceph-config.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/ceph-ansible/ceph-ansible.yaml -e /home/stack/templates/overcloud_images.yaml

The deployed network config data gathered from the deployed nodes after the failed deployment: vlan numbers look like defaults (10 20 30 ..) , not my customised (120 131 132 133..) same with the subnets (compare network-environment-overrides.yaml with controller network info in pastebins)

Some of my console log is attached (watch out for the console control characters in that one) and the output of openstack overcloud failures and my templates folder (all the nic configs are in there but I have selected the single-nic-vlans in network-environment-overrides.yaml).

This is "new", I had this part working last week. What changed? As part of debugging other things I archived my undercloud and started fresh on Thursday (30 May) using almost identical templates.

EXPECTED RESULT: that the vlan tags and subnet ips found when examining nodes match the yaml configs.
Steps to reproduce: Follow tripleos install guide.

Environment: in this case for a 5 node cluster with the undercloud director running in a Centos7 vm.
Logs and configs.
openstack overcloud failures
http://paste.openstack.org/show/752408/

network-environment-overrides.yaml
http://paste.openstack.org/show/752406/

Controller network info (ip r, ip addr, ovs-vsctl show) (other nodes attached but nothing new there)
http://paste.openstack.org/show/752407/

yum list installed (why GitPython.noarch and ansible and more, have suffix -testing?)
http://paste.openstack.org/show/752409/

[Adam Ratcliff (adamjr)]

New info:
One potential change that could have triggered this problem:
In network-environment-overrides.yaml this time I have been using the default:
NeutronNetworkVLANRanges: 'datacentre:1:1000'
I am testing reverted to customised values now.
NeutronNetworkVLANRanges: 'datacentre:135:139'

If this is the trigger for screwing up a deployment a check would be nice (or at least a note in the docs, I will contribute something after testing).

[Adam Ratcliff (adamjr)]

Info:
a) changing NNVlanRanges did not make any diff.
b) SELinux Enforcing.

[Rabi Mishra (rabi)]

Ordering of the environments in the deploy command are important. I guess things in your network-environment.yaml overriding the parameters in network-environment-overrides.yaml.

You can change the order as below and try.

-e /usr/share/openstack-tripleo-heat-templates/environments/network-environment.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml -e /home/stack/templates/network-environment-overrides.yaml

[Adam Ratcliff (adamjr)]

Rabi
Thank you and Thank you
As soon as I saw that I went looking and Redhat mentions it here
https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/14/html/director_installation_and_usage/creating-a-basic-overcloud-with-cli-tools#deployment-command

However Tripleo docs does not: (there is a note for image build but not deploy)
http://tripleo.org/install/basic_deployment/basic_deployment_cli.html
http://tripleo.org/install/advanced_deployment/network_isolation.html

Same for Openstack docs:
https://docs.openstack.org/tripleo-docs/latest/install/basic_deployment/basic_deployment_cli.html
https://docs.openstack.org/tripleo-docs/latest/install/advanced_deployment/network_isolation.html

If it works I will propose updates.

Again Thank you
Adam.

[Adam Ratcliff (adamjr)]

Found this note in
https://docs.openstack.org/tripleo-docs/latest/install/advanced_deployment/template_deploy.html#overriding-specific-templates-with-local-versions

Note

The --environment-file/-e option may be specified multiple times, if duplicate keys are specified in the environment files, the last one takes precedence.

[Adam Ratcliff (adamjr)]

Similarly
http://tripleo.org/install/advanced_deployment/template_deploy.html#overriding-specific-templates-with-local-versions

Note

The --environment-file/-e option may be specified multiple times, if duplicate keys are specified in the environment files, the last one takes precedence.