Triple-O CI overrides jinja2 rendered network-environment.yaml

[Harald Jensås]

The following files are rendered based on the data in network_data.yaml.

network-environment:
http://git.openstack.org/cgit/openstack/tripleo-heat-templates/tree/environments/network-environment.j2.yaml
http://git.openstack.org/cgit/openstack/tripleo-heat-templates/tree/environments/network-environment-v6.j2.yaml

network-isolation:
http://git.openstack.org/cgit/openstack/tripleo-heat-templates/tree/environments/network-isolation.j2.yaml
http://git.openstack.org/cgit/openstack/tripleo-heat-templates/tree/environments/network-isolation-v6.j2.yaml

Yet, in CI we override the entire network-environment.yaml and network-isolation.yaml environment with static versions:

http://git.openstack.org/cgit/openstack/tripleo-heat-templates/tree/ci/environments/network/multiple-nics-ipv6/network-environment.yaml
http://git.openstack.org/cgit/openstack/tripleo-heat-templates/tree/ci/environments/network/multiple-nics/network-environment.yaml
http://git.openstack.org/cgit/openstack/tripleo-heat-templates/tree/ci/environments/network/public-bond/network-environment.yaml

http://git.openstack.org/cgit/openstack/tripleo-heat-templates/tree/ci/environments/network/multiple-nics/network-isolation-absolute.yaml
http://git.openstack.org/cgit/openstack/tripleo-heat-templates/tree/ci/environments/network/multiple-nics-ipv6/network-isolation-absolute.yaml

The featureset in tripleo-quickstart should be updated to use a network_data.yaml file and reference the rendered network-environment.yaml and rendered network-isolation.yaml. Ref:
http://git.openstack.org/cgit/openstack/tripleo-quickstart/tree/config/general_config/featureset001.yml#n65

The CI should use network_data.yaml to get rendered versions of these tripleo-heat-templates environment files. An environment file such as network-environment-overrides.yaml should be included after the rendered version to override parameters that we don't render from network_data.yaml. e.g DnsServers, ControlPlaneDefaultRoute etc.

[Matt Young (halcyondude)]

(triage) - this is a nuanced issue, CI team is investigating

[Ronelle Landy (rlandy)]

From logs investigation, I think this a legitimate bug/observation.

We do generate an environment-specific network-environment file from

https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/overcloud-prep-config/templates/network-environment.yaml.j2

and we place it in the /home/{{ undercloud_user }} dir - but in upstream CI, we never use it in the deploy:

## * Deploy the overcloud!
## ::
openstack overcloud deploy \
    --templates /usr/share/openstack-tripleo-heat-templates \
    --libvirt-type qemu --timeout 219 --ntp-server pool.ntp.org -e /home/jenkins/cloud-names.yaml -e /home/jenkins/neutronl3ha.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/docker-ha.yaml -e /home/jenkins/containers-default-parameters.yaml -e /usr/share/openstack-tripleo-heat-templates/ci/environments/network/multiple-nics/network-isolation-absolute.yaml -e /usr/share/openstack-tripleo-heat-templates/ci/environments/network/multiple-nics/network-environment.yaml -e /home/jenkins/overcloud_network_params.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/low-memory-usage.yaml -e /opt/stack/new/tripleo-ci/test-environments/worker-config.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/debug.yaml -e /home/jenkins/enable-tls.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/tls-endpoints-public-ip.yaml -e /home/jenkins/inject-trust-anchor.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/disable-telemetry.yaml --validation-warnings-fatal --compute-scale 1 --control-scale 3 --ntp-server pool.ntp.org -e /usr/share/openstack-tripleo-heat-templates/ci/environments/ovb-ha.yaml \
    ${DEPLOY_ENV_YAML:+-e $DEPLOY_ENV_YAML} "$@" && status_code=0 || status_code=$?

If we used our own parameter-based customized version, that would be fine, but we create the file and ignore it. As such, the parameterized file pointed to in this bug is probably a better option.

[Harald Jensås (harald-jensas)]

@Ronelle,

I don't think quickstart should generate at network-environment file here:
https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/overcloud-prep-config/templates/network-environment.yaml.j2

It should generate a network_data.yaml[1] file instead ...

The deploy command should have:
 -n <path>/network_data.yaml \
 -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \
 -e /usr/share/openstack-tripleo-heat-templates/environments/network-environment.yaml \
 -e /usr/share/openstack-tripleo-heat-templates/ci/environments/network/multiple-nics/network-environment-override.yaml \

Unless we do this, the jinja rendering in the first two environment files are never tested by CI, (and/or quickstart?)

 btw. our docs need updating as well, cause they still use the "legacy" manually typed network-environment.yaml as well.

[1] https://git.openstack.org/cgit/openstack/tripleo-heat-templates/tree/network_data.yaml

[Matt Young (halcyondude)]

(triage) - discussed and determining next steps

[wes hayutin (weshayutin)]

Related Review https://review.openstack.org/#/c/576053/

[Harald Jensås (harald-jensas)]

Re-opening this, the fix only takes care of one small part of the problem.
https://review.openstack.org/#/c/576053/

Also see: https://review.openstack.org/576053 - Jose ran into an issue, that I belive would not have occured if this was fixed.

[Marios Andreou (marios-b)]

This is an automated action. Bug status has been set to 'Incomplete' and target milestone has been removed due to inactivity. If you disagree please re-set these values and reach out to us on freenode #tripleo