periodic-centos-7-multinode-1ctlr-featureset030-master fails overcloud deploy for Unknown Property ExternalInterfaceDefaultRoute

[Marios Andreou]

periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset030-master fails at [1][2][3] during overcloud deploy with trace like

  2019-03-26 07:24:21 | u'message': u'Error during stack creation: ERROR: Property error: : resources.Controller<nested_stack>.resources.0<http://192.168.24.1:8080/v1/AUTH_b37f17f9c27646e689f33bff529fb264/overcloud/puppet/controller-role.yaml>.resources.NetworkConfig.properties: : Unknown Property ExternalInterfaceDefaultRoute\nTraceback (most recent call last):\n\n File "/usr/lib/python2.7/site-packages/heat/common/context.py", line 410, in wrapped\n return func(self, ctx, *args, **kwargs)\n\n File "/usr/lib/python2.7/site-packages/heat/engine/service.py", line 849, in create_stack\n parent_resource_name, template_id)\n\n File "/usr/lib/python2.7/site-packages/heat/engine/service.py", line 753, in _parse_template_and_validate_stack\n stack.validate()\n\n File "/usr/lib/python2.7/site-packages/osprofiler/profiler.py", line 160, in wrapper\n result = f(*args, **kwargs)\n\n File "/usr/lib/python2.7/site-packages/heat/engine/stack.py", line 898, in validate\n result = res.validate()\n\n File "/usr/lib/python2.7/site-packages/heat/engine/resources/stack_resource.py", line 66, in validate\n self.validate_nested_stack()\n\n File "/usr/lib/python2.7/site-packages/heat/engine/resources/openstack/heat/resource_group.py", line 324, in validate_nested_stack\n ex, path=[self.stack.t.RESOURCES, path])\n\nStackValidationFailed: Property error: resources.Controller<nested_stack>.resources.0<http://192.168.24.1:8080/v1/AUTH_b37f17f9c27646e689f33bff529fb264/overcloud/puppet/controller-role.yaml>.resources.NetworkConfig.properties: Unknown Property ExternalInterfaceDefaultRoute\n',
  2019-03-26 07:24:21 | u'plan_name': u'overcloud',
  2019-03-26 07:24:21 | u'status': u'FAILED'}
  2019-03-26 07:24:21 | Exception occured while running the command
  2019-03-26 07:24:21 | Traceback (most recent call last):
  2019-03-26 07:24:21 | File "/usr/lib/python2.7/site-packages/tripleoclient/command.py", line 29, in run

...

  2019-03-26 07:24:21 | ex, path=[self.stack.t.RESOURCES, path])
  2019-03-26 07:24:21 |
  2019-03-26 07:24:21 | StackValidationFailed: Property error: resources.Controller<nested_stack>.resources.0<http://192.168.24.1:8080/v1/AUTH_b37f17f9c27646e689f33bff529fb264/overcloud/puppet/controller-role.yaml>.resources.NetworkConfig.properties: Unknown Property ExternalInterfaceDefaultRoute
  2019-03-26 07:24:21 |

This is a promotion blocker since thejob is in criteria at [4]

[1] http://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset030-master/6f15af3/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz
[2] http://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset030-master/a0347a0/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz
[3] http://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset030-master/9a9df69/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz
[4] https://github.com/rdo-infra/ci-config/blob/1f53ebf5e5e2a49f2bf66192cb65bc214b2a43e5/ci-scripts/dlrnapi_promoter/config/CentOS-7/master.ini#L14

[Rabi Mishra (rabi)]

It's broken by https://github.com/openstack/tripleo-heat-templates/commit/cf2530487924482b1a5f07b0b58317d5579090b6#diff-f620b68f0bb72070d60fe8e1f3b0238c

We need to add ExternalInterfaceDefaultRoute parameter with default to the nic config templates used in ci.

ex. https://github.com/openstack/tripleo-heat-templates/blob/master/ci/common/net-config-multinode-os-net-config.yaml

I can push a patch if you want, let me know.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/648100

[Harald Jensås (harald-jensas)]

Since the job does'nt use network-isolation. IMO the proper fix is to use a custom roles data file with

  networks: {}
  default_route_networks: ['ControlPlane']

And possibly also -n /dev/null to pass no networks.

However, this becomes related to bug https://bugs.launchpad.net/tripleo/+bug/1776186 which is something that is a larger chunk of work accross THT, tripleo-ci and quickstart repos.

The proposed fix, is the correct approach to immediate unblock promotions.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/648100
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=cced6333b3faebad6e77763c978aee88518972e6
Submitter: Zuul
Branch: master

commit cced6333b3faebad6e77763c978aee88518972e6
Author: Rabi Mishra <email address hidden>
Date: Wed Mar 27 17:08:49 2019 +0530

    Add ExternalInterfaceDefaultRoute parameter to ci nic configs

    Since https://review.openstack.org/645039, we need to add
    ExternalInterfaceDefaultRoute parameter to nic configs.

    It seems we always have 'default_route_networks: ['External']'
    in roles data. So we probably don't need it for other networks.

    Change-Id: I612ee828774d16bff910c6d941aad1e976b6a4a5
    Closes-Bug: #1821871

[Marios Andreou (marios-b)]

closing this out we have a green run now in https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset030-master

thanks

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 10.5.0 release.