Overcloud can't get deployed: Ironic fails to update MAC address on Neutron port

[Emilien Macchi]

How to reproduce:

- Deploy an undercloud on master
- Deploy an overcloud with OVB on master

Results expected:

Overcloud nodes are started and deployment can finish.

Actual results:

Overcloud nodes fail to deploy, Ironic fails to update MAC address on Neutron port:

https://logs.rdoproject.org/23/614023/5/openstack-experimental/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master-vexxhost/c8acacf/logs/undercloud/var/log/containers/ironic/ironic-conductor.log.txt.gz#_2018-11-08_23_12_51_661

https://logs.rdoproject.org/23/614023/5/openstack-experimental/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master-vexxhost/c8acacf/logs/undercloud/var/log/containers/nova/nova-compute.log.txt.gz#_2018-11-08_23_12_51_683

https://logs.rdoproject.org/23/614023/5/openstack-experimental/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master-vexxhost/c8acacf/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz#_2018-11-08_23_29_45

Notes:
- I'm not sure it's related but the undercloud is deployed on Podman.
- the bug looks similar to https://bugs.launchpad.net/tripleo/+bug/1749671

[Harald Jensås (harald-jensas)]

Probably -> https://review.openstack.org/539981 is related here.

BUT, this only happens if a server build failed, and the scheduler later pick the same ironic node for a different nova server. The log's should be checked to find the first server build error.

[Harald Jensås (harald-jensas)]

Following up on my previous comment. The initial errors are timeouts deploying nodes:

  ERROR ironic.conductor.utils [req-<ID> - - - - -] Timeout reached while waiting for callback for node <node-ID>

  https://logs.rdoproject.org/23/614023/5/openstack-experimental/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master-vexxhost/c8acacf/logs/undercloud/var/log/containers/ironic/ironic-conductor.log.txt.gz#_2018-11-08_23_12_13_687
  https://logs.rdoproject.org/23/614023/5/openstack-experimental/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master-vexxhost/c8acacf/logs/undercloud/var/log/containers/ironic/ironic-conductor.log.txt.gz#_2018-11-08_23_12_13_534

[Juan Antonio Osorio Robles (juan-osorio-robles)]

Is this still an issue?

[Alex Schultz (alex-schultz)]

Closing due to inactivity. Feel free to reopen if there is more information