periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-wallaby is failing for a while now[1], but the latest results are showing always the same issue, in node provision step[2][3]:
90f-8a83-bf9bab91e419) to node baremetal-39960-29-50167-1 (UUID 78616239-f41d-474e-9877-4a0f3adaaaac)\nProvisioning started on node baremetal-39960-29-50167-0 (UUID 917f5589-d7c8-408e-8e39-6d90d4d6d6db)\nProvisioning started on node baremetal-39960-29-50167-1 (UUID 78616239-f41d-474e-9877-4a0f3adaaaac)\nProvisioning started on node baremetal-39960-29-50167-3 (UUID 96671d3b-4467-45a2-bf04-99f128d928a0)\n", "msg": "Node 78616239-f41d-474e-9877-4a0f3adaaaac reached failure state \"deploy failed\"; the last error is Operation was aborted due to conductor take over"}
2022-04-26 12:09:19.270488 | fa163e51-ef20-2adc-b71f-00000000001a | FATAL | Provision instances | localhost | error={"changed": false, "logging": "Created port overcloud-controller-0-ctlplane (UUID f28a6197-2ce2-4edb-87d3-b28c9e81e802) for node baremetal-39960-29-50167-2 (UUID 0356b750-b99a-4597-a296-c9ab37e5b72e) with {'network_id': '48407598-91cf-4eec-9ef8-0d59dea95df3', 'name': 'overcloud-controller-0-ctlplane'}\nCreated port overcloud-controller-2-ctlplane (UUID 28294131-ccaa-46ed-9f5b-067a526ae9de) for node baremetal-39960-29-50167-3 (UUID 96671d3b-4467-45a2-bf04-99f128d928a0) with {'network_id': '48407598-91cf-4eec-9ef8-0d59dea95df3', 'name': 'overcloud-controller-2-ctlplane'}\nCreated port overcloud-controller-1-ctlplane (UUID 65be5d00-6f12-40db-8b63-ca9404ccfd03) for node baremetal-39960-29-50167-0 (UUID 917f5589-d7c8-408e-8e39-6d90d4d6d6db) with {'network_id': '48407598-91cf-4eec-9ef8-0d59dea95df3', 'name': 'overcloud-controller-1-ctlplane'}\nCreated port overcloud-novacompute-0-ctlplane (UUID c071437e-5a58-490f-8a83-bf9bab91e419) for node baremetal-39960-29-50167-1 (UUID 78616239-f41d-474e-9877-4a0f3adaaaac) with {'network_id': '48407598-91cf-4eec-9ef8-0d59dea95df3', 'name': 'overcloud-novacompute-0-ctlplane'}\nAttached port overcloud-controller-0-ctlplane (UUID f28a6197-2ce2-4edb-87d3-b28c9e81e802) to node baremetal-39960-29-50167-2 (UUID 0356b750-b99a-4597-a296-c9ab37e5b72e)\nAttached port overcloud-controller-1-ctlplane (UUID 65be5d00-6f12-40db-8b63-ca9404ccfd03) to node baremetal-39960-29-50167-0 (UUID 917f5589-d7c8-408e-8e39-6d90d4d6d6db)\nAttached port overcloud-controller-2-ctlplane (UUID 28294131-ccaa-46ed-9f5b-067a526ae9de) to node baremetal-39960-29-50167-3 (UUID 96671d3b-4467-45a2-bf04-99f128d928a0)\nProvisioning started on node baremetal-39960-29-50167-2 (UUID 0356b750-b99a-4597-a296-c9ab37e5b72e)\nAttached port overcloud-novacompute-0-ctlplane (UUID c071437e-5a58-490f-8a83-bf9bab91e419) to node baremetal-39960-29-50167-1 (UUID 78616239-f41d-474e-9877-4a0f3adaaaac)\nProvisioning started on node baremetal-39960-29-50167-0 (UUID 917f5589-d7c8-408e-8e39-6d90d4d6d6db)\nProvisioning started on node baremetal-39960-29-50167-1 (UUID 78616239-f41d-474e-9877-4a0f3adaaaac)\nProvisioning started on node baremetal-39960-29-50167-3 (UUID 96671d3b-4467-45a2-bf04-99f128d928a0)\n", "msg": "Node 78616239-f41d-474e-9877-4a0f3adaaaac reached failure state \"deploy failed\"; the last error is Operation was aborted due to conductor take over"}
The full error from ironic-conductor is the following[4]:
ERROR ironic.conductor.task_manager [req-d28f7173-3be8-43bc-a610-1064171b7abe - - - - -] Node 78616239-f41d-474e-9877-4a0f3adaaaac moved to provision state "deploy failed" from state "deploying"; target provision state is "active"
WARNING ironic.conductor.utils [req-d28f7173-3be8-43bc-a610-1064171b7abe - - - - -] Aborted the current operation on node 78616239-f41d-474e-9877-4a0f3adaaaac due to conductor take over
[1] https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-wallaby
[2] https://logserver.rdoproject.org/60/39960/29/check/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-wallaby/5a93650/logs/undercloud/home/zuul/overcloud_node_provision.log.txt.gz
[3]https://logserver.rdoproject.org/openstack-periodic-integration-stable1-cs8/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-wallaby/10dbf10/logs/undercloud/home/zuul/overcloud_node_provision.log.txt.gz
[4] https://logserver.rdoproject.org/60/39960/29/check/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-wallaby/5a93650/logs/undercloud/var/log/containers/ironic/ironic-conductor.log.txt.gz
https:/ /logserver. rdoproject. org/15/ 15761b77d91ab3e 398f6fa1d10d2f5 da267b7931/ openstack- periodic- integration- stable1- cs8/periodic- tripleo- ci-centos- 8-ovb-3ctlr_ 1comp-featurese t001-wallaby/ 2f5dc02/ logs/undercloud /home/zuul/ overcloud_ node_provision. log.txt. gz
This is still happening on wallaby c8:
be8241777191) \nProvisioning started on node baremetal-26955-0 (UUID 9187de8c- 59e1-46d1- 8fac-d6cb28fca0 b4)\nProvisioni ng started on node baremetal-26955-3 (UUID ae5c76e4- f754-43f1- a33e-0cd0bb4382 e7)\n", "msg": "Node ae5c76e4- f754-43f1- a33e-0cd0bb4382 e7 reached failure state \"deploy failed\"; the last error is Operation was aborted due to conductor take over"}