periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp_1supp-featureset039-ussuri is failing due container failed to exec
Error logs:
2021-12-30 15:39:51 | 2021-12-30 15:39:51.607399 | fa163e7c-e26f-2ccc-e5e5-00000000aa6c | TIMING | tripleo_container_manage : Check podman exec status | overcloud-controller-0 | 0:29:34.388542 | 5.90s
2021-12-30 15:39:51 | 2021-12-30 15:39:51.624955 | fa163e7c-e26f-2ccc-e5e5-00000000aa6e | TASK | Check container exec status
2021-12-30 15:39:51 | [ERROR]: Container(s) exec commands which failed to execute: []
2021-12-30 15:39:51 | 2021-12-30 15:39:51.703188 | fa163e7c-e26f-2ccc-e5e5-00000000aa6e | FATAL | Check container exec status | overcloud-controller-0 | error={"changed": false, "msg": "Failed container(s): ['keystone_bootstrap'], check logs in /var/log/containers/stdouts/"}
2021-12-30 15:39:51 | 2021-12-30 15:39:51.704390 | fa163e7c-e26f-2ccc-e5e5-00000000aa6e | TIMING | tripleo_container_manage : Check container exec status | overcloud-controller-0 | 0:29:34.485532 | 0.08s
2021-12-30 15:39:51 |
2021-12-30 15:39:51 | PLAY RECAP *********************************************************************
2021-12-30 15:39:51 | overcloud-controller-0 : ok=489 changed=247 unreachable=0 failed=1 skipped=224 rescued=0 ignored=0
2021-12-30 15:39:51 | overcloud-controller-1 : ok=501 changed=248 unreachable=0 failed=0 skipped=236 rescued=0 ignored=0
2021-12-30 15:39:51 | overcloud-controller-2 : ok=499 changed=248 unreachable=0 failed=0 skipped=236 rescued=0 ignored=0
2021-12-30 15:39:51 | overcloud-novacompute-0 : ok=393 changed=176 unreachable=0 failed=0 skipped=224 rescued=0 ignored=0
2021-12-30 15:39:51 | undercloud : ok=766 changed=271 unreachable=0 failed=0 skipped=102 rescued=0 ignored=0
2021-12-30 15:39:51 | 2021-12-30 15:39:51.726526 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Summary Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2021-12-30 15:39:51 | 2021-12-30 15:39:51.727022 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Total Tasks: 2663 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2021-12-30 15:39:51 | 2021-12-30 15:39:51.727448 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Elapsed Time: 0:29:34.508595 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2021-12-30 15:39:51 | 2021-12-30 15:39:51.727838 | UUID | Info | Host | Task Name | Run Time
2021-12-30 15:39:51 | 2021-12-30 15:39:51.728321 | fa163e7c-e26f-2ccc-e5e5-000000003393 | SUMMARY | undercloud | Run tripleo_container_image_prepare logged to: /var/log/tripleo-container-image-prepare.log | 249.82s
2021-12-30 15:39:51 | 2021-12-30 15:39:51.728793 | fa163e7c-e26f-2ccc-e5e5-000000006bd6 | SUMMARY | overcloud-controller-0 | tripleo_ha_wrapper : Run init bundle puppet on the host for haproxy | 140.88s
2021-12-30 15:39:51 | 2021-12-30 15:39:51.729264 | fa163e7c-e26f-2ccc-e5e5-0000000044fd | SUMMARY | overcloud-controller-1 | Pre-fetch all the containers | 92.80s
2021-12-30 15:39:51 | 2021-12-30 15:39:51.729724 | fa163e7c-e26f-2ccc-e5e5-000000005077 | SUMMARY | overcloud-controller-0 | Wait for puppet host configuration to finish | 92.35s
2021-12-30 15:39:51 | 2021-12-30 15:39:51.730286 | fa163e7c-e26f-2ccc-e5e5-0000000045a6 | SUMMARY | overcloud-controller-0 | Pre-fetch all the containers | 91.55s
2021-12-30 15:39:51 | 2021-12-30 15:39:51.730817 | fa163e7c-e26f-2ccc-e5e5-0000000050a7 | SUMMARY | overcloud-controller-2 | Wait for puppet host configuration to finish | 90.22s
2021-12-30 15:39:51 | 2021-12-30 15:39:51.731265 | fa163e7c-e26f-2ccc-e5e5-00000000444e | SUMMARY | overcloud-controller-2 | Pre-fetch all the containers | 88.51s
2021-12-30 15:39:51 | 2021-12-30 15:39:51.731721 | fa163e7c-e26f-2ccc-e5e5-000000004627 | SUMMARY | overcloud-novacompute-0 | Pre-fetch all the containers | 87.15s
2021-12-30 15:39:51 | 2021-12-30 15:39:51.732225 | fa163e7c-e26f-2ccc-e5e5-0000000050de | SUMMARY | overcloud-controller-1 | Wait for puppet host configuration to finish | 82.90s
2021-12-30 15:39:51 | 2021-12-30 15:39:51.732705 | fa163e7c-e26f-2ccc-e5e5-0000000087d0 | SUMMARY | overcloud-controller-0 | tripleo_ha_wrapper : Run init bundle puppet on the host for ovn_dbs | 59.67s
2021-12-30 15:39:51 | 2021-12-30 15:39:51.733194 | fa163e7c-e26f-2ccc-e5e5-000000001814 | SUMMARY | overcloud-controller-2 | tripleo_firewall : Manage firewall rules | 48.92s
2021-12-30 15:39:51 | 2021-12-30 15:39:51.733626 | fa163e7c-e26f-2ccc-e5e5-000000001814 | SUMMARY | overcloud-controller-1 | tripleo_firewall : Manage firewall rules | 48.57s
2021-12-30 15:39:51 | 2021-12-30 15:39:51.734075 | fa163e7c-e26f-2ccc-e5e5-000000001814 | SUMMARY | overcloud-controller-0 | tripleo_firewall : Manage firewall rules | 42.14s
2021-12-30 15:39:51 | 2021-12-30 15:39:51.734586 | fa163e7c-e26f-2ccc-e5e5-000000006be6 | SUMMARY | overcloud-controller-0 | tripleo_ha_wrapper : Run init bundle puppet on the host for mysql | 41.38s
2021-12-30 15:39:51 | 2021-12-30 15:39:51.735058 | fa163e7c-e26f-2ccc-e5e5-000000006c09 | SUMMARY | overcloud-controller-0 | tripleo_ha_wrapper : Run init bundle puppet on the host for redis | 39.01s
2021-12-30 15:39:51 | 2021-12-30 15:39:51.735493 | fa163e7c-e26f-2ccc-e5e5-000000006bf9 | SUMMARY | overcloud-controller-0 | tripleo_ha_wrapper : Run init bundle puppet on the host for oslo_messaging_rpc | 38.47s
2021-12-30 15:39:51 | 2021-12-30 15:39:51.735936 | fa163e7c-e26f-2ccc-e5e5-000000000052 | SUMMARY | overcloud-controller-2 | tripleo_network_config : Run NetworkConfig script | 34.88s
2021-12-30 15:39:51 | 2021-12-30 15:39:51.736414 | fa163e7c-e26f-2ccc-e5e5-000000000052 | SUMMARY | overcloud-controller-0 | tripleo_network_config : Run NetworkConfig script | 34.84s
2021-12-30 15:39:51 | 2021-12-30 15:39:51.736806 | fa163e7c-e26f-2ccc-e5e5-000000000052 | SUMMARY | overcloud-controller-1 | tripleo_network_config : Run NetworkConfig script | 34.24s
2021-12-30 15:39:51 | 2021-12-30 15:39:51.737262 | fa163e7c-e26f-2ccc-e5e5-000000000052 | SUMMARY | overcloud-novacompute-0 | tripleo_network_config : Run NetworkConfig script | 28.46s
2021-12-30 15:39:51 | 2021-12-30 15:39:51.737742 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ End Summary Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2021-12-30 15:39:51 | 2021-12-30 15:39:51.738200 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ State Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2021-12-30 15:39:51 | 2021-12-30 15:39:51.738700 | ~~~~~~~~~~~~~~~~~~ Number of nodes which did not deploy successfully: 1 ~~~~~~~~~~~~~~~~~
2021-12-30 15:39:51 | 2021-12-30 15:39:51.739209 | The following node(s) had failures: overcloud-controller-0
2021-12-30 15:39:51 | 2021-12-30 15:39:51.739668 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2021-12-30 15:39:52 | Ansible execution failed. playbook: /home/zuul/config-download/overcloud/deploy_steps_playbook.yaml, Run Status: failed, Return Code: 2, To rerun the failed command manually execute the following script: /home/zuul/config-download/overcloud/ansible-playbook-command.sh
2021-12-30 15:39:53 | Host overcloud.ooo.test not found in /home/zuul/.ssh/known_hosts
2021-12-30 15:39:53 | Exception occured while running the command
2021-12-30 15:39:53 | Traceback (most recent call last):
2021-12-30 15:39:53 | File "/usr/lib/python3.6/site-packages/tripleoclient/command.py", line 34, in run
2021-12-30 15:39:53 | super(Command, self).run(parsed_args)
2021-12-30 15:39:53 | File "/usr/lib/python3.6/site-packages/osc_lib/command/command.py", line 41, in run
2021-12-30 15:39:53 | return super(Command, self).run(parsed_args)
2021-12-30 15:39:53 | File "/usr/lib/python3.6/site-packages/cliff/command.py", line 187, in run
2021-12-30 15:39:53 | return_code = self.take_action(parsed_args) or 0
2021-12-30 15:39:53 | File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 1173, in take_action
2021-12-30 15:39:53 | raise(deploy_trace)
2021-12-30 15:39:53 | File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 1138, in take_action
2021-12-30 15:39:53 | forks=parsed_args.ansible_forks
2021-12-30 15:39:53 | File "/usr/lib/python3.6/site-packages/tripleoclient/workflows/deployment.py", line 494, in config_download
2021-12-30 15:39:53 | forks=forks
2021-12-30 15:39:53 | File "/usr/lib/python3.6/site-packages/tripleoclient/utils.py", line 728, in run_ansible_playbook
2021-12-30 15:39:53 | raise RuntimeError(err_msg)
2021-12-30 15:39:53 | RuntimeError: Ansible execution failed. playbook: /home/zuul/config-download/overcloud/deploy_steps_playbook.yaml, Run Status: failed, Return Code: 2, To rerun the failed command manually execute the following script: /home/zuul/config-download/overcloud/ansible-playbook-command.sh
2021-12-30 15:39:53 | Ansible execution failed. playbook: /home/zuul/config-download/overcloud/deploy_steps_playbook.yaml, Run Status: failed, Return Code: 2, To rerun the failed command manually execute the following script: /home/zuul/config-download/overcloud/ansible-playbook-command.sh
2021-12-30 15:39:53 | Overcloud Endpoint: https://overcloud.ooo.test:13000
2021-12-30 15:39:53 | Overcloud Horizon Dashboard URL: https://overcloud.ooo.test:443/dashboard
2021-12-30 15:39:53 | Overcloud rc file: /home/zuul/overcloudrc
2021-12-30 15:39:53 | Overcloud Deployed with error
2021-12-30 15:39:53 | + status_code=1
2021-12-30 15:39:53 | + openstack stack list
2021-12-30 15:39:53 | + grep -q overcloud
2021-12-30 15:39:57 | + openstack stack list
2021-12-30 15:39:57 | + grep -Eq '(CREATE|UPDATE)_COMPLETE'
2021-12-30 15:39:59 | + exit 1
#links:
[1]: https://logserver.rdoproject.org/openstack-periodic-integration-stable3/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp_1supp-featureset039-ussuri/f6f9eca/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz
[2]: https://logserver.rdoproject.org/openstack-periodic-integration-stable3/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp_1supp-featureset039-ussuri/f6f9eca/job-output.txt
[3]: https://logserver.rdoproject.org/openstack-periodic-integration-stable3/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp_1supp-featureset039-ussuri/f6f9eca/logs/overcloud-controller-0/var/log/extra/errors.txt.gz
[4]: https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp_1supp-featureset039-ussuri
Same as the victoria failure[1] after backport https:/
[1] https:/