At [1][2][3] the tripleo-ci-centos-8-undercloud-upgrade-wallaby fails during the upgrade but after upgrade tasks are completed. During the post upgrade deploy tasks step 5 there is the following trace:
2021-07-08 13:34:11 | 2021-07-08 13:34:11.341074 | fa163ee1-1b2f-aef2-0d3f-0000000029b9 | FATAL | Run kolla_set_configs to copy ring files | undercloud | item=swift_proxy | error={"ansible_loop_var": "item", "changed": true, "cmd": "podman exec -u root swift_proxy /usr/local/bin/kolla_set_configs ", "delta": "0:00:00.254828", "end": "2021-07-08 13:34:11.306745", "failed_when_result": true, "item": "swift_proxy", "msg": "non-zero return code", "rc": 137, "start": "2021-07-08 13:34:11.051917", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}
This is potentially a gate blocker as the job is being wired up with [4] and has already merged for puppet-tripleo there [5]
[1] https://d393ab92b65d6ff2eea5-fc707f543607a38fac44776c15f601da.ssl.cf5.rackcdn.com/800186/1/check/tripleo-ci-centos-8-undercloud-upgrade-wallaby/f002075/logs/undercloud/home/zuul/undercloud_upgrade.log
[2] https://0229dac170a36c80720a-0e9efd411d5f516ecd1b5a61c03e35b7.ssl.cf2.rackcdn.com/800011/1/check/tripleo-ci-centos-8-undercloud-upgrade-wallaby/3d191f2/logs/undercloud/home/zuul/undercloud_upgrade.log
[3] https://d3ba52c45612db77e201-20f8b612e9341a4bfb8242ee0493df0d.ssl.cf2.rackcdn.com/793135/3/check/tripleo-ci-centos-8-undercloud-upgrade-wallaby/d7f682e/logs/undercloud/home/zuul/undercloud_upgrade.log
[4] https://review.opendev.org/q/topic:wallaby-upgrade-jobs
[5] https://review.opendev.org/c/openstack/puppet-tripleo/+/793124
I suspect this may be a race condition. It has been seen to pass and then fail on the same review, for example [1] pass then [2] fail.
The failing task is from [3] added in [4]
Not clear if this is wallaby only or if it will also affect the master branch.
[1] https:/ /review. opendev. org/c/openstack /puppet- tripleo/ +/800186/ 1#message- 74632e92dc45e43 5ebd78ad090f582 24e80e93d9 * tripleo- ci-centos- 8-undercloud- upgrade- wallaby https:/ /zuul.opendev. org/t/openstack /build/ 7a13f3f5c9724d6 4b1615ebf621b86 87 : SUCCESS in 1h 13m 20s
[2] https:/ /review. opendev. org/c/openstack /puppet- tripleo/ +/800186/ 1#message- 9a203e19265dc06 4fbf5cbf66c94bb f42556d2eb * tripleo- ci-centos- 8-undercloud- upgrade- wallaby https:/ /zuul.opendev. org/t/openstack /build/ f002075c98f84a6 8ba48c1fb5b573b ab : FAILURE in 1h 29m 13s
[3] https:/ /opendev. org/openstack/ tripleo- heat-templates/ src/commit/ a59039d29ca7837 24e911de0021d67 f939e76ad2/ deployment/ swift/swift- storage- container- puppet. yaml#L695- L696
[4] https:/ /review. opendev. org/q/Ibdd783b4 84a84c0fdfaac84 d892a8ea46be85f de