At [1][2] the periodic-tripleo-ci-centos-9-scenario001-standalone-wallaby fails during the standalone deploy with trace like:
* 2022-03-07 13:36:09.261895 | fa163e01-389b-8a12-9130-0000000026ca | FATAL | search triple_run_cephadm_output of cephadm run(s) non-zero return codes | undercloud | error={"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result"}
2022-03-07 13:36:09.263944 | fa163e01-389b-8a12-9130-0000000026ca | TIMING | tripleo_run_cephadm : search triple_run_cephadm_output of cephadm run(s) non-zero return codes | undercloud | 0:36:04.360107 | 0.03s
Looks like the issue is timeout waiting for OSD [3]
2022-03-08 14:10:25,196 p=104898 u=root n=ansible | 2022-03-08 14:10:25.195284 | fa163e84-153e-1b31-161f-0000000002ab | FATAL | Wait for expected number of osds to be running | standalone | error={"attempts": 40, "changed": true, "cmd": "podman run --rm --net=host --ipc=host --volume /etc/ceph:/etc/ceph:z --volume /home/ceph-admin/assimilate_ceph.conf:/home/assimilate_ceph.conf:z --volume /home/ceph-admin/specs/ceph_spec.yaml:/home/ceph_spec.yaml:z --entrypoint ceph 192.168.24.1:8787/ceph/daemon:v6.0.6-stable-6.0-pacific-centos-8-x86_64 --fsid 4b5c8c0a-ff60-454b-a1b4-9747aa737d19 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring status --format json | jq .osdmap.num_up_osds", "delta": "0:00:00.880216", "end": "2022-03-08 14:10:25.163897", "msg": "", "rc": 0, "start": "2022-03-08 14:10:24.283681", "stderr": "", "stderr_lines": [], "stdout": "0", "stdout_lines": ["0"]}
This is Centos 9 wallaby promotion blocker
[1] https://logserver.rdoproject.org/3d/3de35c67026f664c2dc7fca0bce8500664a5a0b0/openstack-periodic-integration-stable1-cs9/periodic-tripleo-ci-centos-9-scenario001-standalone-wallaby/94d57f3/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz
[2] https://logserver.rdoproject.org/openstack-periodic-integration-stable1-cs9/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-scenario001-standalone-wallaby/5504f1d/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz
[3] https://logserver.rdoproject.org/openstack-periodic-integration-stable1-cs9/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-scenario001-standalone-wallaby/5504f1d/logs/undercloud/home/zuul/tripleo-deploy/standalone-ansible-ttaesph_/cephadm/cephadm_command.log.txt.gz
Though tripleo- ci-centos- 9-scenario001- standalone [0] failed this morning the failure was in tempest and all OSDs came up [1] before tempest ran. Is there a relevant difference between these two jobs?
- periodic- tripleo- ci-centos- 9-scenario001- standalone- wallaby ci-centos- 9-scenario001- standalone
- tripleo-
[0] https:/ /zuul.opendev. org/t/openstack /build/ 5de6e6e2e2b441f d91382f2b7b2d6e d0/logs
[1] https:/ /d2b5bc52e78cdf 005618- 65472637883373d b40924f3bc98462 64.ssl. cf2.rackcdn. com/831860/ 1/check/ tripleo- ci-centos- 9-scenario001- standalone/ 5de6e6e/ logs/undercloud /home/zuul/ tripleo- deploy/ standalone- ansible- dazmxuhp/ cephadm/ cephadm_ command. log