scenario001-standalone jobs failing consistently with error "Service multipathd has not started yet" with below traceback :
2021-10-18 05:57:41.377462 | fa163e2d-137b-0c96-4cde-000000003007 | TASK | Check if /etc/sysconfig/podman_drop_in exists
2021-10-18 05:57:41.599118 | fa163e2d-137b-0c96-4cde-000000003007 | OK | Check if /etc/sysconfig/podman_drop_in exists | standalone
2021-10-18 05:57:41.600356 | fa163e2d-137b-0c96-4cde-000000003007 | TIMING | tripleo_container_manage : Check if /etc/sysconfig/podman_drop_in exists | standalone | 0:25:43.678348 | 0.22s
2021-10-18 05:57:41.620841 | fa163e2d-137b-0c96-4cde-000000003008 | TASK | Set podman_drop_in fact
2021-10-18 05:57:41.648901 | fa163e2d-137b-0c96-4cde-000000003008 | OK | Set podman_drop_in fact | standalone
2021-10-18 05:57:41.650181 | fa163e2d-137b-0c96-4cde-000000003008 | TIMING | tripleo_container_manage : Set podman_drop_in fact | standalone | 0:25:43.728151 | 0.03s
2021-10-18 05:57:41.672870 | fa163e2d-137b-0c96-4cde-000000003009 | TASK | Manage container systemd services and cleanup old systemd healthchecks for /var/lib/tripleo-config/container-startup-config/step_3
2021-10-18 05:58:14.714582 | fa163e2d-137b-0c96-4cde-000000003009 | FATAL | Manage container systemd services and cleanup old systemd healthchecks for /var/lib/tripleo-config/container-startup-config/step_3 | standalone | error={"changed": false, "msg": "Service multipathd has not started yet"}
2021-10-18 05:58:14.716851 | fa163e2d-137b-0c96-4cde-000000003009 | TIMING | tripleo_container_manage : Manage container systemd services and cleanup old systemd healthchecks for /var/lib/tripleo-config/container-startup-config/step_3 | standalone | 0:26:16.794816 | 33.04s
This issue is facing at integration and component lines as well.
Detailed Logs :
https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-scenario001-standalone-master/a6dd1fb/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz
https://logserver.rdoproject.org/openstack-component-glance/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-scenario001-standalone-glance-master/f2b496a/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz
https://logserver.rdoproject.org/openstack-component-common/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-scenario001-standalone-common-master/b0f3aa2/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz
I think something else is causing this. [1] does indicate a number of failures, but not runs fail and not all the failures are are due to "Service multipathd has not started yet"
Furthermore, [2] shows the multipathd container is running and [3] doesn't show any errors.
[1] https:/ /zuul.openstack .org/builds? job_name= tripleo- ci-centos- 8-scenario001- standalone /logserver. rdoproject. org/openstack- periodic- integration- main/opendev. org/openstack/ tripleo- ci/master/ periodic- tripleo- ci-centos- 8-scenario001- standalone- master/ a6dd1fb/ logs/undercloud /var/log/ extra/podman/ podman_ allinfo. log.txt. gz /logserver. rdoproject. org/openstack- periodic- integration- main/opendev. org/openstack/ tripleo- ci/master/ periodic- tripleo- ci-centos- 8-scenario001- standalone- master/ a6dd1fb/ logs/undercloud /var/log/ extra/podman/ containers/ multipathd/ stdout. log.txt. gz
[2] https:/
[3] https:/