Tripleo wallaby jobs are failing with with "ERROR:__main__:MissingRequiredSource: /var/lib/kolla/config_files/src/* file is not found".

Bug #2030841 reported by Sandeep Yadav
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
In Progress
Critical
Unassigned

Bug Description

Description:

Tripleo wallaby jobs are failing with with "ERROR:__main__:MissingRequiredSource: /var/lib/kolla/config_files/src/* file is not found".

Error snippet:

https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_f4d/890596/3/gate/tripleo-ci-centos-9-standalone/f4d02dd/logs/undercloud/home/zuul/standalone_deploy.log

~~~
2023-08-08 20:43:20.929916 | | WARNING | ERROR: Can't run container mysql_bootstrap
stderr: INFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json
INFO:__main__:Validating config file
INFO:__main__:Kolla config strategy set to: COPY_ALWAYS
INFO:__main__:Copying service configuration files
INFO:__main__:Copying /dev/null to /etc/libqb/force-filesystem-sockets
INFO:__main__:Setting permission for /etc/libqb/force-filesystem-sockets
ERROR:__main__:MissingRequiredSource: /var/lib/kolla/config_files/src/* file is not found
~~~

Failure started since 08th Aug as per build history:

https://zuul.opendev.org/t/openstack/builds?job_name=tripleo-ci-centos-9-standalone

Issue started after got latest version of podman in Centos9
~~~
last good version: podman-4.6.0-1.el9.x86_64
affected version: podman-4.6.0-3.el9.x86_64
~~~

we pass the puppet manifest to be executed in containers using the STEP_CONFIG environment. Now podman does not give the full value that results in puppet executing incomplete manifests and that's why we've seen no config files extracted under /var/lib/config-date/puppet-generated

Thanks to Takashi for the debugging and reporting bug against Podman: https://bugzilla.redhat.com/show_bug.cgi?id=2230212

Changed in tripleo:
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to tripleo-quickstart (master)

Reviewed: https://review.opendev.org/c/openstack/tripleo-quickstart/+/890838
Committed: https://opendev.org/openstack/tripleo-quickstart/commit/2aa133c2aecb379ae6234a8879edb9f46bbeb166
Submitter: "Zuul (22348)"
Branch: master

commit 2aa133c2aecb379ae6234a8879edb9f46bbeb166
Author: Ronelle Landy <email address hidden>
Date: Tue Aug 8 20:20:28 2023 -0400

    Exclude affected podman version

    Latest version of podman broke tripleo C9 jobs,
    details are in Closes-Bug

    ~~~
    last good version: podman-4.6.0-1.el9.x86_64
    affected version: podman-4.6.0-3.el9.x86_64
    ~~~

    Excluding affected podman version to get around this issue.

    Related-Bug: #2030841
    Change-Id: Ia8bc8c9ca8c65181e049f1d579757883a40a31b0

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to tripleo-quickstart (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/tripleo-quickstart/+/891013

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to tripleo-quickstart (master)

Reviewed: https://review.opendev.org/c/openstack/tripleo-quickstart/+/891013
Committed: https://opendev.org/openstack/tripleo-quickstart/commit/5467cd5a3312b0a58b4b3aadf3774ee6ea8a1fb7
Submitter: "Zuul (22348)"
Branch: master

commit 5467cd5a3312b0a58b4b3aadf3774ee6ea8a1fb7
Author: Sandeep Yadav <email address hidden>
Date: Thu Aug 10 12:23:44 2023 +0530

    Downgrade podman

    We excluded latest version of podman in [1] after which
    tripleo jobs are green.

    But adoption job install podman in pre, adding explicit
    downgrade to fix those jobs.

    [1] https://review.opendev.org/c/openstack/tripleo-quickstart/+/890838

    Related-Bug: #2030841
    Change-Id: I5c4c3e0ce3b18a0ac93271103fc0875cb38b05ac

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.