No such file or directory @ rb_sysopen - /etc/systemd/system/virtproxyd-tls.socket in container-puppet-nova_libvirt

[Ananya Banerjee]

periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-master and periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-master are failing overcloud deploy

FATAL | Manage Puppet containers (generate config) for step 1 | overcloud-novacompute-0 | error={"changed": false, "msg": "Failed containers: container-puppet-nova_libvirt"}

https://logserver.rdoproject.org/74/47974/5/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-master/8fbba80/logs/overcloud-novacompute-0/var/log/containers/stdouts/container-puppet-nova_libvirt.log.txt.gz

https://logserver.rdoproject.org/74/47974/6/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-master/704feb8/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz

We are seeing this from 2023-03-29 03:57:24

[Ananya Banerjee (frenzyfriday)]

In the logs I don not see it moving to task https://opendev.org/openstack/tripleo-heat-templates/src/branch/master/deployment/podman/podman-baremetal-ansible.yaml#L158
probably that is where the jobs is failing

[Ananya Banerjee (frenzyfriday)]

https://logserver.rdoproject.org/74/47974/5/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-master/8fbba80/logs/overcloud-controller-1/var/log/extra/podman/podman_allinfo.log.txt.gz - dont see the nova neutron etc openstacky containers in podman ps

[Ananya Banerjee (frenzyfriday)]

2023-03-30T09:17:49.509723014+00:00 stderr F <13>Mar 30 09:17:49 puppet-user: Error: /Stage[main]/Nova::Migration::Libvirt/File_line[virtproxyd-tls.socket ListenStream]: Could not evaluate: No such file or directory @ rb_sysopen - /etc/systemd/system/virtproxyd-tls.socket

https://logserver.rdoproject.org/74/47974/6/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-master/704feb8/logs/overcloud-novacompute-0/var/log/containers/stdouts/container-puppet-nova_libvirt.log.txt.gz

[Takashi Kajinami (kajinamit)]

This is the issue caused by recent change in puppet-nova which fixed incomplete settings.
 https://review.opendev.org/c/openstack/puppet-nova/+/878541

We can probably fix the problem, but IMO we should stop monitoring master jobs because TripleO master is being abandoned.

[Marios Andreou (marios-b)]

same issue also hitting the multinode-ipa job [1][2]

per Takashi's comment and our current plans of teardown for the master rdo lines after next week (ish) ... this is unlikely to be worked on

we might remove these 3 jobs from criteria for now and until teardown

[1] https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-standalone-on-multinode-ipa-master/d0b87bc/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz
[2] https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-standalone-on-multinode-ipa-master/d69002e/logs/undercloud/var/log/containers/stdouts/container-puppet-nova_libvirt.log.txt.gz

[Takashi Kajinami (kajinamit)]

This is probably fixed by https://review.opendev.org/c/openstack/puppet-nova/+/879145, though it's no longer needed as we agree abandoning all master jobs.

[Marios Andreou (marios-b)]

Yeah @Takashi as posted in [1] the ci team has started the process of removing the master periodic rdo lines [2]

I am going to mark the bug as "won't fix" for the cix tracking.

[1] https://lists.openstack.org/pipermail/openstack-discuss/2023-March/033080.html
[2] https://review.rdoproject.org/r/c/config/+/48074