'pci-irq-affinity-agent' pod in CrashLoopBackOff (ImportError)

[Heitor Matsui]

Brief Description
-----------------
Tests failed to install stx-openstack on all labs due to pci-irq-affinity-agent pod.

Kubernetes describe suggests an import error:

Traceback (most recent call last): File "/tmp/health-probe.py", line 29, in <module> from pci_irq_affinity.nova_provider import novaClient ImportError: cannot import name novaClient

Severity
-----------------
Critical

Steps to Reproduce
-----------------
Install latest stx-openstack

Expected Behavior
-----------------
stx-openstack installed

Actual Behavior
-----------------
stx-openstack fails to install due to pci-irq-affinity-agent pod in CrashLoopBackOff

Reproducibility
-----------------
100%

System Configuration
-----------------
DX and Standard

Branch/Pull Time/Commit
-----------------
master

Last Pass
-----------------
2022-01-24

Timestamp/Logs
-----------------
[sysadmin@controller-0 ~(keystone_admin)]$ kubectl get pod -n openstack | grep -v Completed | grep -v Running NAME READY STATUS RESTARTS AGE pci-irq-affinity-agent-bs8m9 0/1 CrashLoopBackOff 31 3h17m pci-irq-affinity-agent-gszq9 0/1 CrashLoopBackOff 31 3h17m pci-irq-affinity-agent-hqf6p 0/1 CrashLoopBackOff 31 3h17m
  Normal Killing 3h7m kubelet Container pci-irq-affinity-agent failed liveness probe, will be restarted
  Normal Created 3h7m (x2 over 3h11m) kubelet Created container pci-irq-affinity-agent
  Normal Started 3h7m (x2 over 3h11m) kubelet Started container pci-irq-affinity-agent
  Normal Pulled 3h7m kubelet Container image "registry.local:9001/docker.io/starlingx/stx-pci-irq-affinity-agent:master-centos-stable-20220128T165659Z.0" already present on machine
  Warning Unhealthy 122m (x14 over 3h8m) kubelet Readiness probe failed: Traceback (most recent call last):
  File "/tmp/health-probe.py", line 29, in <module>
    from pci_irq_affinity.nova_provider import novaClient
ImportError: cannot import name novaClient
  Warning Unhealthy 7m51s (x93 over 3h9m) kubelet Liveness probe failed: Traceback (most recent call last):
  File "/tmp/health-probe.py", line 29, in <module>
    from pci_irq_affinity.nova_provider import novaClient
ImportError: cannot import name novaClient
  Warning BackOff 2m42s (x307 over 159m) kubelet Back-off restarting failed container

Test Activity
-----------------
Regression

Workaround
-----------------
NA

[Heitor Matsui (heitormatsui)]

Fix: https://review.opendev.org/c/starlingx/openstack-armada-app/+/826957

[Heitor Matsui (heitormatsui)]

https://review.opendev.org/c/starlingx/openstack-armada-app/+/826957