NetApp Failing Pod''s node affinity and Missing routing

[Gabriel de Araújo Cabral]

Brief Description
-----------------
1) Routing scripts define at netapp-backend-routes.yaml did not execute on all nodes.
2) Trident is failing on pod trident-csi-5f75d6cb6f-t8t62

NAMESPACE NAME READY STATUS RESTARTS AGE
trident trident-csi-5f75d6cb6f-t8t62 0/6 Pending 0 14m
trident trident-csi-624d7 1/2 Running 0 14m
trident trident-csi-77hhm 1/2 Running 0 14m
trident trident-csi-b8wqj 1/2 Running 0 14m

ansible error:
Warning FailedScheduling 5m10s default-scheduler 0/3 nodes are available: 1 node(s) didn't match Pod's node affinity/selector, 2 node(s) had untolerated taint {node-role.kubernetes.io/master: }. preemption: 0/3 nodes are available: 3 Preemption is not helpful for scheduling.

Severity
--------
Critical: Feature is not usable after the defect

Steps to Reproduce
------------------
kubectl get pods -A Pending status
kubectl describe pod trident-csi-5f75d6cb6f-t8t62 -n trident

Expected Behavior
------------------
All the pod is up and running

Actual Behavior
-----------------
As per describe

Reproducibility
---------------
Reproducible

System Configuration
--------------------
multinode IPV4

Branch/Pull Time/Commit
-----------------------
2022-12-30_18-00-09

Last Pass
---------
N/A

Timestamp/Logs
--------------
N/A

Workaround
----------
N/A

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/starlingx/ansible-playbooks/+/871522

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/starlingx/ansible-playbooks/+/871522
Committed: https://opendev.org/starlingx/ansible-playbooks/commit/37a83950e8aaa9b637a4a653d188860e365e8d3e
Submitter: "Zuul (22348)"
Branch: master

commit 37a83950e8aaa9b637a4a653d188860e365e8d3e
Author: Gabriel de Araújo Cabral <email address hidden>
Date: Mon Jan 23 12:07:16 2023 -0500

    Update trident deployment tolerations config

    The empty tolerations config in "trident-deployment.yaml.j2" file
    prevents correct trident installation on all nodes. This commit
    adds master and control-plane tolerations.

    Test Plan:
    -PASS: Verify successful Trident installation in standard system.

    Closes-Bug: 2003734

    Signed-off-by: Gabriel de Araújo Cabral <email address hidden>
    Change-Id: Iaca68a93ed8dc0f4909cfa3fd4072de30549f0e0