STX-Openstack | Helm overrides are not being applied

[Lucas de Ataides Barreto]

Brief Description
-----------------
After installing and applying STX-Openstack in an AIO-SX server, both ingress pods (ingress-94b94bf55-
ingress-error-pages-78987877f5-5nzhc) are being duplicated and are stuck in pending status

Update: Looks like ANY override is not being applied on AIO labs, for any openstack component, the overrides (except for static overrides) are not being applied, although listed on the helm-override-show

Severity
--------
Major

Steps to Reproduce
------------------
Apply STX-Openstack in an AIO-SX server

Expected Behavior
------------------
Only 1 ingress pod for each component is present

Actual Behavior
----------------
2 ingress pods for each component are present

Reproducibility
---------------
Reproducible

System Configuration
--------------------
Bare metal AIO-SX

Branch/Pull Time/Commit
-----------------------
20230828T060000Z

Timestamp/Logs
--------------
|                    |   replicas: {api_metadata: 1, conductor: 1, consoleauth: 1, novncproxy: 1, osapi: 1,                                                                                                                                                                                                                                                                                                                               |
|                    |     placement: 1, scheduler: 1}                                                                                                                                                                                                                                                                                                                                                                                    |
|                    | secrets:                                                                                                                                                                                                                                                                                                                                                                                                           |
|                    |   tls:                                                                                                                                                                                                                                                                                                                                                                                                             |
|                    |     compute_metadata:                                                                                                                                                                                                                                                                                                                                                                                              |
|                    |       metadata: {public: nova-tls-public}                                                                                                                                                                                                                                                                                                                                                                          |
|                    |                                                                                                                                                                                                                                                                                                                                                                                                                    |
| user_overrides     | None                                                                                                                                                                                                                                                                                                                                                                                                               |
+--------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
[sysadmin@controller-1 ~(keystone_admin)]$ kubectl -n openstack get pods | grep nova
nova-api-metadata-68689bcfd9-ns5ct                     1/1     Running     0             10h
nova-api-metadata-68689bcfd9-xxhwf                     1/1     Running     0             10h
nova-api-osapi-5874fc8f7f-tdrhg                        1/1     Running     0             10h
nova-api-osapi-5874fc8f7f-wp6sb                        1/1     Running     0             10h
nova-api-proxy-b66f45fd7-6dz55                         1/1     Running     0             10h
nova-api-proxy-b66f45fd7-jk5r4                         1/1     Running     0             11h
nova-bootstrap-5nx2l                                   0/1     Completed   0             10h
nova-cell-setup-f2n5g                                  0/1     Completed   0             10h
nova-compute-controller-0-937646f6-4gp6s               2/2     Running     2 (10h ago)   10h
nova-compute-controller-1-cab72f56-vsw6g               2/2     Running     0             10h
nova-conductor-7755cfb659-f8f28                        1/1     Running     0             10h
nova-conductor-7755cfb659-jfn57                        1/1     Running     0             10h
nova-db-init-cntfm                                     0/3     Completed   0             10h
nova-db-sync-6x2k5                                     0/1     Completed   0             10h
nova-ks-service-4nrvn                                  0/1     Completed   0             10h
nova-ks-user-dpln8                                     0/1     Completed   0             10h
nova-novncproxy-5c85c44455-jx2m2                       1/1     Running     0             10h
nova-novncproxy-5c85c44455-jxss4                       1/1     Running     0             10h
nova-rabbit-init-p5s5t                                 0/1     Completed   0             10h
nova-scheduler-7c876fd77c-979jk                        1/1     Running     0             10h
nova-scheduler-7c876fd77c-d9ff6                        1/1     Running     0             10h
nova-service-cleaner-28236240-b5jm4                    0/1     Completed   0             125m
nova-service-cleaner-28236300-qnnpm                    0/1     Completed   0             65m
nova-service-cleaner-28236360-66zn7                    0/1     Completed   0             5m14s

Test Activity
-------------
Developer Testing

Workaround
----------
N/A

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/starlingx/openstack-armada-app/+/895854

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/starlingx/openstack-armada-app/+/895854
Committed: https://opendev.org/starlingx/openstack-armada-app/commit/aaddc97db459764590c486acac652142a74f0a63
Submitter: "Zuul (22348)"
Branch: master

commit aaddc97db459764590c486acac652142a74f0a63
Author: Lucas de Ataides <email address hidden>
Date: Tue Sep 19 16:32:12 2023 -0300

    Add `get_namespace` function to OpenstackBaseHelm

    After the introdution of [1], it was noted that Helm overrides were not
    working properly for STX-Openstack. Althought they show up on
    helm-override-show, they we're not being passed to Helm and Kubernetes.
    This was caused because the [1] change updated the way that the
    overrides are applied, requiring the `get_namespace` function on the
    plugin to define where to apply these overrides, and, if this function
    did not exist, it would default to the `default` namespace.

    This change fixes that by adding the `get_namespace` function to the
    OpenstackBaseHelm plugin, which is extended by the other STX-O plugins.

    This change also does the following:

    1) Updates the base class for ingress, memcached and garbd from
       BaseHelm to OpenstackBaseHelm, as openstack.BaseHelm is the old base
       class for the armada app and probably should be cleaned now that we
       no longer support Armada.
    2) Remove ingress customizations for the kube-system namespace, as
       they're not used anymore and can be removed without any impact.

    [1] https://review.opendev.org/c/starlingx/config/+/887430

    Test Plan:
    PASS: Build python3-k8sapp-openstack and stx-openstack-helm-fluxcd
    PASS: Build STX-Openstack tarball
    PASS: Upload / apply STX-Openstack on an AIO-SX machine with HTTPS
          enabled
    PASS: Verify that there's only one ingress pod per component
    PASS: After passing helm overrides, verify that they're applied
          (Example: change number of pods from 1 to 2)
    PEND: Confirm overrides work for every Openstack component

    Closes-Bug: 2036633

    Change-Id: Id01a7367335213015531e1e9186b2b1cc677ef8e
    Signed-off-by: Lucas de Ataides <email address hidden>

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: f/antelope
Review: https://review.opendev.org/c/starlingx/openstack-armada-app/+/895918

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/starlingx/openstack-armada-app/+/895918
Committed: https://opendev.org/starlingx/openstack-armada-app/commit/d8aa4293c6a2f04a3b5b0e1eef70c677bbb74572
Submitter: "Zuul (22348)"
Branch: f/antelope

commit d8aa4293c6a2f04a3b5b0e1eef70c677bbb74572
Author: Lucas de Ataides <email address hidden>
Date: Tue Sep 19 16:32:12 2023 -0300

    Add `get_namespace` function to OpenstackBaseHelm

    After the introdution of [1], it was noted that Helm overrides were not
    working properly for STX-Openstack. Althought they show up on
    helm-override-show, they we're not being passed to Helm and Kubernetes.
    This was caused because the [1] change updated the way that the
    overrides are applied, requiring the `get_namespace` function on the
    plugin to define where to apply these overrides, and, if this function
    did not exist, it would default to the `default` namespace.

    This change fixes that by adding the `get_namespace` function to the
    OpenstackBaseHelm plugin, which is extended by the other STX-O plugins.

    This change also does the following:

    1) Updates the base class for ingress, memcached and garbd from
       BaseHelm to OpenstackBaseHelm, as openstack.BaseHelm is the old base
       class for the armada app and probably should be cleaned now that we
       no longer support Armada.
    2) Remove ingress customizations for the kube-system namespace, as
       they're not used anymore and can be removed without any impact.

    [1] https://review.opendev.org/c/starlingx/config/+/887430

    Test Plan:
    PASS: Build python3-k8sapp-openstack and stx-openstack-helm-fluxcd
    PASS: Build STX-Openstack tarball
    PASS: Upload / apply STX-Openstack on an AIO-SX machine with HTTPS
          enabled
    PASS: Verify that there's only one ingress pod per component
    PASS: After passing helm overrides, verify that they're applied
          (Example: change number of pods from 1 to 2)
    PEND: Confirm overrides work for every Openstack component

    Closes-Bug: 2036633

    Change-Id: Id01a7367335213015531e1e9186b2b1cc677ef8e
    Signed-off-by: Lucas de Ataides <email address hidden>
    (cherry picked from commit aaddc97db459764590c486acac652142a74f0a63)