Cannot apply OpenStack after adding a certificate to Starlingx

[Lucas]

Brief Description
-----------------
Adding a certificate and ca_certificate using certificate-install -m {openstack | openstack_ca} ends up breaking openstack application. This happens because since https://review.opendev.org/c/starlingx/openstack-armada-app/+/749624 we force all request through public ingress, and after a certificate install public ingress activates TLS, however openstack services are unaware of this and do not trust this certificate.

Severity
--------
Provide the severity of the defect.
Critical

Steps to Reproduce
------------------
system modify --https_enabled=True
system certificate-install -m openstack <keyAndCert.pem>
system certificate-install -m openstack_ca <CACert.pem>
system application-apply stx-openstack

Expected Behavior
------------------
OpenStack is applied

Actual Behavior
----------------
Apply fails with pods reporting `certificate verify failed`

Reproducibility
---------------
100% reproducible

System Configuration
--------------------
All

Branch/Pull Time/Commit
-----------------------
Master (but it also happens on 5.0)

Last Pass
---------
Probably worked before https://review.opendev.org/c/starlingx/openstack-armada-app/+/749624

Timestamp/Logs
--------------

 stx-openstack | 1.0-56-centos-stable- | armada-manifest | stx-openstack.yaml | apply-failed | operation aborted, check |
| | versioned | | | | logs for detail

[sysadmin@controller-0 ~(keystone_admin)]$ ko logs keystone-bootstrap-xqz75
+ openstack role create --or-show member
SSL exception connecting to https://XXX/v3/roles: HTTPSConnectionPool(host='XXX', port=443): Max retries exceeded with url: /v3/roles (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",),))

Test Activity
-------------
Feature testing

Workaround
----------
Use Certificate from a Known CA

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/starlingx/helm-charts/+/802637

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/starlingx/openstack-armada-app/+/801778
Committed: https://opendev.org/starlingx/openstack-armada-app/commit/4fa28660d39382f5132ee29fd856ba3665828a31
Submitter: "Zuul (22348)"
Branch: master

commit 4fa28660d39382f5132ee29fd856ba3665828a31
Author: Lucas Cavalcante <email address hidden>
Date: Thu Jul 22 09:40:12 2021 -0300

    Add support for trust public ingress

    Adding a certificate and ca_certificate using:
    `certificate-install -m {openstack | openstack_ca}` ends up breaking
    openstack application. OS-STX forces public endpoint and when such
    endpoint has TLS enabled everything breaks, therefore based on the
    implementation of tls support for openstack-helm that enables tls
    for the openstack services we picked the trust cert code without
    actually enabling tls backends

    Signed-off-by: Lucas Cavalcante <email address hidden>
    Change-Id: I2dfc7c12defcc948fcdc353251301980e65f3011
    Closes-Bug: 1937260

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/starlingx/helm-charts/+/802637
Committed: https://opendev.org/starlingx/helm-charts/commit/4e77c25a2089f14416b95d743982eb04afb2b59a
Submitter: "Zuul (22348)"
Branch: master

commit 4e77c25a2089f14416b95d743982eb04afb2b59a
Author: Lucas Cavalcante <email address hidden>
Date: Tue Jul 27 20:48:12 2021 -0300

    Add support for trust public ingress

    Adding a certificate and ca_certificate using:
    `certificate-install -m {openstack | openstack_ca}` ends up breaking
    openstack application. OS-STX forces public endpoint and when such
    endpoint has TLS enabled everything breaks, therefore based on the
    implementation of tls support for openstack-helm that enables tls
    for the openstack services we picked the trust cert code without
    actually enabling tls backends

    Signed-off-by: Lucas Cavalcante <email address hidden>
    Partial-Bug: 1937260
    Change-Id: Idd6b67253d2e0d9817635c108a2cd1e89a35e0c8
    Depends-On: I2dfc7c12defcc948fcdc353251301980e65f3011