kube-apiserver service parameter oidc_issuer_url value is overwritten

[Michel Thebeau [WIND]]

Brief Description
-----------------

The OAM IP address value of kubernetes service parameter oidc_issuer_url is replaced with the platform cluster_host address.

Severity
--------

Major: System/Feature is usable but degraded

Steps to Reproduce
------------------

OAM_IP=10.10.105.2 # per the cluster configuration
system service-parameter-modify kubernetes kube_apiserver \
    oidc_issuer_url=https://${OAM_IP}:30556/dex
system service-parameter-apply kubernetes

# watch a minute or two until kube-apiserver process is restarted
ps -ef | grep "kube-api" | sed "s; --;\n --;g" | grep "kube-apiserver\|oidc"

Note how the URL is change from what was specified on the command-line

Expected Behavior
------------------

The value specified for oidc_issuer_url will be presented on the command line for kube-apiserver

Actual Behavior
----------------

The text of OAM IP is replaced with the value of hieradata platform::network::cluster_host::params::controller0_address

Reproducibility
---------------

100%

The following examples were tested, demonstrating that it is the OAM IP address being replaced, with dots interpreted as wildcard:
https://my.host.wrs.com:30556/dex ==> unaffected
https://10.10.105.134:30556/dex ==> unaffected
https://10.10.105.2:30556/dex ==> https://192.168.206.2:30556/dex
https://10.10.105.234:30556/dex ==> https://192.168.206.234:30556/dex
https://10.10.10.105:20556/dex ==> https://10.192.168.206.20556/dex

The last example highlights the loose replacement, affecting port as well.

Where 10.10.105.2 is the OAM address of my cluster and 192.168.206.2 is the cluster_host address.

System Configuration
--------------------

Nothing special, but consider also the OIDC configuration described in Starlingx docs:
https://docs.starlingx.io/security/kubernetes/overview-of-windows-active-directory.html

Branch/Pull Time/Commit
-----------------------
starlingx/master
2022-04-20 03:37:44
starlingx build 20220420T033744Z

Last Pass
---------
Unknown

Timestamp/Logs
--------------
Command line examples in Steps to Reproduce is sufficient to observe the result. But logs for kubernetes oidc authentication failure could be made available. (the replacement URL does not match the dex/oidc server "issuer" url and so authentication fails).

Test Activity
-------------
Developer Testing

Workaround
----------

Under test use the value of platform::network::cluster_host::params::controller0_address in the issuer URL configuration of dex, oidc-client and kube-apiserver

In the field, use a FQDN for the issuer url

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/starlingx/stx-puppet/+/846237

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/starlingx/stx-puppet/+/846237
Committed: https://opendev.org/starlingx/stx-puppet/commit/f6a29166ec00bd1a94459d838fa3f9f7117bf6f0
Submitter: "Zuul (22348)"
Branch: master

commit f6a29166ec00bd1a94459d838fa3f9f7117bf6f0
Author: Andy Ning <email address hidden>
Date: Thu Jun 16 15:52:49 2022 -0400

    Fix WAD user cannot access k8s API by oidc

    Currently when oidc-auth-apps is applied and oidc service
    parameters are applied, kube-apiserver's oidc_issuer_url points
    to cluster host floating IP instead of the OAM floating IP. This
    causes mis-match of oidc issuer that kube-apiserver is configured
    and the actual oidc issuer's IP address. User can no longer access
    k8s API even with a valid token.

    The issue is introduced by a sed substitution in
    kube-apiserver-change-params.erb where it replaces all the OAM IPs
    with kube-apisever's advertise address, including oidc-issuer-url.
    This fixed it by excluding oidc-issuer-url from the substitution.

    Test Plan for CentOS and Debian:
    PASS: oidc service parameters apply, helm overrides update and
          oidc-auth-apps apply
    PASS: run oidc-auth cli to get a token
    PASS: use the token to access k8s API by kubectl

    Closes-Bug: 1971500
    Closes-Bug: 1979006
    Signed-off-by: Andy Ning <email address hidden>
    Change-Id: I19d434c6322b4423d2e5b1732ff8af3f486b73f2