DC: subcloud public endpoint unreachable

[Yang Liu]

Brief Description
-----------------
Subcloud public (OAM) auth URL is unreachable from system controller or other remote servers.

Impact: this blocks access to subcloud via remote CLI or remote API.

From Andy:
controller-0:/home/sysadmin# kubectl get globalnetworkpolicy
NAME AGE
controller-oam-if-gnp 16h

The gnp seems to be the cause. 5000 is not in the allowed ingress list.

From Greg:
Is the default globalnetworkpolicy different for subclouds ? ... it must be.
There could be other OAM/public ports that are not in the gnp for subclouds as well.

Severity
--------
Major

Steps to Reproduce
------------------
# Run a CLI using subcloud oam auth url (keystone public auth url from openstack end point list on subcloud)
[sysadmin@controller-0 ~(keystone_admin)]$ system --os-auth-url https://[2620:10a:a001:a103::1027]:5000/v3 --os-region-name subcloud1 --debug host-list
DEBUG (base:187) Making authentication request to https://[2620:10a:a001:a103::1027]:5000/v3/auth/tokens
DEBUG (connectionpool:818) Starting new HTTPS connection (1): 2620:10a:a001:a103::1027

Expected Behavior
------------------
subcloud oam auth URL should be reachable

Actual Behavior
----------------
Command hangs

Reproducibility
---------------
Reproducible

System Configuration
--------------------
DC

Branch/Pull Time/Commit
-----------------------

Last Pass
---------
Unknown

Timestamp/Logs
--------------

[sysadmin@controller-0 ~(keystone_admin)]$ date; system --os-auth-url https://[2620:10a:a001:a103::1027]:5000/v3 --os-region-name subcloud1 --debug host-list
Tue Jul 7 19:39:26 UTC 2020
DEBUG (base:187) Making authentication request to https://[2620:10a:a001:a103::1027]:5000/v3/auth/tokens
DEBUG (connectionpool:818) Starting new HTTPS connection (1): 2620:10a:a001:a103::1027

Test Activity
-------------
Normal use

[Frank Miller (sensfan22)]

Marking stx.5.0 gating. Users will want to rum remote CLI commands on subclouds.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/744965

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/744965
Committed: https://git.openstack.org/cgit/starlingx/stx-puppet/commit/?id=7b106c0a8ff507e9179ec199c6b447e99e8f318e
Submitter: Zuul
Branch: master

commit 7b106c0a8ff507e9179ec199c6b447e99e8f318e
Author: Andy Ning <email address hidden>
Date: Wed Aug 5 10:27:11 2020 -0400

    Enable subcloud keystone public endpoint access on OAM IF

    Currently keystone public endpoint on subcloud is not accessible from
    OAM IF for two reasons:

    - haproxy is not configured for keystone
    - The OAM global network policy doesn't allow keystone public endpoint
      port (5000) to go through.

    Inaccessible to keystone causes remote CLI not working as keystone is
    the first service the CLI needs to access to authenticate the requests.

    The change enabled the access by updating haproxy configuration and
    the OAM gnp.

    Change-Id: I540b44994c8c20f13d14d2e26eda6351b0f916e7
    Closes-Bug: 1886712
    Signed-off-by: Andy Ning <email address hidden>

[Yang Liu (yliu12)]

Verification passed on 0806 load on DC system.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: f/centos8
Review: https://review.opendev.org/762919