openstack application apply failure

[Austin Sun]

Please use the template below when opening StarlingX bugs.

Brief Description
-----------------
in Multi Node, before apply openstack, apply other deployment and expose service, it will cause openstack apply failure.

Severity
--------
Provide the severity of the defect.
<Critical: System/Feature is not usable due to the defect>
<Major: System/Feature is usable but degraded>
<Minor: System/Feature is usable with minor issue>
Major
Steps to Reproduce
------------------
1) deploy and provision multi-node,
2) kubectl apply -f centos.yaml # in attachment.
3) system application-apply stx-openstack

Expected Behavior
------------------
openstack can be applied successfully

Actual Behavior
----------------
openstack applied fail

Reproducibility
---------------
<Reproducible/Intermittent/Seen once>
100%
System Configuration
--------------------
Multi-node system

Branch/Pull Time/Commit
-----------------------
Master

Last Pass
---------
NA
Use this section to also indicate if this is a new test scenario.

Timestamp/Logs
--------------

Test Activity
-------------
Developer Testing

Workaround
----------
tee nova.yaml << EOF
pod:
  use_fqdn:
    compute: false
EOF

system helm-override-update stx-openstack nova openstack --values nova.yaml

before apply , disable fqdn in nova, and apply again.

[Austin Sun (sunausti)]

nova-cell-setup-bkkcr 0/1 Init:0/2 0 12m
nova-compute-compute-0-5621f953-gt6cq 1/2 Running 0 12m
nova-compute-compute-1-532206f8-fvhhp 1/2 Running 0 12m

compute readness probe will fail.

 Warning Unhealthy 117s kubelet, compute-0 Readiness probe failed: Health probe timed out. Agent is down or response timed out

[sysadmin@controller-0 ~(keystone_admin)]$ kubectl exec -it -n openstack nova-compute-compute-0-5621f953-gt6cq -- bash
Defaulting container name to nova-compute.
Use 'kubectl describe pod/nova-compute-compute-0-5621f953-gt6cq -n openstack' to see all of the containers in this pod.
[root@compute-0 /]# hostname -A
192-188-204-33.centos.kube-system.svc.cluster.local compute-0-cluster-host compute-0 compute-0
[root@compute-0 /]#

The FQDN of po will be the daemon fqdn as deploy before. not compute-0, which will cause /tmp/health-probe.py failure.

[Austin Sun (sunausti)]

somehow, the gerrit is not connected to LP. manually paste
https://review.opendev.org/#/c/749462/

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/749462
Committed: https://git.openstack.org/cgit/starlingx/openstack-armada-app/commit/?id=13c8455b4ae3a6a853df2be78693df4f00964cf5
Submitter: Zuul
Branch: master

commit 13c8455b4ae3a6a853df2be78693df4f00964cf5
Author: Austin Sun <email address hidden>
Date: Wed Sep 2 23:14:02 2020 +0800

    Disable use_fqdn as default for nova helm

    the fqdn in nova-compute pod will be incorrect if expose service before
    apply openstack.
    for example:
    in nova-compute pod, it will get the fqdn
    [root@compute-0 /]# hostname -A
    192-188-204-33.centos.kube-system.svc.cluster.local
    compute-0-cluster-host compute-0 compute-0

    This will cause nova readness failure

    Closes-Bug: 1893908
    Change-Id: I85c34873528cca8fabf10ad7ddcb8997b23c8008
    Signed-off-by: Austin Sun <email address hidden>