250.001 controller Configuration is out-of-date alarm not cleared after system installation

[Peng Peng]

Brief Description
-----------------
Configuration out-of-date alarms are not cleared on controller-0 after fresh install

Severity
--------
Minor

Steps to Reproduce
------------------
Install and configure a system

Expected Behavior
------------------
- system is alarm free after all hosts are unlocked and available

Actual Behavior
----------------
- All hosts are unlocked and available, but the config out-of-date status on controller-0 nodes are not cleared

Reproducibility
---------------
Unknown - first time this is seen in sanity, will monitor

System Configuration
--------------------
Dedicated storage
Lab-name: wcp_99-103

Branch/Pull Time/Commit
-----------------------
stx master as of 2020-03-29_16-39-59

Last Pass
---------
2020-03-29_16-39-59 in SX system

Timestamp/Logs
--------------

[2020-03-30 00:59:25,318] 314 DEBUG MainThread ssh.send :: Send 'fm --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://[abcd:204::1]:5000/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne alarm-list --nowrap --uuid'
[2020-03-30 00:59:26,742] 436 DEBUG MainThread ssh.expect :: Output:
+--------------------------------------+----------+---------------------------------------------------------------------------------------------------+------------------------------------------+----------+----------------------------+
| UUID | Alarm ID | Reason Text | Entity ID | Severity | Time Stamp |
+--------------------------------------+----------+---------------------------------------------------------------------------------------------------+------------------------------------------+----------+----------------------------+
| 7ba0b5f1-b932-46da-91e4-823ea335cb8c | 750.004 | Application Apply In Progress | k8s_application=platform-integ-apps | warning | 2020-03-30T00:55:50.555058 |
| f62b813e-128f-4745-8e17-d6eca8072e90 | 100.114 | NTP address 64:ff9b::23b7:39a9 is not a valid or a reachable NTP server. | host=controller-0.ntp=64:ff9b::23b7:39a9 | minor | 2020-03-30T00:53:13.957666 |
| 173dd559-f6f9-427e-8d71-eea1326b0832 | 100.114 | NTP address 64:ff9b::d173:b566 is not a valid or a reachable NTP server. | host=controller-0.ntp=64:ff9b::d173:b566 | minor | 2020-03-30T00:53:13.916205 |
| 9e536031-4c73-4fc4-9bd4-73e831f3c052 | 400.003 | Evaluation license key will expire on 30-jun-2020; there are 92 days remaining in this evaluation | host=controller-0 | minor | 2020-03-30T00:52:49.526093 |
+--------------------------------------+----------+---------------------------------------------------------------------------------------------------+------------------------------------------+----------+----------------------------+

Test Activity
-------------
Sanity

[Peng Peng (ppeng)]

log at
https://files.starlingx.kube.cengn.ca/launchpad/1869739

[Ghada Khalil (gkhalil)]

Marking as low / not gating - seems to be a one-time occurrence
If this becomes more frequent and reproducible, please request the priority to be increased.

[Peng Peng (ppeng)]

Issue was reproduced on
Lab: SM_3
Load: 2020-05-03_20-00-00

log added at:
https://files.starlingx.kube.cengn.ca/launchpad/1869739

[Ramaswamy Subramanian (rsubrama)]

May already be fixed. Please re-test and raise a new issues if it persists.

[John Kung (john-kung)]

Checked-In based on

Fix Released: Merged in stx5.0 Closes-Bug: https://bugs.launchpad.net/starlingx/+bug/1917229
Gerrit: https://opendev.org/starlingx/stx-puppet/commit/13ba2d4a7e6f9337eda22d98df872cb40ec983ac

PV Verified on Verified on same lab subcloud1 with stx5.0 2021-05-18_20-00-09.

Alarm 250.001 "Configuration is out-of-date" is not raised when enabling/disabling https.