platform-integ-apps apply-failed after lock/unlock controller in AIO-SX system

[Peng Peng]

Brief Description
-----------------
In AIO-SX, after controller lock/unlock, platform-integ-apps apply-failed
"the server could not find the requested resource (get configmaps)","grpc_status":2}"

Severity
--------
Major

Steps to Reproduce
------------------
As description

TC-name: mtc/test_lock_unlock_host.py::test_lock_unlock_host[controller]

Expected Behavior
------------------

Actual Behavior
----------------

Reproducibility
---------------
Seen once

System Configuration
--------------------
One node system

Lab-name: SM-4

Branch/Pull Time/Commit
-----------------------
2019-11-06_10-52-51

Last Pass
---------
2019-11-06_10-52-51 (another sanity run)

Timestamp/Logs
--------------
[2019-11-07 08:03:04,488] 311 DEBUG MainThread ssh.send :: Send 'system --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://192.168.204.1:5000/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne application-list'
[2019-11-07 08:03:06,191] 433 DEBUG MainThread ssh.expect :: Output:
+---------------------+---------+-------------------------------+---------------+---------+-----------+
| application | version | manifest name | manifest file | status | progress |
+---------------------+---------+-------------------------------+---------------+---------+-----------+
| platform-integ-apps | 1.0-9 | platform-integration-manifest | manifest.yaml | applied | completed |

[2019-11-07 08:03:08,095] 311 DEBUG MainThread ssh.send :: Send 'system --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://192.168.204.1:5000/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne host-lock controller-0'

[2019-11-07 08:03:51,758] 311 DEBUG MainThread ssh.send :: Send 'system --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://192.168.204.1:5000/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne host-unlock controller-0'

[2019-11-07 08:11:02,758] 311 DEBUG MainThread ssh.send :: Send 'system --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://192.168.204.1:5000/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne application-list'
[2019-11-07 08:11:04,524] 433 DEBUG MainThread ssh.expect :: Output:
+---------------------+---------+-------------------------------+---------------+---------+-----------+
| application | version | manifest name | manifest file | status | progress |
+---------------------+---------+-------------------------------+---------------+---------+-----------+
| platform-integ-apps | 1.0-9 | platform-integration-manifest | manifest.yaml | applied | completed |
+---------------------+---------+-------------------------------+---------------+---------+-----------+

[2019-11-07 08:11:28,688] 311 DEBUG MainThread ssh.send :: Send 'system --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://192.168.204.1:5000/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne application-list'
[2019-11-07 08:11:30,316] 433 DEBUG MainThread ssh.expect :: Output:
+---------------------+---------+-------------------------------+---------------+--------------+------------------------------------------+
| application | version | manifest name | manifest file | status | progress |
+---------------------+---------+-------------------------------+---------------+--------------+------------------------------------------+
| platform-integ-apps | 1.0-9 | platform-integration-manifest | manifest.yaml | apply-failed | operation aborted, check logs for detail |

/var/log/armada/platform-integ-apps-apply_2019-11-07-08-11-20.log:
2019-11-07 08:11:23.932 11 ERROR armada.cli [-] Caught unexpected exception: grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with:
        status = StatusCode.UNKNOWN
        details = "the server could not find the requested resource (get configmaps)"
        debug_error_string = "{"created":"@1573114283.067302643","description":"Error received from peer","file":"src/core/lib/surface/call.cc","file_line":1017,"grpc_message":"the server could not find the requested resource (get configmaps)","grpc_status":2}"

Test Activity
-------------
Sanity

[Peng Peng (ppeng)]

Seems it same as
https://bugs.launchpad.net/starlingx/+bug/1850189

[Ghada Khalil (gkhalil)]

Marking as duplicate

[Ghada Khalil (gkhalil)]

Duplicate LP: https://bugs.launchpad.net/starlingx/+bug/1850189 was resolved for stx.3.0

master: https://review.opendev.org/696311 merged on 2019-12-06
r/stx.3.0: https://review.opendev.org/698025 merged on 2019-12-09