platform-integ-apps fail to apply initially due to could not find the requested resource (get configmaps)

Bug #1859645 reported by Peng Peng on 2020-01-14
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
StarlingX
High
Stefan Dinescu

Bug Description

Brief Description
-----------------
After initial install and config, platform-integ-apps in apply-failed status due to "the server could not find the requested resource (get configmaps)"

Severity
--------
minor

Steps to Reproduce
------------------
Install and configure a stx system

Expected Behavior
------------------
- platform-integ-apps automatically applied successfully

Actual Behavior
----------------
- platform-integ-apps occasionally in apply-failed

Reproducibility
---------------
Unknown - first time this is seen it
looks similar with LP-1850189

System Configuration
--------------------
One node system

Lab-name: SM-4

Branch/Pull Time/Commit
-----------------------
20200111T023000Z

Last Pass
---------
20200107T000000Z

Timestamp/Logs
--------------
system application-list
+---------------------+---------+-------------------------------+---------------+--------------+------------------------------------------+
| application | version | manifest name | manifest file | status | progress |
+---------------------+---------+-------------------------------+---------------+--------------+------------------------------------------+
| platform-integ-apps | 1.0-8 | platform-integration-manifest | manifest.yaml | apply-failed | operation aborted, check logs for detail |
+---------------------+---------+-------------------------------+---------------+--------------+------------------------------------------+

2020-01-14 07:57:24.225 107 ERROR armada.cli [-] Caught unexpected exception: grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with:
        status = StatusCode.UNKNOWN
        details = "the server could not find the requested resource (get configmaps)"
        debug_error_string = "{"created":"@1578988643.425059681","description":"Error received from peer","file":"src/core/lib/surface/call.cc","file_line":1017,"grpc_message":"the server could not find the requested resource (get configmaps)","grpc_status":2}"
>
2020-01-14 07:57:24.225 107 ERROR armada.cli Traceback (most recent call last):
2020-01-14 07:57:24.225 107 ERROR armada.cli File "/usr/local/lib/python3.6/dist-packages/armada/cli/__init__.py", line 38, in safe_invoke
2020-01-14 07:57:24.225 107 ERROR armada.cli self.invoke()
2020-01-14 07:57:24.225 107 ERROR armada.cli File "/usr/local/lib/python3.6/dist-packages/armada/cli/apply.py", line 213, in invoke
2020-01-14 07:57:24.225 107 ERROR armada.cli resp = self.handle(documents, tiller)
2020-01-14 07:57:24.225 107 ERROR armada.cli File "/usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py", line 81, in func_wrapper
2020-01-14 07:57:24.225 107 ERROR armada.cli return future.result()
2020-01-14 07:57:24.225 107 ERROR armada.cli File "/usr/lib/python3.6/concurrent/futures/_base.py", line 425, in result
2020-01-14 07:57:24.225 107 ERROR armada.cli return self.__get_result()
2020-01-14 07:57:24.225 107 ERROR armada.cli File "/usr/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result
2020-01-14 07:57:24.225 107 ERROR armada.cli raise self._exception
2020-01-14 07:57:24.225 107 ERROR armada.cli File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run
2020-01-14 07:57:24.225 107 ERROR armada.cli result = self.fn(*self.args, **self.kwargs)
2020-01-14 07:57:24.225 107 ERROR armada.cli File "/usr/local/lib/python3.6/dist-packages/armada/cli/apply.py", line 256, in handle
2020-01-14 07:57:24.225 107 ERROR armada.cli return armada.sync()
2020-01-14 07:57:24.225 107 ERROR armada.cli File "/usr/local/lib/python3.6/dist-packages/armada/handlers/armada.py", line 189, in sync
2020-01-14 07:57:24.225 107 ERROR armada.cli known_releases = self.tiller.list_releases()
2020-01-14 07:57:24.225 107 ERROR armada.cli File "/usr/local/lib/python3.6/dist-packages/armada/handlers/tiller.py", line 252, in list_releases
2020-01-14 07:57:24.225 107 ERROR armada.cli releases = get_results()
2020-01-14 07:57:24.225 107 ERROR armada.cli File "/usr/local/lib/python3.6/dist-packages/armada/handlers/tiller.py", line 220, in get_results
2020-01-14 07:57:24.225 107 ERROR armada.cli for message in response:
2020-01-14 07:57:24.225 107 ERROR armada.cli File "/usr/local/lib/python3.6/dist-packages/grpc/_channel.py", line 364, in __next__
2020-01-14 07:57:24.225 107 ERROR armada.cli return self._next()
2020-01-14 07:57:24.225 107 ERROR armada.cli File "/usr/local/lib/python3.6/dist-packages/grpc/_channel.py", line 358, in _next
2020-01-14 07:57:24.225 107 ERROR armada.cli raise self
2020-01-14 07:57:24.225 107 ERROR armada.cli grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with:
2020-01-14 07:57:24.225 107 ERROR armada.cli status = StatusCode.UNKNOWN
2020-01-14 07:57:24.225 107 ERROR armada.cli details = "the server could not find the requested resource (get configmaps)"
2020-01-14 07:57:24.225 107 ERROR armada.cli debug_error_string = "{"created":"@1578988643.425059681","description":"Error received from peer","file":"src/core/lib/surface/call.cc","file_line":1017,"grpc_message":"the server could not find the requested resource (get configmaps)","grpc_status":2}"

Test Activity
-------------
Sanity

Ghada Khalil (gkhalil) on 2020-01-15
tags: added: stx.containers
Yang Liu (yliu12) on 2020-01-16
tags: added: stx.retestneeded
Anujeyan Manokeran (anujeyan) wrote :

This issue was seen in SM-2 as well on load 2020-01-17 02:30:05.

Ghada Khalil (gkhalil) wrote :

stx.4.0 / high priority - issue seen twice recently. According to Yang, a re-apply doesn't work in this case.

Changed in starlingx:
importance: Undecided → Medium
status: New → Triaged
assignee: nobody → Stefan Dinescu (stefandinescu)
tags: added: stx.4.0
Changed in starlingx:
importance: Medium → High
Peng Peng (ppeng) wrote :

Issue reproduced on
Lab: WCP_71_75
Load: 2020-01-22_20-00-00
log @
https://files.starlingx.kube.cengn.ca/launchpad/1859645

Peng Peng (ppeng) wrote :

Issue reproduced on
Lab: WCP_71_75
Load: 2020-01-28_18-49-15
log @
https://files.starlingx.kube.cengn.ca/launchpad/1859645

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers