stx-openstack: Some pods are failing readiness/liveness probes
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
StarlingX |
Fix Released
|
Medium
|
Lucas de Ataides Barreto |
Bug Description
Brief Description
-----------------
It has been observed that some pods are *constantly* failing readiness/liveness probes, namely:
* cinder-api;
* glance-api;
* heat-api.
Functionality may have been affected.
Severity
--------
Major.
Steps to Reproduce
------------------
Upload and apply stx-openstack, and check the pods response to probes with:
$ kubectl -n openstack describe pod/<pod_name>
Expected Behavior
------------------
Pods are responding to probes as expected.
Actual Behavior
----------------
Pods are failing to respond to probes.
Reproducibility
---------------
Reproducible.
System Configuration
-------
Seen on AIO-DX.
Possibly happening on all system setups.
Branch/Pull Time/Commit
-------
StarlingX (master)
StarlingX OpenStack (master)
Last Pass
---------
N/A.
Timestamp/Logs
--------------
As an example, logs from glance-api:
```
Warning ProbeWarning 18m (x11057 over 31h) kubelet Liveness probe warning: {"versions": [{"id": "v2.9", "status": "CURRENT", "links": [{"rel": "self", "href": "http://
Warning ProbeWarning 3m35s (x11147 over 31h) kubelet Readiness probe warning: {"versions": [{"id": "v2.9", "status": "CURRENT", "links": [{"rel": "self", "href": "http://
```
Similar logs can be seen on cinder-api and heat-api.
Test Activity
-------------
Developer Testing.
Workaround
----------
N/A.
tags: | added: stx.9.0 stx.distro.openstack |
Changed in starlingx: | |
assignee: | nobody → Lucas de Ataides Barreto (ldeataid) |
Changed in starlingx: | |
importance: | Undecided → Medium |
Fix proposed to branch: master /review. opendev. org/c/starlingx /openstack- armada- app/+/910987
Review: https:/