Neutron_api (unhealthy) after few days
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
neutron |
Won't Fix
|
Undecided
|
Ahmed Zaid | ||
oslo.service |
Confirmed
|
High
|
Ahmed Zaid |
Bug Description
Description
===========
on the undercloud ( pretty sure we also seen it on overcloud, i'll update when sure )
Without any action, we notice that neutron_api service is in "unhealthy" state and stop functioning.
Log shows -
2018-12-26 00:00:35.774 7 INFO oslo_service.
2018-12-26 00:00:36.077 40997 ERROR oslo_service.
openstack commands that needs neutron fails ( e.g openstack server list )
Restarting the docker ( neutron_api ) resolves the problem.
Steps to reproduce
==================
Deploy.
Wait 4 days.
Expected result
===============
Service should remain healthy..
Actual result
=============
not healthy ..
Environment
===========
Rocky , container based.
Logs & Configs
==============
Logs : http://
More info:
==========
Google showed this -
https:/
follow by -
http://
It seems that if we'll add "eventlet.sleep(0)" in <<<HERE>>> below, it might resolve the issue. -
def run_service(
"""Service start wrapper.
:param service: service to run
:param done: event to wait on until a shutdown is triggered
:returns: None
"""
try:
<<<<< HERE >>>>>>>>
except Exception:
raise SystemExit(1)
else:
The problem is that I didnt come up with an easy way to reproduce the issue in order to confirm it.
Any suggestions ?
Changed in oslo.service: | |
status: | New → Confirmed |
importance: | Undecided → High |
Changed in neutron: | |
assignee: | Yossi Ovadia (jabadia) → Ahmed Zaid (ahmedzaid10) |
Changed in oslo.service: | |
assignee: | nobody → Ahmed Zaid (ahmedzaid10) |
Code above is taken from "/usr/lib/ python2. 7/site- packages/ oslo_service/ service. py" Line 794