When I set the 2 or more each designate service workers, those service was not working.
At first, I found this problem on newton version. but it happened this problem newer version better then newton.
I set the number of worker is 5 and set the same as logging point.
2019-04-04 12:37:24.512 98588 DEBUG designate.service_status [-] designate/service_status.HeartBeatEmitter.__init__: 98588 __init__ /usr/lib/python2.7/site-packages/designate/service_status.py:53
2019-04-04 12:37:24.533 98598 DEBUG designate.service_status [-] designate/service_status.HeartBeatEmitter.start: 98598 start /usr/lib/python2.7/site-packages/designate/service_status.py:100
2019-04-04 12:37:24.533 98598 DEBUG designate.service_status [-] designate/service_status.HeartBeatEmitter.start: _running=True start /usr/lib/python2.7/site-packages/designate/service_status.py:102
2019-04-04 12:37:24.536 98599 DEBUG designate.service_status [-] designate/service_status.HeartBeatEmitter.start: 98599 start /usr/lib/python2.7/site-packages/designate/service_status.py:100
2019-04-04 12:37:24.536 98599 DEBUG designate.service_status [-] designate/service_status.HeartBeatEmitter.start: _running=True start /usr/lib/python2.7/site-packages/designate/service_status.py:102
2019-04-04 12:37:24.537 98600 DEBUG designate.service_status [-] designate/service_status.HeartBeatEmitter.start: 98600 start /usr/lib/python2.7/site-packages/designate/service_status.py:100
2019-04-04 12:37:24.538 98600 DEBUG designate.service_status [-] designate/service_status.HeartBeatEmitter.start: _running=True start /usr/lib/python2.7/site-packages/designate/service_status.py:102
2019-04-04 12:37:24.563 98601 DEBUG designate.service_status [-] designate/service_status.HeartBeatEmitter.start: 98601 start /usr/lib/python2.7/site-packages/designate/service_status.py:100
2019-04-04 12:37:24.563 98601 DEBUG designate.service_status [-] designate/service_status.HeartBeatEmitter.start: _running=True start /usr/lib/python2.7/site-packages/designate/service_status.py:102
2019-04-04 12:37:24.636 98588 DEBUG designate.service_status [-] designate/service_status.HeartBeatEmitter._emit_heartbeat: 98588 _emit_heartbeat /usr/lib/python2.7/site-packages/designate/service_status.py:75
2019-04-04 12:37:24.636 98588 DEBUG designate.service_status [-] designate/service_status.HeartBeatEmitter._emit_heartbeat: _running=False _emit_heartbeat /usr/lib/python2.7/site-packages/designate/service_status.py:76
2019-04-04 12:37:29.637 98588 DEBUG designate.service_status [-] designate/service_status.HeartBeatEmitter._emit_heartbeat: 98588 _emit_heartbeat /usr/lib/python2.7/site-packages/designate/service_status.py:75
2019-04-04 12:37:29.637 98588 DEBUG designate.service_status [-] designate/service_status.HeartBeatEmitter._emit_heartbeat: _running=False _emit_heartbeat /usr/lib/python2.7/site-packages/designate/service_status.py:76
__init__ and _emit_heartbeat are same process id. But start is not same as those.
__init__ , _emit_heartbeat: 98588
start : 98598, 98598, 98599, 98600, 98601
then I checked concern about process IDs.
# ps auxwwf | grep designate-api
designa+ 98588 1.9 0.7 347336 62380 ? Ss 12:37 0:02 /usr/bin/python2 /usr/bin/designate-api --config-file /etc/designate/designate.conf --log-file /var/log/designate/api.log
designa+ 98598 0.2 0.8 392348 69792 ? S 12:37 0:00 \_ /usr/bin/python2 /usr/bin/designate-api --config-file /etc/designate/designate.conf --log-file /var/log/designate/api.log
designa+ 98599 0.2 0.8 392344 69784 ? S 12:37 0:00 \_ /usr/bin/python2 /usr/bin/designate-api --config-file /etc/designate/designate.conf --log-file /var/log/designate/api.log
designa+ 98600 0.2 0.8 392344 69784 ? S 12:37 0:00 \_ /usr/bin/python2 /usr/bin/designate-api --config-file /etc/designate/designate.conf --log-file /var/log/designate/api.log
designa+ 98601 0.2 0.8 392348 69784 ? S 12:37 0:00 \_ /usr/bin/python2 /usr/bin/designate-api --config-file /etc/designate/designate.conf --log-file /var/log/designate/api.log
98588 is parent process ID, but others are child process.
So if _running flag update to True, It never start the _emit_heartbeat.
How do I fix this problem
# diff -u /usr/lib/python2.7/site-packages/designate/cmd/api.py.upstream/usr/lib/python2.7/site-packages/designate/cmd/api.py
--- /usr/lib/python2.7/site-packages/designate/cmd/api.py.upstream 2019-04-03 18:10:09.908918965 +0900
+++ /usr/lib/python2.7/site-packages/designate/cmd/api.py 2019-04-03 18:09:49.886038819 +0900
@@ -40,4 +40,5 @@
server = api_service.Service(threads=CONF['service:api'].threads)
service.serve(server, workers=CONF['service:api'].workers)
+ server.heartbeat_emitter.start()
service.wait()
I fixed to call the start method from parent process. I think all process have same problem. So I think It is need to fix same way.
But I couldn't know how do I write the unittest in this case. Do you guys have any idea? I couldn't find the multi workers case in designate's unittest.
reference and detail: https://<email address hidden>/when-increasing-the-designate-worker-it-doesnt-work-the-service-statuses-6a4372d8f3e0
Fix proposed to branch: master /review. opendev. org/657382
Review: https:/