glance-api does not exit gracefully on sigkill and sigterm signal

[Abhishek Kekane]

When user sends either sigkill or sigterm signal to glance-api's parent process then it exits immediately without completing the current request. As per openstack serivce standards it should wait until current request is completed and should not accept new request for processing till exiting.

Note:
For kill signal parent process exits immediately where as child process keeps running forever whereas for terminate signal parent and child processes exits immediately.

Expected behavior:
1. glance-api should complete current request before exiting
2. glance-api should not accept new request for processing

[Abhishek Kekane (abhishek-kekane)]

For graceful shutdown user need to mention hook ( --hook-master-start "unix_signal:15 gracefully_kill_them_all") while starting the service.

This support is available from version 2.0.15
Reference: https://github.com/unbit/uwsgi/commit/1535b8ee139bdf9e9cca0b7ed12f1a1b8c15ffd0

Even after specifying the same workers doesn't exits gracefully. Another way is to set 'die-on-term' to False in uwsgi.ini file but after sending TERM signal it kills existing child immediately and spawns new child's (similar to reload concept) which is confusing.

Workaround:
1. Remove the KillSignal = SIGQUIT from [service] section of /<email address hidden>
2. Edit the /<email address hidden> file's [service] section
    Add below parameter [1]:
    KillMode = process

    Add the graceful shutdown hook as stated below:
    ExecStart = /usr/local/bin/uwsgi --ini /etc/glance/glance-uwsgi.ini --hook-master-start "unix_signal:15 gracefully_kill_them_all"

3. Edit the /etc/glance/glance-uwsgi.ini file of service:

    Add the below parameter [2]:
    worker-reload-mercy = <set the maximum time you think the request will take to complete>

[1] https://www.freedesktop.org/software/systemd/man/systemd.kill.html#KillMode=
[2] http://uwsgi-docs.readthedocs.io/en/latest/Options.html#worker-reload-mercy

Note:
I checked that whether is there any way to set the "worker-reload-mercy" to infinite time but there is no such way. I talked with uwsgi community on channel #uwsgi but according to <damjan> and <unixwitch> there is no way.

https://review.openstack.org/#/c/490903/ patch will solve this issue.