thanks matt, these errors happened occasionally, I could not reproduce that with a stable way. I'm running with OpenStack Newton, and some db driver dependent python packages as follows: MySQL-python (1.2.5) PyMySQL (0.7.6) SQLAlchemy (1.0.14) sqlalchemy-migrate (0.10.0) sqlparse (0.2.1) the nova database configs [database] connection = mysql+pymysql://xxxx:3306/nova max_overflow = 20 max_pool_size = 20 max_retries = -1 I got some clues from pymysql project https://github.com/PyMySQL/PyMySQL/issues/275, and it notice to openstack dev http://lists.openstack.org/pipermail/openstack-dev/2014-December/052344.html, but I'm not pretty sure that this is the root cause. hope would get some help from nova community. I got about 30 error records from nova log, but no errors occurred in other openstack components like neutron/cinder api-osapi.node-2.20190702.log:2019-07-02 10:08:42 +0800 | nova-api-osapi-5475cd965-9wbl2 | 2019-07-02 10:08:42.789 16 INFO nova.osapi_compute.wsgi.server [req-edd4d5e5-05ee-4a94-974b-c127fc4ce86f 1c827fbc129a4025bcb2a2d8cacc6b3d d8ac61ac9f1d4e5e9aa9c4313c668834 - default default] 10.233.66.23 "GET /v2.1/d8ac61ac9f1d4e5e9aa9c4313c668834/servers/detail?host=node-4.domain.tld&project_id=d8ac61ac9f1d4e5e9aa9c4313c668834&redirect=detail_x HTTP/1.1" status: 500 len: 568 time: 0.1924100 api-osapi.node-2.20190702.log:2019-07-02 10:08:42 +0800 | nova-api-osapi-5475cd965-9wbl2 | 2019-07-02 10:08:42.792 16 INFO nova.osapi_compute.wsgi.server [req-179abec8-6e3e-4ee2-a732-43709f25aa91 1c827fbc129a4025bcb2a2d8cacc6b3d d8ac61ac9f1d4e5e9aa9c4313c668834 - default default] 10.233.66.23 "GET /v2.1/d8ac61ac9f1d4e5e9aa9c4313c668834/servers/detail?host=node-2.domain.tld&project_id=d8ac61ac9f1d4e5e9aa9c4313c668834&redirect=detail_x HTTP/1.1" status: 500 len: 578 time: 0.1922901 api-osapi.node-2.20190702.log:2019-07-02 10:58:18 +0800 | nova-api-osapi-5475cd965-9wbl2 | 2019-07-02 10:58:18.438 16 INFO nova.osapi_compute.wsgi.server [req-b66fd2e0-428c-4d72-b1b3-82da136dbcad 1c827fbc129a4025bcb2a2d8cacc6b3d d8ac61ac9f1d4e5e9aa9c4313c668834 - default default] 10.233.66.23 "GET /v2.1/d8ac61ac9f1d4e5e9aa9c4313c668834/flavors/detail HTTP/1.1" status: 500 len: 566 time: 0.0666962 api-osapi.node-2.20190702.log:2019-07-02 10:58:18 +0800 | nova-api-osapi-5475cd965-9wbl2 | 2019-07-02 10:58:18.441 16 INFO nova.osapi_compute.wsgi.server [req-404eebfb-2e1e-48fe-ae5c-b44df4ff4395 1c827fbc129a4025bcb2a2d8cacc6b3d d8ac61ac9f1d4e5e9aa9c4313c668834 - default default] 10.233.66.23 "GET /v2.1/d8ac61ac9f1d4e5e9aa9c4313c668834/flavors/detail?is_public=False HTTP/1.1" status: 500 len: 576 time: 0.0668118 api-osapi.node-2.20190702.log:2019-07-02 10:59:19 +0800 | nova-api-osapi-5475cd965-9wbl2 | 2019-07-02 10:59:19.435 16 INFO nova.osapi_compute.wsgi.server [req-00fc27b8-d199-4510-adaf-a3c313b9dccc 1c827fbc129a4025bcb2a2d8cacc6b3d d8ac61ac9f1d4e5e9aa9c4313c668834 - default default] 10.233.66.23 "GET /v2.1/d8ac61ac9f1d4e5e9aa9c4313c668834/servers/detail?host=node-3.domain.tld&project_id=d8ac61ac9f1d4e5e9aa9c4313c668834&redirect=detail_x HTTP/1.1" status: 500 len: 568 time: 0.1546311 api-osapi.node-2.20190702.log:2019-07-02 10:59:19 +0800 | nova-api-osapi-5475cd965-9wbl2 | 2019-07-02 10:59:19.438 16 INFO nova.osapi_compute.wsgi.server [req-680717a3-a9e0-449b-bbb9-74ba2493af22 1c827fbc129a4025bcb2a2d8cacc6b3d d8ac61ac9f1d4e5e9aa9c4313c668834 - default default] 10.233.66.23 "GET /v2.1/d8ac61ac9f1d4e5e9aa9c4313c668834/servers/detail?host=node-2.domain.tld&project_id=d8ac61ac9f1d4e5e9aa9c4313c668834&redirect=detail_x HTTP/1.1" status: 500 len: 578 time: 0.1542120 api-osapi.node-2.20190702.log:2019-07-02 10:59:35 +0800 | nova-api-osapi-5475cd965-9wbl2 | 2019-07-02 10:59:35.866 16 INFO nova.osapi_compute.wsgi.server [req-eb08c21b-64e5-4af5-87b5-689e9e511bb8 1c827fbc129a4025bcb2a2d8cacc6b3d d8ac61ac9f1d4e5e9aa9c4313c668834 - default default] 10.233.66.23 "GET /v2.1/d8ac61ac9f1d4e5e9aa9c4313c668834/servers/detail?host=node-2.domain.tld&project_id=d8ac61ac9f1d4e5e9aa9c4313c668834&redirect=detail_x HTTP/1.1" status: 500 len: 568 time: 0.0986638 api-osapi.node-2.20190702.log:2019-07-02 10:59:35 +0800 | nova-api-osapi-5475cd965-9wbl2 | 2019-07-02 10:59:35.868 16 INFO nova.osapi_compute.wsgi.server [req-4683ee73-0ed3-48ad-91ed-9498bd5c7e96 1c827fbc129a4025bcb2a2d8cacc6b3d d8ac61ac9f1d4e5e9aa9c4313c668834 - default default] 10.233.66.23 "GET /v2.1/d8ac61ac9f1d4e5e9aa9c4313c668834/servers/detail?all_tenants=True&redirect=detail_x HTTP/1.1" status: 500 len: 578 time: 0.1148541 api-osapi.node-2.20190702.log:2019-07-02 10:59:59 +0800 | nova-api-osapi-5475cd965-9wbl2 | 2019-07-02 10:59:59.553 16 INFO nova.osapi_compute.wsgi.server [req-7abe6e71-9c14-4b90-a098-26fcb2c89420 1c827fbc129a4025bcb2a2d8cacc6b3d d8ac61ac9f1d4e5e9aa9c4313c668834 - default default] 10.233.65.164 "GET /v2.1/d8ac61ac9f1d4e5e9aa9c4313c668834/servers/detail?host=node-4.domain.tld&project_id=d8ac61ac9f1d4e5e9aa9c4313c668834&redirect=detail_x HTTP/1.1" status: 500 len: 568 time: 0.2493899 api-osapi.node-2.20190702.log:2019-07-02 10:59:59 +0800 | nova-api-osapi-5475cd965-9wbl2 | 2019-07-02 10:59:59.556 16 INFO nova.osapi_compute.wsgi.server [req-6ec1d6f5-3142-404f-89cc-fc4fb8e3218a 1c827fbc129a4025bcb2a2d8cacc6b3d d8ac61ac9f1d4e5e9aa9c4313c668834 - default default] 10.233.65.164 "GET /v2.1/d8ac61ac9f1d4e5e9aa9c4313c668834/servers/detail?host=node-2.domain.tld&project_id=d8ac61ac9f1d4e5e9aa9c4313c668834&redirect=detail_x HTTP/1.1" status: 500 len: 578 time: 0.2341990 api-osapi.node-2.20190702.log:2019-07-02 11:00:39 +0800 | nova-api-osapi-5475cd965-9wbl2 | 2019-07-02 11:00:39.972 16 INFO nova.osapi_compute.wsgi.server [req-afb737a0-0f50-475f-b208-db1bcdb599fd 1c827fbc129a4025bcb2a2d8cacc6b3d d8ac61ac9f1d4e5e9aa9c4313c668834 - default default] 10.233.66.23 "GET /v2.1/d8ac61ac9f1d4e5e9aa9c4313c668834/servers/detail?all_tenants=True&redirect=detail_x HTTP/1.1" status: 500 len: 568 time: 0.2033510 api-osapi.node-2.20190702.log:2019-07-02 11:05:09 +0800 | nova-api-osapi-5475cd965-9wbl2 | 2019-07-02 11:05:09.980 16 INFO nova.osapi_compute.wsgi.server [req-d9b0ca2b-51df-4987-b59f-f39b68539cc2 1c827fbc129a4025bcb2a2d8cacc6b3d d8ac61ac9f1d4e5e9aa9c4313c668834 - default default] 10.233.66.23 "GET /v2.1/d8ac61ac9f1d4e5e9aa9c4313c668834/servers/detail?host=node-4.domain.tld&project_id=d8ac61ac9f1d4e5e9aa9c4313c668834&redirect=detail_x HTTP/1.1" status: 500 len: 568 time: 0.2005029 api-osapi.node-2.20190702.log:2019-07-02 11:05:09 +0800 | nova-api-osapi-5475cd965-9wbl2 | 2019-07-02 11:05:09.983 16 INFO nova.osapi_compute.wsgi.server [req-bc642bdb-a847-487b-9273-0697cfdf14d9 1c827fbc129a4025bcb2a2d8cacc6b3d d8ac61ac9f1d4e5e9aa9c4313c668834 - default default] 10.233.66.23 "GET /v2.1/d8ac61ac9f1d4e5e9aa9c4313c668834/servers/detail?host=node-2.domain.tld&project_id=d8ac61ac9f1d4e5e9aa9c4313c668834&redirect=detail_x HTTP/1.1" status: 500 len: 578 time: 0.2071099 api-osapi.node-2.20190702.log:2019-07-02 11:05:35 +0800 | nova-api-osapi-5475cd965-9wbl2 | 2019-07-02 11:05:35.367 16 INFO nova.osapi_compute.wsgi.server [req-827d05e3-b3a1-49be-aca3-db8614243ad0 1c827fbc129a4025bcb2a2d8cacc6b3d d8ac61ac9f1d4e5e9aa9c4313c668834 - default default] 10.233.67.122 "GET /v2.1/d8ac61ac9f1d4e5e9aa9c4313c668834/servers/detail?host=node-3.domain.tld&project_id=d8ac61ac9f1d4e5e9aa9c4313c668834&redirect=detail_x HTTP/1.1" status: 500 len: 568 time: 0.1658230 api-osapi.node-2.20190702.log:2019-07-02 11:06:26 +0800 | nova-api-osapi-5475cd965-9wbl2 | 2019-07-02 11:06:26.115 16 INFO nova.osapi_compute.wsgi.server [req-52cc0de7-7cd2-4ed9-b10e-64440c006afa 1c827fbc129a4025bcb2a2d8cacc6b3d d8ac61ac9f1d4e5e9aa9c4313c668834 - default default] 10.233.66.23 "GET /v2.1/d8ac61ac9f1d4e5e9aa9c4313c668834/servers/detail?host=node-2.domain.tld&project_id=d8ac61ac9f1d4e5e9aa9c4313c668834&redirect=detail_x HTTP/1.1" status: 500 len: 568 time: 0.2764349 api-osapi.node-2.20190702.log:2019-07-02 11:06:26 +0800 | nova-api-osapi-5475cd965-9wbl2 | 2019-07-02 11:06:26.118 16 INFO nova.osapi_compute.wsgi.server [req-3d2aebf2-bab7-4d17-ae28-b7b0ddb26f9e 1c827fbc129a4025bcb2a2d8cacc6b3d d8ac61ac9f1d4e5e9aa9c4313c668834 - default default] 10.233.66.23 "GET /v2.1/d8ac61ac9f1d4e5e9aa9c4313c668834/servers/detail?project_id=d8ac61ac9f1d4e5e9aa9c4313c668834&redirect=detail_x&status=ERROR HTTP/1.1" status: 500 len: 578 time: 0.2550640 api-osapi.node-2.20190702.log:2019-07-02 11:20:30 +0800 | nova-api-osapi-5475cd965-9wbl2 | 2019-07-02 11:20:30.526 16 INFO nova.osapi_compute.wsgi.server [req-2cb77092-f92c-448f-9a5a-f283abff6c2c 1c827fbc129a4025bcb2a2d8cacc6b3d d8ac61ac9f1d4e5e9aa9c4313c668834 - default default] 10.233.67.122 "GET /v2.1/d8ac61ac9f1d4e5e9aa9c4313c668834/flavors/detail HTTP/1.1" status: 500 len: 566 time: 0.0660131 api-osapi.node-2.20190702.log:2019-07-02 11:31:50 +0800 | nova-api-osapi-5475cd965-9wbl2 | 2019-07-02 11:31:50.726 16 INFO nova.osapi_compute.wsgi.server [req-4416a16b-54c0-4ad2-a36e-e522b22f83f2 1c827fbc129a4025bcb2a2d8cacc6b3d d8ac61ac9f1d4e5e9aa9c4313c668834 - default default] 10.233.67.122 "GET /v2.1/d8ac61ac9f1d4e5e9aa9c4313c668834/servers/detail?host=node-4.domain.tld&project_id=d8ac61ac9f1d4e5e9aa9c4313c668834&redirect=detail_x HTTP/1.1" status: 500 len: 568 time: 0.1068242 api-osapi.node-2.20190702.log:2019-07-02 11:31:50 +0800 | nova-api-osapi-5475cd965-9wbl2 | 2019-07-02 11:31:50.729 16 INFO nova.osapi_compute.wsgi.server [req-95480b17-f258-46ca-a4b3-dc0500e0f40c 1c827fbc129a4025bcb2a2d8cacc6b3d d8ac61ac9f1d4e5e9aa9c4313c668834 - default default] 10.233.67.122 "GET /v2.1/d8ac61ac9f1d4e5e9aa9c4313c668834/servers/detail?project_id=d8ac61ac9f1d4e5e9aa9c4313c668834&redirect=detail_x&status=ERROR HTTP/1.1" status: 500 len: 578 time: 0.1145289 api-osapi.node-2.20190702.log:2019-07-02 11:32:49 +0800 | nova-api-osapi-5475cd965-9wbl2 | 2019-07-02 11:32:49.900 16 INFO nova.osapi_compute.wsgi.server [req-b72a1068-2b9d-4946-80ab-59c15fb5527a 1c827fbc129a4025bcb2a2d8cacc6b3d d8ac61ac9f1d4e5e9aa9c4313c668834 - default default] 10.233.67.122 "GET /v2.1/d8ac61ac9f1d4e5e9aa9c4313c668834/servers/d57a387c-bf4f-4e73-9b2b-080d2794684e HTTP/1.1" status: 500 len: 568 time: 0.3209841 api-osapi.node-2.20190702.log:2019-07-02 11:32:49 +0800 | nova-api-osapi-5475cd965-9wbl2 | 2019-07-02 11:32:49.903 16 INFO nova.osapi_compute.wsgi.server [req-cc5270fd-cc1f-49a1-9cac-78bff8c44640 1c827fbc129a4025bcb2a2d8cacc6b3d d8ac61ac9f1d4e5e9aa9c4313c668834 - default default] 10.233.67.122 "GET /v2.1/d8ac61ac9f1d4e5e9aa9c4313c668834/servers/a96d84ef-72b4-4267-8337-40f61c355696 HTTP/1.1" status: 500 len: 578 time: 0.3278821 api-osapi.node-2.20190702.log:2019-07-02 11:44:09 +0800 | nova-api-osapi-5475cd965-9wbl2 | 2019-07-02 11:44:09.463 16 INFO nova.osapi_compute.wsgi.server [req-9a970a8c-eab3-4810-ab73-6fe5bd306715 1c827fbc129a4025bcb2a2d8cacc6b3d d8ac61ac9f1d4e5e9aa9c4313c668834 - default default] 10.233.67.122 "GET /v2.1/d8ac61ac9f1d4e5e9aa9c4313c668834/flavors/detail HTTP/1.1" status: 500 len: 566 time: 0.0566730 api-osapi.node-2.20190702.log:2019-07-02 11:44:09 +0800 | nova-api-osapi-5475cd965-9wbl2 | 2019-07-02 11:44:09.466 16 INFO nova.osapi_compute.wsgi.server [req-3a2610a6-7580-42ef-851e-b626e31953b1 1c827fbc129a4025bcb2a2d8cacc6b3d d8ac61ac9f1d4e5e9aa9c4313c668834 - default default] 10.233.67.122 "GET /v2.1/d8ac61ac9f1d4e5e9aa9c4313c668834/flavors/detail?is_public=False HTTP/1.1" status: 500 len: 576 time: 0.0621471 api-osapi.node-2.20190702.log:2019-07-02 12:06:05 +0800 | nova-api-osapi-5475cd965-9wbl2 | 2019-07-02 12:06:05.850 16 INFO nova.osapi_compute.wsgi.server [req-3c849000-612d-4c77-a175-85ac5e3a0be7 1c827fbc129a4025bcb2a2d8cacc6b3d d8ac61ac9f1d4e5e9aa9c4313c668834 - default default] 10.233.67.122 "GET /v2.1/d8ac61ac9f1d4e5e9aa9c4313c668834/servers/detail?host=node-3.domain.tld&project_id=d8ac61ac9f1d4e5e9aa9c4313c668834&redirect=detail_x HTTP/1.1" status: 500 len: 568 time: 0.0923002 api-osapi.node-2.20190702.log:2019-07-02 12:06:05 +0800 | nova-api-osapi-5475cd965-9wbl2 | 2019-07-02 12:06:05.853 16 INFO nova.osapi_compute.wsgi.server [req-b1190eaf-1e67-421f-8f0c-3f12eb041a19 1c827fbc129a4025bcb2a2d8cacc6b3d d8ac61ac9f1d4e5e9aa9c4313c668834 - default default] 10.233.67.122 "GET /v2.1/d8ac61ac9f1d4e5e9aa9c4313c668834/servers/detail?project_id=d8ac61ac9f1d4e5e9aa9c4313c668834&redirect=detail_x&status=BUILD HTTP/1.1" status: 500 len: 578 time: 0.0974100 api-osapi.node-2.20190702.log:2019-07-02 13:30:48 +0800 | nova-api-osapi-5475cd965-9wbl2 | 2019-07-02 13:30:48.342 16 INFO nova.osapi_compute.wsgi.server [req-464bcc52-07ab-425d-a22d-b9614d754ee2 1c827fbc129a4025bcb2a2d8cacc6b3d d8ac61ac9f1d4e5e9aa9c4313c668834 - default default] 10.233.67.122 "GET /v2.1/d8ac61ac9f1d4e5e9aa9c4313c668834/servers/detail?project_id=d8ac61ac9f1d4e5e9aa9c4313c668834&redirect=detail_x&status=BUILD HTTP/1.1" status: 500 len: 568 time: 0.0822620 api-osapi.node-2.20190702.log:2019-07-02 13:30:48 +0800 | nova-api-osapi-5475cd965-9wbl2 | 2019-07-02 13:30:48.345 16 INFO nova.osapi_compute.wsgi.server [req-6d6556bc-2576-44f8-af12-5d6d6787defd 1c827fbc129a4025bcb2a2d8cacc6b3d d8ac61ac9f1d4e5e9aa9c4313c668834 - default default] 10.233.67.122 "GET /v2.1/d8ac61ac9f1d4e5e9aa9c4313c668834/servers/detail?host=node-2.domain.tld&project_id=d8ac61ac9f1d4e5e9aa9c4313c668834&redirect=detail_x HTTP/1.1" status: 500 len: 572 time: 0.0868950 api-osapi.node-2.20190702.log:2019-07-02 13:53:51 +0800 | nova-api-osapi-5475cd965-9wbl2 | 2019-07-02 13:53:51.859 16 INFO nova.osapi_compute.wsgi.server [req-1f378baf-85bc-43cc-97fb-80e5c0427dc4 1c827fbc129a4025bcb2a2d8cacc6b3d d8ac61ac9f1d4e5e9aa9c4313c668834 - default default] 10.233.65.164 "GET /v2.1/d8ac61ac9f1d4e5e9aa9c4313c668834/os-hypervisors/detail HTTP/1.1" status: 500 len: 566 time: 0.0327928 api-osapi.node-2.20190702.log:2019-07-02 13:53:51 +0800 | nova-api-osapi-5475cd965-9wbl2 | 2019-07-02 13:53:51.861 16 INFO nova.osapi_compute.wsgi.server [req-681fb122-cd67-4557-99dc-998dd5f7056a 35d4f894f4de422a9056722abe668d79 b028c099cf6b4cd1b76e6fe33c8233b2 - default default] 10.233.67.109 "GET /v2.1/b028c099cf6b4cd1b76e6fe33c8233b2/os-hypervisors/detail HTTP/1.1" status: 500 len: 570 time: 0.0230091 api-osapi.node-2.20190702.log:2019-07-02 13:55:43 +0800 | nova-api-osapi-5475cd965-9wbl2 | 2019-07-02 13:55:43.937 16 INFO nova.osapi_compute.wsgi.server [req-8127b93b-3869-4163-822e-ed6726fea0f1 1c827fbc129a4025bcb2a2d8cacc6b3d d8ac61ac9f1d4e5e9aa9c4313c668834 - default default] 10.233.65.164 "GET /v2.1/d8ac61ac9f1d4e5e9aa9c4313c668834/servers/detail?host=node-3.domain.tld&project_id=d8ac61ac9f1d4e5e9aa9c4313c668834&redirect=detail_x HTTP/1.1" status: 500 len: 568 time: 0.0963371 api-osapi.node-2.20190702.log:2019-07-02 13:55:43 +0800 | nova-api-osapi-5475cd965-9wbl2 | 2019-07-02 13:55:43.940 16 INFO nova.osapi_compute.wsgi.server [req-bfb0dcb2-6f03-4072-8851-d56eacb54881 1c827fbc129a4025bcb2a2d8cacc6b3d d8ac61ac9f1d4e5e9aa9c4313c668834 - default default] 10.233.65.164 "GET /v2.1/d8ac61ac9f1d4e5e9aa9c4313c668834/servers/detail?host=node-4.domain.tld&project_id=d8ac61ac9f1d4e5e9aa9c4313c668834&redirect=detail_x HTTP/1.1" status: 500 len: 572 time: 0.1042821