Excessively high greenlet default leads to connection pool timeouts in oslo.db
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
oslo.db |
Fix Released
|
Medium
|
Roman Podoliaka | ||
oslo.service |
Fix Released
|
Medium
|
Roman Podoliaka |
Bug Description
In oslo.service we default to 1000 greenlets in the pool for a WSGI server:
cfg.
Which means up to $wsgi_default_
It turned out that this does not play well with oslo.db defaults for a pool of DB connections, which default to SQLAlchemy settings, which only allow to have up to 5 connections in pool and 10 overflow (will be closed once returned to the pool).
With such defaults it's easy to start seeing timeout errors (SQLAlchemy gives up waiting on a connection to be available from pool/overflow after 30s by default) in service logs even with moderate concurrency.
For DB oriented services (most of OpenStack APIs) we should decrease the number of greenlets in pool and increase max_overflow value for DB connections.
More context: http://
Changed in oslo.service: | |
assignee: | nobody → Roman Podoliaka (rpodolyaka) |
Changed in oslo.db: | |
assignee: | nobody → Roman Podoliaka (rpodolyaka) |
status: | New → Confirmed |
Changed in oslo.service: | |
status: | New → Confirmed |
Changed in oslo.db: | |
importance: | Undecided → Medium |
Changed in oslo.service: | |
importance: | Undecided → Medium |
description: | updated |
Changed in oslo.service: | |
status: | Confirmed → Fix Released |
Changed in oslo.db: | |
status: | Confirmed → Fix Released |
This issue was fixed in the openstack/oslo.db 4.6.0 release.