With wsgi_default_pool_size=100[1], if the keepalived state change server gets 100 requests at the same time, while processing the requests heavy load is seen on cpu, making the network node unresponsive. For each request, keepalived state change server spawns a new meta data proxy process(i.e neutron-ns-metadata-proxy). During heavy cpu load, with "top" command, I can see many metadata proxy processes in "running" state at same time(see the attachment).
When wsgi_default_pool_size=8, I see state change server spawning 8 metadata proxy processes at a time("top" command shows 8 meta data proxy processes in "running" state at a time), cpu load is less and metadata proxy processes(for example, 100) spawned for all requests without failures.
We can keep wsgi_default_pool_size=100 for neutron API server, and use seperate configuration for UnixDomainWSGIServer(for example CONF.unix_domain_wsgi_default_pool_size).
neutron/agent/linux/utils.py
class UnixDomainWSGIServer(wsgi.Server):
def _run(self, application, socket):
"""Start a WSGI service in a new green thread."""
logger = logging.getLogger('eventlet.wsgi.server') eventlet.wsgi.server(socket, application, max_size=CONF.unix_domain_wsgi_default_pool_size, protocol=UnixDomainHttpProtocol, log=logger)
With wsgi_default_ pool_size= 100[1], if the keepalived state change server gets 100 requests at the same time, while processing the requests heavy load is seen on cpu, making the network node unresponsive. For each request, keepalived state change server spawns a new meta data proxy process(i.e neutron- ns-metadata- proxy). During heavy cpu load, with "top" command, I can see many metadata proxy processes in "running" state at same time(see the attachment).
When wsgi_default_ pool_size= 8, I see state change server spawning 8 metadata proxy processes at a time("top" command shows 8 meta data proxy processes in "running" state at a time), cpu load is less and metadata proxy processes(for example, 100) spawned for all requests without failures.
We can keep wsgi_default_ pool_size= 100 for neutron API server, and use seperate configuration for UnixDomainWSGIS erver(for example CONF.unix_ domain_ wsgi_default_ pool_size) .
neutron/ agent/linux/ utils.py erver(wsgi. Server) :
class UnixDomainWSGIS
def _run(self, application, socket): getLogger( 'eventlet. wsgi.server' )
eventlet. wsgi.server( socket,
application,
max_size= CONF.unix_ domain_ wsgi_default_ pool_size,
protocol= UnixDomainHttpP rotocol,
log=logger)
"""Start a WSGI service in a new green thread."""
logger = logging.
[1] https:/ /github. com/openstack/ neutron/ commit/ 9d573387f1e33ce 85269d3ed9be501 717eed4807