The current client creates new .Session() on each request,
but since Horizon is a stateless app, each Session creates
new HttpAdapter, which itself has its own connection pool,
and each connection there is used (almost) once and then
is being kept in the pool(with Keep-Alive) for a certain
amount of time(waiting for inactivity timeout). The problem
is that the connection cannot be used anymore from next Django
calls - they create new connection pool with new connections, etc.
This keeps lots of open connections on the server.
Now the client will store an HTTPAdapter for each URL into
a singleton object, and will reuse its connections between
Django calls, but still taking advantage of Sessions during
a single page load(although we do not fully use this).
Note: the default pool behavior is non-blocking, which means
that if the max_pool_size is reached, a new connection will
still be opened, and when released - will be discarded.
It could be useful to add max_pool_size param into settings,
for performance fine-tuning. The default max_pool_size is 10.
Since python-novaclient is also used from non-Django projects,
I'd expect feedback from more people on the impact this change
could have over other projects.
Patch Set 3: Removed explicit connection closing, leaving
connections open in the pool.
Reviewed: https:/ /review. openstack. org/75196 /git.openstack. org/cgit/ openstack/ python- novaclient/ commit/ ?id=36db3b95f55 6d5f57a2bf49303 b24a0b25b4b7e8
Committed: https:/
Submitter: Jenkins
Branch: master
commit 36db3b95f556d5f 57a2bf49303b24a 0b25b4b7e8
Author: Tihomir Trifonov <email address hidden>
Date: Thu Feb 20 23:11:34 2014 +0200
Fix in in novaclient, to avoid excessive conns
The current client creates new .Session() on each request,
but since Horizon is a stateless app, each Session creates
new HttpAdapter, which itself has its own connection pool,
and each connection there is used (almost) once and then
is being kept in the pool(with Keep-Alive) for a certain
amount of time(waiting for inactivity timeout). The problem
is that the connection cannot be used anymore from next Django
calls - they create new connection pool with new connections, etc.
This keeps lots of open connections on the server.
Now the client will store an HTTPAdapter for each URL into
a singleton object, and will reuse its connections between
Django calls, but still taking advantage of Sessions during
a single page load(although we do not fully use this).
Note: the default pool behavior is non-blocking, which means
that if the max_pool_size is reached, a new connection will
still be opened, and when released - will be discarded.
It could be useful to add max_pool_size param into settings,
for performance fine-tuning. The default max_pool_size is 10.
Since python-novaclient is also used from non-Django projects,
I'd expect feedback from more people on the impact this change
could have over other projects.
Patch Set 3: Removed explicit connection closing, leaving
connections open in the pool.
Change-Id: Icc9dc2fa2863d0 e0e26a86c8180f2 e0fbcd1fcff
Closes-Bug: #1247056