Comment 5 for bug 1243251

The issue seems occurs less than before.

My last checks report 3 occurs in 1 weeks: http://bit.ly/IGUCx9

My understanding of the issue:

greenthread1(
   rpc IO: get a sample form rpc wire
   DB IO: record this sample to db
          * acquire a session from the sqlachemy pool
          * do the mysql requests (eventlet can switch to an other greenthread here)
          * release the session
) > perhaps this green_thread will be pause during the DB IO and an other one is started:
   greenthread2(
       rpc IO: get an other sample try to record it
       DB IO: record the second sample to db
   )

when does eventlet switch back to the first one ?
The question is how does eventlet orchestrate all of this IO ? Seems badly sometimes

during a tempest test (with errors) ~ 6 samples/sec are retrieved by the collector but one sample is really recorded in db every 4 seconds, just before the sqlachemy error we can see that the previous db operation have take 1 minute.

The default max sqlachemy pools size is 15 (with 5 persistent and 10 created ondemand if needed).

Conclusion if eventlet schedule the 16 rpc IO before doing the first sql request finish, the sqlachemy session pool is exhausted.