I attempted to reproduce this by doing the following:
* heat.conf [database] max_pool_size=1, max_overflow=1
* launch 8 stacks each with 4 resources including one server
* call resource-metadata on all 8 stacks in a tight loop in 8 different processes
I failed to reproduce the issue but I may have some dependencies which are not up to date. To me this doesn't rule out a regression caused by a newer dependency.
I do agree that there is a straight-up optimisation that needs to happen to minimise the number of db calls for a stack load. For my part I'm going to look at replacing all the calls to db_api.resource_data_get with a db_api.resource_data_get_all that is called only once per resource load.
Clint's approach will help too, but access rules are decided by callback in the resource rather than a static rule, which may complicate things.
I attempted to reproduce this by doing the following:
* heat.conf [database] max_pool_size=1, max_overflow=1
* launch 8 stacks each with 4 resources including one server
* call resource-metadata on all 8 stacks in a tight loop in 8 different processes
I failed to reproduce the issue but I may have some dependencies which are not up to date. To me this doesn't rule out a regression caused by a newer dependency.
I do agree that there is a straight-up optimisation that needs to happen to minimise the number of db calls for a stack load. For my part I'm going to look at replacing all the calls to db_api. resource_ data_get with a db_api. resource_ data_get_ all that is called only once per resource load.
Clint's approach will help too, but access rules are decided by callback in the resource rather than a static rule, which may complicate things.