datasource sync taking very long (10+ sec) on datasource delete

[Eric K]

When deleting a datasource, the sync often takes 10+ seconds to complete. It appears that the deleting greenthread relinquished control (expected and on datasource ops) and takes a long time to regain control. The effect on the user/deployer is that deleting a datasource tends to take more than 10 seconds. In my testing, it appears to be a consistent 10-15 seconds.

The abnormally long sync time does not appear to take place on routine periodic datasource sync or on datasource add (surprising). Selected log follows:

2017-01-23 17:19:59.359 DEBUG congress.dse2.datasource_manager [-] Deleting nova2 datasource from (pid=84546) delete_datasource /opt/stack/congress/congress/dse2/datasource_manager.py:103
2017-01-23 17:19:59.364 DEBUG oslo_concurrency.lockutils [-] Lock "congress_synchronize_datasources" acquired by "congress.synchronizer.datasource_synchronizer.sync_datasource" :: waited 0.000s from (pid=84546) inner /usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:270

[a lot of other things happen. see attached log file for details]

2017-01-23 17:20:11.002 INFO congress.synchronizer.datasource_synchronizer [-] service nova2 unregistered by synchronizer
2017-01-23 17:20:11.002 DEBUG oslo_concurrency.lockutils [-] Lock "congress_synchronize_datasources" released by "congress.synchronizer.datasource_synchronizer.sync_datasource" :: held 11.638s from (pid=84546) inner /usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:282