When using the mysql, the same high concurrency results in a different problem (pasted from http://logs.openstack.org/67/591367/4/check/nova-next/f8892df/logs/screen-placement-api.txt.gz#_Aug_13_23_11_22_561724 )

Aug 13 23:11:22.553969 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: DEBUG nova.api.openstack.placement.requestlog [None req-b524d09f-0bae-4855-9e3b-776abe7c6c2f None None] Starting request: 198.72.124.103 "PUT /placement/resource_providers/381ee627-4cc6-4dee-88a1-4f442de6e553/aggregates" {{(pid=634) __call__ /opt/stack/new/nova/nova/api/openstack/placement/requestlog.py:38}}
Aug 13 23:11:22.561724 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap [None req-216bb3ea-5d1e-40af-a7f6-d8f27dbfbc28 None None] Placement API unexpected error: (pymysql.err.OperationalError) (1213, u'Deadlock found when trying to get lock; try restarting transaction') [SQL: u'INSERT INTO resource_provider_aggregates (resource_provider_id, aggregate_id, created_at) SELECT 93, placement_aggregates.id, %(created_at)s AS anon_1 \nFROM placement_aggregates \nWHERE placement_aggregates.uuid IN (%(uuid_1)s, %(uuid_2)s)'] [parameters: {u'uuid_2': u'66d98e7c-3c25-485d-a0dc-1cea651884de', 'created_at': datetime.datetime(2018, 8, 13, 23, 11, 22, 548464), u'uuid_1': u'14a5c8a3-5a99-4e8f-88be-00d85fcb1c17'}] (Background on this error at: http://sqlalche.me/e/e3q8): DBDeadlock: (pymysql.err.OperationalError) (1213, u'Deadlock found when trying to get lock; try restarting transaction') [SQL: u'INSERT INTO resource_provider_aggregates (resource_provider_id, aggregate_id, created_at) SELECT 93, placement_aggregates.id, %(created_at)s AS anon_1 \nFROM placement_aggregates \nWHERE placement_aggregates.uuid IN (%(uuid_1)s, %(uuid_2)s)'] [parameters: {u'uuid_2': u'66d98e7c-3c25-485d-a0dc-1cea651884de', 'created_at': datetime.datetime(2018, 8, 13, 23, 11, 22, 548464), u'uuid_1': u'14a5c8a3-5a99-4e8f-88be-00d85fcb1c17'}] (Background on this error at: http://sqlalche.me/e/e3q8)
Aug 13 23:11:22.562429 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap Traceback (most recent call last):
Aug 13 23:11:22.562605 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap   File "/opt/stack/new/nova/nova/api/openstack/placement/fault_wrap.py", line 40, in __call__
Aug 13 23:11:22.562773 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap     return self.application(environ, start_response)
Aug 13 23:11:22.562959 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap   File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 129, in __call__
Aug 13 23:11:22.563123 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap     resp = self.call_func(req, *args, **kw)
Aug 13 23:11:22.563297 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap   File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 193, in call_func
Aug 13 23:11:22.563460 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap     return self.func(req, *args, **kwargs)
Aug 13 23:11:22.563635 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap   File "/usr/local/lib/python2.7/dist-packages/microversion_parse/middleware.py", line 80, in __call__
Aug 13 23:11:22.563797 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap     response = req.get_response(self.application)
Aug 13 23:11:22.563960 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap   File "/usr/local/lib/python2.7/dist-packages/webob/request.py", line 1313, in send
Aug 13 23:11:22.564128 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap     application, catch_exc_info=False)
Aug 13 23:11:22.564315 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap   File "/usr/local/lib/python2.7/dist-packages/webob/request.py", line 1277, in call_application
Aug 13 23:11:22.564477 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap     app_iter = application(self.environ, start_response)
Aug 13 23:11:22.564634 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap   File "/opt/stack/new/nova/nova/api/openstack/placement/handler.py", line 209, in __call__
Aug 13 23:11:22.564795 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap     return dispatch(environ, start_response, self._map)
Aug 13 23:11:22.564979 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap   File "/opt/stack/new/nova/nova/api/openstack/placement/handler.py", line 146, in dispatch
Aug 13 23:11:22.565142 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap     return handler(environ, start_response)
Aug 13 23:11:22.565303 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap   File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 129, in __call__
Aug 13 23:11:22.565467 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap     resp = self.call_func(req, *args, **kw)
Aug 13 23:11:22.565640 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap   File "/opt/stack/new/nova/nova/api/openstack/placement/wsgi_wrapper.py", line 29, in call_func
Aug 13 23:11:22.565814 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap     super(PlacementWsgify, self).call_func(req, *args, **kwargs)
Aug 13 23:11:22.565977 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap   File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 193, in call_func
Aug 13 23:11:22.566139 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap     return self.func(req, *args, **kwargs)
Aug 13 23:11:22.566301 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap   File "/opt/stack/new/nova/nova/api/openstack/placement/util.py", line 191, in decorated_function
Aug 13 23:11:22.566462 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap     return f(req)
Aug 13 23:11:22.566623 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap   File "/opt/stack/new/nova/nova/api/openstack/placement/microversion.py", line 164, in decorated_func
Aug 13 23:11:22.566785 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap     return _find_method(f, version, status_code)(req, *args, **kwargs)
Aug 13 23:11:22.567030 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap   File "/opt/stack/new/nova/nova/api/openstack/placement/handlers/aggregate.py", line 131, in set_aggregates
Aug 13 23:11:22.567196 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap     increment_generation=consider_generation)
Aug 13 23:11:22.567367 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap   File "/opt/stack/new/nova/nova/api/openstack/placement/handlers/aggregate.py", line 72, in _set_aggregates
Aug 13 23:11:22.567530 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap     aggregate_uuids, increment_generation=increment_generation)
Aug 13 23:11:22.567692 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap   File "/opt/stack/new/nova/nova/api/openstack/placement/objects/resource_provider.py", line 954, in set_aggregates
Aug 13 23:11:22.567854 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap     increment_generation=increment_generation)
Aug 13 23:11:22.568017 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap   File "/usr/local/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 993, in wrapper
Aug 13 23:11:22.568195 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap     return fn(*args, **kwargs)
Aug 13 23:11:22.568371 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap   File "/opt/stack/new/nova/nova/api/openstack/placement/objects/resource_provider.py", line 545, in _set_aggregates
Aug 13 23:11:22.568533 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap     context.session.execute(insert_aggregates)
Aug 13 23:11:22.568696 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 1176, in execute
Aug 13 23:11:22.568880 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap     bind, close_with_result=True).execute(clause, params or {})
Aug 13 23:11:22.569045 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 948, in execute
Aug 13 23:11:22.569207 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap     return meth(self, multiparams, params)
Aug 13 23:11:22.569383 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py", line 269, in _execute_on_connection
Aug 13 23:11:22.569546 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap     return connection._execute_clauseelement(self, multiparams, params)
Aug 13 23:11:22.569708 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1060, in _execute_clauseelement
Aug 13 23:11:22.569899 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap     compiled_sql, distilled_params
Aug 13 23:11:22.570257 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1200, in _execute_context
Aug 13 23:11:22.570423 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap     context)
Aug 13 23:11:22.570585 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1409, in _handle_dbapi_exception
Aug 13 23:11:22.570746 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap     util.raise_from_cause(newraise, exc_info)
Aug 13 23:11:22.570927 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 265, in raise_from_cause
Aug 13 23:11:22.571091 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap     reraise(type(exception), exception, tb=exc_tb, cause=cause)
Aug 13 23:11:22.571252 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
Aug 13 23:11:22.571415 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap     context)
Aug 13 23:11:22.571586 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 509, in do_execute
Aug 13 23:11:22.571749 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap     cursor.execute(statement, parameters)
Aug 13 23:11:22.571911 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap   File "/usr/local/lib/python2.7/dist-packages/pymysql/cursors.py", line 170, in execute
Aug 13 23:11:22.572072 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap     result = self._query(query)
Aug 13 23:11:22.572249 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap   File "/usr/local/lib/python2.7/dist-packages/pymysql/cursors.py", line 328, in _query
Aug 13 23:11:22.572412 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap     conn.query(q)
Aug 13 23:11:22.572574 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap   File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 516, in query
Aug 13 23:11:22.572735 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap     self._affected_rows = self._read_query_result(unbuffered=unbuffered)
Aug 13 23:11:22.572906 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap   File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 727, in _read_query_result
Aug 13 23:11:22.573068 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap     result.read()
Aug 13 23:11:22.573232 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap   File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 1066, in read
Aug 13 23:11:22.573407 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap     first_packet = self.connection._read_packet()
Aug 13 23:11:22.573584 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap   File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 683, in _read_packet
Aug 13 23:11:22.573747 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap     packet.check_error()
Aug 13 23:11:22.573909 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap   File "/usr/local/lib/python2.7/dist-packages/pymysql/protocol.py", line 220, in check_error
Aug 13 23:11:22.574071 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap     err.raise_mysql_exception(self._data)
Aug 13 23:11:22.574232 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap   File "/usr/local/lib/python2.7/dist-packages/pymysql/err.py", line 109, in raise_mysql_exception
Aug 13 23:11:22.574394 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap     raise errorclass(errno, errval)
Aug 13 23:11:22.574557 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap DBDeadlock: (pymysql.err.OperationalError) (1213, u'Deadlock found when trying to get lock; try restarting transaction') [SQL: u'INSERT INTO resource_provider_aggregates (resource_provider_id, aggregate_id, created_at) SELECT 93, placement_aggregates.id, %(created_at)s AS anon_1 \nFROM placement_aggregates \nWHERE placement_aggregates.uuid IN (%(uuid_1)s, %(uuid_2)s)'] [parameters: {u'uuid_2': u'66d98e7c-3c25-485d-a0dc-1cea651884de', 'created_at': datetime.datetime(2018, 8, 13, 23, 11, 22, 548464), u'uuid_1': u'14a5c8a3-5a99-4e8f-88be-00d85fcb1c17'}] (Background on this error at: http://sqlalche.me/e/e3q8)
Aug 13 23:11:22.575164 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: ERROR nova.api.openstack.placement.fault_wrap 
Aug 13 23:11:22.575328 ubuntu-xenial-inap-mtl01-0001304255 <email address hidden>[628]: INFO nova.api.openstack.placement.requestlog [None req-216bb3ea-5d1e-40af-a7f6-d8f27dbfbc28 None None] 198.72.124.103 "PUT /placement/resource_providers/6417c672-91f8-42b7-8adc-2dbbf50e44de/aggregates" status: 500 len: 863 microversion: 1.29