DB deadlock when bulk delete subnet

[shihanzhang]

My environment is stable/newton, I use rally to do stability tests, the details is below:
1. config a network with 2 dhcp-agent
2. config rally to do 40 bulk operation, each process is:create network->create subnet->delete subnet->delete network

I total do 100 times, the success rate is 1.6%, i check the neutron-server log with many db deadlock erros, there were
5 tables happen deadlock:ipamallocations,ipamsubnets,standardattributes,ports,provisioningblocks.
the neutron-server log is http://paste.openstack.org/show/613359/

[Brian Haley (brian-haley)]

Is it possible for you to retry your testing on the master (Pike) branch? It would be good to know if it was broken there, or if there is simply a backport to be made to stable/newton. Thanks.

[shihanzhang (shihanzhang)]

hi Brian Haley, thanks for your quikly respose, I checked the mysql logs,find the below error:

*** (1) TRANSACTION:
TRANSACTION 102395507, ACTIVE 0 sec starting index read
mysql tables in use 1, locked 1
LOCK WAIT 4 lock struct(s), heap size 1184, 2 row lock(s), undo log entries 1
MySQL thread id 3154249, OS thread handle 0x7fdf267b6b00, query id 118985056 10.127.2.62 neutron updating
UPDATE ports SET status='BUILD' WHERE ports.id = '942d25e5-d71d-4492-b3ec-61a82bbce26c'
*** (1) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 360 page no 11 n bits 120 index `PRIMARY` of table `neutron`.`ports` trx table locks 2 total table locks 3 trx id 102395507 lock_mode X locks rec but not gap waiting lock hold time 0 wait time before grant 0
*** (2) TRANSACTION:
TRANSACTION 102395498, ACTIVE 0 sec updating or deleting
mysql tables in use 1, locked 1
73 lock struct(s), heap size 13864, 40 row lock(s), undo log entries 4
MySQL thread id 3155807, OS thread handle 0x7fdf26894b00, query id 118985101 10.127.2.62 neutron updating
DELETE FROM ports WHERE ports.id = '942d25e5-d71d-4492-b3ec-61a82bbce26c'
*** (2) HOLDS THE LOCK(S):
RECORD LOCKS space id 360 page no 11 n bits 120 index `PRIMARY` of table `neutron`.`ports` trx table locks 33 total table locks 3 trx id 102395498 lock_mode X locks rec but not gap lock hold time 0 wait time before grant 0
*** (2) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 387 page no 8 n bits 104 index `PRIMARY` of table `neutron`.`standardattributes` trx table locks 33 total table locks 2 trx id 102395498 lock mode S locks rec but not gap waiting lock hold time 0 wait time before grant 0
*** WE ROLL BACK TRANSACTION (1)

[Zachary Ma (mazengxie)]

can you find the reason ?

[Rodolfo Alonso (rodolfo-alonso-hernandez)]

Bug closed due to lack of activity, please feel free to reopen if needed.