Lock wait timeout in update VIP status

Bug #1298355 reported by Salvatore Orlando
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
neutron
Invalid
Medium
Salvatore Orlando

Bug Description

Please note that this bug is similar to bug 1253822. Possibly included in it (but that one is now closed), but surely not a duplicate.

The root cause is a lock wait timeout occurring while updating VIPs.
given the nature of load balancing tests, this is non critical in 90% of cases, meaning the job succeeds anyway.

But this does not mean that it's not a bug when the job does not fail.

Occurences past 7 days: 141 (15 fails)

logstash queries:
http://logstash.openstack.org/#eyJzZWFyY2giOiJ0YWdzOlwic2NyZWVuLXEtc3ZjLnR4dFwiIEFORCBtZXNzYWdlOlwiTG9jayB3YWl0IHRpbWVvdXQgZXhjZWVkZWRcIiBBTkQgbWVzc2FnZTpcIlVQREFURSB2aXBzIFNFVCBzdGF0dXNcIiBBTkQgbWVzc2FnZTpcIlJldHVybmluZyBleGNlcHRpb24gKE9wZXJhdGlvbmFsRXJyb3IpXCIiLCJmaWVsZHMiOltdLCJvZmZzZXQiOjAsInRpbWVmcmFtZSI6IjYwNDgwMCIsImdyYXBobW9kZSI6ImNvdW50IiwidGltZSI6eyJ1c2VyX2ludGVydmFsIjowfSwic3RhbXAiOjEzOTU5MjQxMDg0NTksIm1vZGUiOiIiLCJhbmFseXplX2ZpZWxkIjoiIn0=

Tags: lbaas
Changed in neutron:
assignee: nobody → Salvatore Orlando (salvatore-orlando)
tags: added: neutron-rc-potential
tags: removed: neutron-rc-potential
Revision history for this message
Salvatore Orlando (salvatore-orlando) wrote :

with 110 hits in the past 7 days, "update vips" is now accounting for about 20% of all "lock wait timeout" errors in neutron.
Overall failure rate for the jobs where this error is hit is 20.9%

The error however occurred only 9 times in gate queue and caused no failure.
Hits are evenly distributed among patches, so hits related to "bad patches" might be ruled out.

Changed in neutron:
importance: Undecided → High
status: New → Triaged
tags: added: lbaas
Revision history for this message
Elena Ezhova (eezhova) wrote :

This bug seems to be interconnected with https://bugs.launchpad.net/neutron/+bug/1312964
The failure happens when one test tries to delete VIP and its port while some other process tries to update the status of this port and both of them get semaphore "db-access" lock. [1]

The bug may be fixed by https://review.openstack.org/#/c/100934/8

http://logs.openstack.org/13/92013/3/check/check-tempest-dsvm-neutron-2/dc8b822/logs/screen-q-svc.txt.gz?#_2014-06-30_11_40_48_293

Changed in neutron:
importance: High → Medium
Revision history for this message
Eugene Nikanorov (enikanorov) wrote :

Not seeing in gate for quite a long time. Hopefully it was fixed by some other commit.
Setting as Incomplete.

Let's do nothing until it's expires.

Changed in neutron:
status: Triaged → Incomplete
Revision history for this message
Joe Gordon (jogo) wrote :

The expiration timer hasn't started for some reason.

Changed in neutron:
status: Incomplete → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.