DBDuplicateEntry: (IntegrityError) duplicate key value violates unique constraint "ipavailabilityranges_pkey"

[Rossella Sblendido]

It affects only postgresql
To reproduce this bug:

1) Create a network with few IPs (eg /29)
2) Create 2 VMs that use the /29 network.
3) Destroy the 2 VMs
4) From horizon create a number of VMs > IP available (eg. 8)

When there's no IP available, _rebuild_availability_ranges will be called to recycle the IPs of the VMs that no longer exist (the ones that we created at step 2 ) . From the logs i see that when there's a bulk creation of ports and the IP range is over, 2 or more _rebuild_availability_ranges are triggered at the same time by different port creation operation. This leads to the DBDuplicateEntry, since the operation that is performed last, will try to insert in the DB stale data.

See log:

 362399 2014-08-26 10:01:01.926 27821 DEBUG neutron.db.db_base_plugin_v2 [-] All IPs from subnet a77b383d-e881-49c1-8143-910ec46fe42a (10.238.192.0 /18) allocated _try_generate_ip /usr/lib64/python2.6/site-packages/neutron/db/db_base_plugin_v2.py:384
 362400 2014-08-26 10:01:01.927 27821 DEBUG neutron.db.db_base_plugin_v2 [-] Rebuilding availability ranges for subnet {'allocation_pools': [{'star t': u'10.238.200.129', 'end': u'10.238.200.254'}], 'host_routes': [], 'cidr': u'10.238.192.0/18', 'id': u'a77b383d-e881-49c1-8143-910ec46fe 42a', 'name': u'floating', 'enable_dhcp': False, 'network_id': u'a4f3c5ac-de4a-44c5-94b8-bd07a14c4d1a', 'tenant_id': u'774289027a8441babaed f5774e49e971', 'dns_nameservers': [], 'gateway_ip': u'10.238.192.3', 'ip_version': 4, 'shared': False} _rebuild_availability_ranges /usr/li b64/python2.6/site-packages/neutron/db/db_base_plugin_v2.py:414
 362401 2014-08-26 10:01:01.928 27821 DEBUG neutron.db.db_base_plugin_v2 [-] Generated mac for network a4f3c5ac-de4a-44c5-94b8-bd07a14c4d1a is fa:1 6:3e:6f:f3:02 _generate_mac /usr/lib64/python2.6/site-packages/neutron/db/db_base_plugin_v2.py:305
 362402 2014-08-26 10:01:01.952 27821 INFO neutron.wsgi [-] 10.1.100.1 - - [26/Aug/2014 10:01:01] "GET /v2.0/subnets.json?id=fa32a700-57ce-4bfc-a2b f-7fb19e20f81b HTTP/1.1" 200 504 0.091873
 362403
 362404 2014-08-26 10:01:01.966 27821 INFO neutron.wsgi [-] (27821) accepted ('10.1.100.1', 21561)
 362405
 362406 2014-08-26 10:01:01.977 27821 DEBUG neutron.db.db_base_plugin_v2 [-] All IPs from subnet a77b383d-e881-49c1-8143-910ec46fe42a (10.238.192.0 /18) allocated _try_generate_ip /usr/lib64/python2.6/site-packages/neutron/db/db_base_plugin_v2.py:384
 362407 2014-08-26 10:01:01.977 27821 DEBUG neutron.db.db_base_plugin_v2 [-] Rebuilding availability ranges for subnet {'allocation_pools': [{'star t': u'10.238.200.129', 'end': u'10.238.200.254'}], 'host_routes': [], 'cidr': u'10.238.192.0/18', 'id': u'a77b383d-e881-49c1-8143-910ec46fe 42a', 'name': u'floating', 'enable_dhcp': False, 'network_id': u'a4f3c5ac-de4a-44c5-94b8-bd07a14c4d1a', 'tenant_id': u'774289027a8441babaed f5774e49e971', 'dns_nameservers': [], 'gateway_ip': u'10.238.192.3', 'ip_version': 4, 'shared': False} _rebuild_availability_ranges /usr/lib64/python2.6/site-packages/neutron/db/db_base_plugin_v2.py:414

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/128709

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/128710

[Rossella Sblendido (rossella-o)]

I tested this again, this bug appear only in havana. Since havana is EOF I think this can be closed

[OpenStack Infra (hudson-openstack)]

Change abandoned by Rossella Sblendido (<email address hidden>) on branch: master
Review: https://review.openstack.org/128709
Reason: The bug affects havana but not newer releases. Abandoning this patch since havana is EOF

[OpenStack Infra (hudson-openstack)]

Change abandoned by Rossella Sblendido (<email address hidden>) on branch: master
Review: https://review.openstack.org/128710
Reason: This patch doesn't solve the bug. To give more context, postgresql throws an error if select for update is applied when an outer join returns an empty row (this happens when there's no correspondence for that row in one of the 2 tables involved in the join). See "SELECT FOR UPDATE/SHARE cannot be applied to the nullable side of an outer join " http://<email address hidden>
This is not the case here, since IPAvailabilityRange has allocation_pool_id as foreign key, so there's a 1 to 1 correspondence for every row

[Rossella Sblendido (rossella-o)]

This bug affects only Postgresql on Havana, I wasn't able to reproduce it on newer releases. Since havana is EOF, I guess this bug can be closed