NSXv: overlapping IP exception

[Gary Kotton]

Under load the following occurs:

2015-07-14 10:50:42.223 25663 ERROR neutron.api.v2.resource [req-3063ef53-30ff-4390-b86e-95206e1ba11e ] create failed
2015-07-14 10:50:42.223 25663 TRACE neutron.api.v2.resource Traceback (most recent call last):
2015-07-14 10:50:42.223 25663 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/neutron/api/v2/resource.py", line 83, in resource
2015-07-14 10:50:42.223 25663 TRACE neutron.api.v2.resource result = method(request=request, **args)
2015-07-14 10:50:42.223 25663 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/neutron/api/v2/base.py", line 461, in create
2015-07-14 10:50:42.223 25663 TRACE neutron.api.v2.resource obj = obj_creator(request.context, **kwargs)
2015-07-14 10:50:42.223 25663 TRACE neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/vmware_nsx/neutron/plugins/vmware/plugins/nsx_v.py", line 1046, in create_subnet
2015-07-14 10:50:42.223 25663 TRACE neutron.api.v2.resource self.delete_subnet(context, s['id'])
2015-07-14 10:50:42.223 25663 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__
2015-07-14 10:50:42.223 25663 TRACE neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb)
2015-07-14 10:50:42.223 25663 TRACE neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/vmware_nsx/neutron/plugins/vmware/plugins/nsx_v.py", line 1043, in create_subnet
2015-07-14 10:50:42.223 25663 TRACE neutron.api.v2.resource self._update_dhcp_service_with_subnet(context, s)
2015-07-14 10:50:42.223 25663 TRACE neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/vmware_nsx/neutron/plugins/vmware/plugins/nsx_v.py", line 1141, in _update_dhcp_service_with_subnet
2015-07-14 10:50:42.223 25663 TRACE neutron.api.v2.resource subnet['id'])
2015-07-14 10:50:42.223 25663 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__
2015-07-14 10:50:42.223 25663 TRACE neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb)
2015-07-14 10:50:42.223 25663 TRACE neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/vmware_nsx/neutron/plugins/vmware/plugins/nsx_v.py", line 1128, in _update_dhcp_service_with_subnet
2015-07-14 10:50:42.223 25663 TRACE neutron.api.v2.resource context, network_id, address_groups=address_groups)
2015-07-14 10:50:42.223 25663 TRACE neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/vmware_nsx/neutron/plugins/vmware/vshield/edge_utils.py", line 834, in update_dhcp_edge_service
2015-07-14 10:50:42.223 25663 TRACE neutron.api.v2.resource 'tunnel_index': tunnel_index})
2015-07-14 10:50:42.223 25663 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__
2015-07-14 10:50:42.223 25663 TRACE neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb)
2015-07-14 10:50:42.223 25663 TRACE neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/vmware_nsx/neutron/plugins/vmware/vshield/edge_utils.py", line 826, in update_dhcp_edge_service
2015-07-14 10:50:42.223 25663 TRACE neutron.api.v2.resource address_groups)
2015-07-14 10:50:42.223 25663 TRACE neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/vmware_nsx/neutron/plugins/vmware/vshield/edge_utils.py", line 424, in _update_dhcp_internal_interface
2015-07-14 10:50:42.223 25663 TRACE neutron.api.v2.resource address_groups=iface_list)
2015-07-14 10:50:42.223 25663 TRACE neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/vmware_nsx/neutron/plugins/vmware/vshield/edge_appliance_driver.py", line 255, in update_interface
2015-07-14 10:50:42.223 25663 TRACE neutron.api.v2.resource self.vcns.update_interface(edge_id, config)
2015-07-14 10:50:42.223 25663 TRACE neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/vmware_nsx/neutron/plugins/vmware/vshield/vcns.py", line 162, in update_interface
2015-07-14 10:50:42.223 25663 TRACE neutron.api.v2.resource return self.do_request(HTTP_PUT, uri, vnic, decode=True)
2015-07-14 10:50:42.223 25663 TRACE neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/vmware_nsx/neutron/plugins/vmware/vshield/vcns.py", line 101, in do_request
2015-07-14 10:50:42.223 25663 TRACE neutron.api.v2.resource headers, encodeParams)
2015-07-14 10:50:42.223 25663 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/retrying.py", line 68, in wrapped_f
2015-07-14 10:50:42.223 25663 TRACE neutron.api.v2.resource return Retrying(*dargs, **dkw).call(f, *args, **kw)
2015-07-14 10:50:42.223 25663 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/retrying.py", line 223, in call
2015-07-14 10:50:42.223 25663 TRACE neutron.api.v2.resource return attempt.get(self._wrap_exception)
2015-07-14 10:50:42.223 25663 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/retrying.py", line 261, in get
2015-07-14 10:50:42.223 25663 TRACE neutron.api.v2.resource six.reraise(self.value[0], self.value[1], self.value[2])
2015-07-14 10:50:42.223 25663 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/retrying.py", line 217, in call
2015-07-14 10:50:42.223 25663 TRACE neutron.api.v2.resource attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
2015-07-14 10:50:42.223 25663 TRACE neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/vmware_nsx/neutron/plugins/vmware/vshield/vcns.py", line 87, in _client_request
2015-07-14 10:50:42.223 25663 TRACE neutron.api.v2.resource return client(method, uri, params, headers, encodeParams)
2015-07-14 10:50:42.223 25663 TRACE neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/vmware_nsx/neutron/plugins/vmware/vshield/common/VcnsApiClient.py", line 112, in request
2015-07-14 10:50:42.223 25663 TRACE neutron.api.v2.resource raise cls(uri=uri, status=status, header=header, response=response)
2015-07-14 10:50:42.223 25663 TRACE neutron.api.v2.resource RequestBad: Request https://192.168.1.64/api/4.0/edges/edge-49/vnics/1?async=true is Bad, response {"details":"Overlapping IP Addresses are not allowed for different addressGroups. Vnic 10 ip assignments overlaps 9.9.0.2.","errorCode":10041,"moduleName":"vShield Edge"}

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/204474
Committed: https://git.openstack.org/cgit/openstack/vmware-nsx/commit/?id=bc62e7a442fc1bc33d8fbfc6b6bdd2f58331b493
Submitter: Jenkins
Branch: master

commit bc62e7a442fc1bc33d8fbfc6b6bdd2f58331b493
Author: Gary Kotton <email address hidden>
Date: Wed Jul 22 00:28:52 2015 -0700

    NSXv: ensure that update_interface is synchronous

    When multiple updates occur at the same time we need to make sure
    that the edge interface update is atomic.

    Change-Id: Ic2f82cd23473efd41911ea41e2175e2833cc4a30
    Closes-bug: #1476970

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/kilo
Review: https://review.openstack.org/208928

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/208928
Committed: https://git.openstack.org/cgit/openstack/vmware-nsx/commit/?id=e87249e23fee845eb90eef628bf59ba21304f95e
Submitter: Jenkins
Branch: stable/kilo

commit e87249e23fee845eb90eef628bf59ba21304f95e
Author: Gary Kotton <email address hidden>
Date: Wed Jul 22 00:28:52 2015 -0700

    NSXv: ensure that update_interface is synchronous

    When multiple updates occur at the same time we need to make sure
    that the edge interface update is atomic.

    Change-Id: Ic2f82cd23473efd41911ea41e2175e2833cc4a30
    Closes-bug: #1476970
    (cherry picked from commit bc62e7a442fc1bc33d8fbfc6b6bdd2f58331b493)