Neutron returns HTTP500 when deleting subnet on the gate

[Ken'ichi Ohmichi]

test_dhcpv6_64_subnets of gate-tempest-dsvm-neutron-full-ubuntu-trusty-liberty failed with an internal server error:

http://logs.openstack.org/39/351939/3/check/gate-tempest-dsvm-neutron-full-ubuntu-trusty-liberty/a78ddac

Traceback (most recent call last):
  File "tempest/api/network/test_dhcp_ipv6.py", line 249, in test_dhcpv6_64_subnets
    self._clean_network()
  File "tempest/api/network/test_dhcp_ipv6.py", line 81, in _clean_network
    self.subnets_client.delete_subnet(subnet['id'])
  File "tempest/lib/services/network/subnets_client.py", line 49, in delete_subnet
    return self.delete_resource(uri)
  File "tempest/lib/services/network/base.py", line 41, in delete_resource
    resp, body = self.delete(req_uri)
  File "tempest/lib/common/rest_client.py", line 304, in delete
    return self.request('DELETE', url, extra_headers, headers, body)
  File "tempest/lib/common/rest_client.py", line 667, in request
    resp, resp_body)
  File "tempest/lib/common/rest_client.py", line 831, in _error_checker
    message=message)
tempest.lib.exceptions.ServerFault: Got server fault
Details: Request Failed: internal server error while processing your request.

[Ken'ichi Ohmichi (oomichi)]

The log of screen-q-svc.txt.gz

2016-08-11 00:13:42.104 ERROR neutron.api.v2.resource [req-02e15fba-9fcb-43df-bdbb-0a1e7db78136 tempest-NetworksTestDHCPv6-653771437 tempest-NetworksTestDHCPv6-653771437] delete failed
2016-08-11 00:13:42.104 31475 ERROR neutron.api.v2.resource Traceback (most recent call last):
2016-08-11 00:13:42.104 31475 ERROR neutron.api.v2.resource File "/opt/stack/new/neutron/neutron/api/v2/resource.py", line 83, in resource
2016-08-11 00:13:42.104 31475 ERROR neutron.api.v2.resource result = method(request=request, **args)
2016-08-11 00:13:42.104 31475 ERROR neutron.api.v2.resource File "/opt/stack/new/neutron/neutron/api/v2/base.py", line 532, in delete
2016-08-11 00:13:42.104 31475 ERROR neutron.api.v2.resource return self._delete(request, id, **kwargs)
2016-08-11 00:13:42.104 31475 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 146, in wrapper
2016-08-11 00:13:42.104 31475 ERROR neutron.api.v2.resource ectxt.value = e.inner_exc
2016-08-11 00:13:42.104 31475 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 195, in __exit__
2016-08-11 00:13:42.104 31475 ERROR neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb)
2016-08-11 00:13:42.104 31475 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 136, in wrapper
2016-08-11 00:13:42.104 31475 ERROR neutron.api.v2.resource return f(*args, **kwargs)
2016-08-11 00:13:42.104 31475 ERROR neutron.api.v2.resource File "/opt/stack/new/neutron/neutron/api/v2/base.py", line 554, in _delete
2016-08-11 00:13:42.104 31475 ERROR neutron.api.v2.resource obj_deleter(request.context, id, **kwargs)
2016-08-11 00:13:42.104 31475 ERROR neutron.api.v2.resource File "/opt/stack/new/neutron/neutron/plugins/ml2/plugin.py", line 977, in delete_subnet
2016-08-11 00:13:42.104 31475 ERROR neutron.api.v2.resource LOG.debug("Port %s deleted concurrently", a.port_id)
2016-08-11 00:13:42.104 31475 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/attributes.py", line 237, in __get__
2016-08-11 00:13:42.104 31475 ERROR neutron.api.v2.resource return self.impl.get(instance_state(instance), dict_)
2016-08-11 00:13:42.104 31475 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/attributes.py", line 578, in get
2016-08-11 00:13:42.104 31475 ERROR neutron.api.v2.resource value = state._load_expired(state, passive)
2016-08-11 00:13:42.104 31475 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/state.py", line 474, in _load_expired
2016-08-11 00:13:42.104 31475 ERROR neutron.api.v2.resource self.manager.deferred_scalar_loader(self, toload)
2016-08-11 00:13:42.104 31475 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/loading.py", line 669, in load_scalar_attributes
2016-08-11 00:13:42.104 31475 ERROR neutron.api.v2.resource raise orm_exc.ObjectDeletedError(state)
2016-08-11 00:13:42.104 31475 ERROR neutron.api.v2.resource Object...

[Ken'ichi Ohmichi (oomichi)]

This problem has been fixed with the commit https://review.openstack.org/#/c/333601/ on the master.

[Ken'ichi Ohmichi (oomichi)]

Liberty contains this problem on https://github.com/openstack/neutron/blob/stable/liberty/neutron/plugins/ml2/plugin.py#L977

Patches are https://review.openstack.org/#/c/354445/ and https://review.openstack.org/#/c/354444/