Description of problem:
=======================
An attempt to delete a loadbalancer with no listener, pool or members attached to it fails with an error.
How reproducible:
=================
3/3
Steps to Reproduce:
===================
0. Use Devstack with the following local.conf:
[[local|localrc]]
DEST=/opt/openstack
DATA_DIR=$DEST/data
LOGFILE=$DATA_DIR/logs/stack.log
SCREEN_LOGDIR=$DATA_DIR/logs
VERBOSE=False
MYSQL_PASSWORD=1
RABBIT_PASSWORD=1
SERVICE_TOKEN=1
SERVICE_PASSWORD=1
ADMIN_PASSWORD=1
RECLONE=True
disable_service n-net
enable_service q-svc
enable_service q-agt
enable_service q-dhcp
enable_service q-l3
enable_service q-meta
enable_service neutron
enable_service q-lbaasv2
enable_plugin neutron-lbaas git+ssh://localhost:/home/stack/neutron-lbaas
enable_service octavia
enable_service o-api
enable_service o-cw
enable_service o-hk
enable_service o-hm
enable_plugin octavia https://git.openstack.org/openstack/octavia master
[[post-config|/etc/neutron/neutron_lbaas.conf]]
[service_providers]
service_provider = LOADBALANCERV2:Octavia:neutron_lbaas.drivers.octavia.driver.OctaviaDriver:default
HOST_IP=192.168.6.61
Note: The reason for the local neutron-lbaas is that in addition to it's master branch I cherry-picked: https://review.openstack.org/#/c/174114/
1. Create a loadbalancer:
$ neutron lbaas-loadbalancer-create ba7cfdc0-00ee-4112-9ab3-17a7baf613bb --name test
2. Verify your aphora is ACTIVE
$ nova list --all-tenant
3. Delete the loadbalaner
$ neutron --debug lbaas-loadbalancer-delete a538db76-51d7-47ab-9deb-e9c034365827
Actual results:
===============
neutronclient:
DEBUG: keystoneclient.session REQ: curl -g -i -X DELETE http://192.168.6.61:9696/v2.0/lbaas/loadbalancers/a538db76-51d7-47ab-9deb-e9c034365827.json -H "User-Agent: python-neutronclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}bdd693761432aa021659322bf4ee70fce43d8fcc"
DEBUG: keystoneclient.session RESP: [500] date: Thu, 25 Jun 2015 10:17:49 GMT connection: keep-alive content-type: application/json; charset=UTF-8 content-length: 101 x-openstack-request-id: req-ddf86912-f98a-4e2a-9a81-f669f889bd64
RESP BODY: {"NeutronError": {"message": "An error happened in the driver", "type": "DriverError", "detail": ""}}
DEBUG: neutronclient.v2_0.client Error message: {"NeutronError": {"message": "An error happened in the driver", "type": "DriverError", "detail": ""}}
ERROR: neutronclient.shell An error happened in the driver
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/neutronclient/shell.py", line 781, in run_subcommand
return run_command(cmd, cmd_parser, sub_argv)
File "/usr/lib/python2.7/site-packages/neutronclient/shell.py", line 106, in run_command
return cmd.run(known_args)
File "/usr/lib/python2.7/site-packages/neutronclient/neutron/v2_0/__init__.py", line 585, in run
obj_deleter(_id)
File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 102, in with_params
ret = self.function(instance, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 904, in delete_loadbalancer
(lbaas_loadbalancer))
File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 289, in delete
headers=headers, params=params)
File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 270, in retry_request
headers=headers, params=params)
File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 211, in do_request
self._handle_fault_response(status_code, replybody)
File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 185, in _handle_fault_response
exception_handler_v20(status_code, des_error_body)
File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 70, in exception_handler_v20
status_code=status_code)
InternalServerError: An error happened in the driver
q-svc:
2015-06-25 06:17:49.279 INFO neutron_lbaas.services.loadbalancer.plugin [req-ddf86912-f98a-4e2a-9a81-f669f889bd64 admin 91119519f30442c4a4f6ce0ca4047a05] Calling driver operation LoadBalancerManager.delete
2015-06-25 06:17:49.279 ERROR neutron_lbaas.drivers.octavia.driver [req-ddf86912-f98a-4e2a-9a81-f669f889bd64 admin 91119519f30442c4a4f6ce0ca4047a05] url = http://127.0.0.1:9876/v1/loadbalancers/a538db76-51d7-47ab-9deb-e9c034365827
2015-06-25 06:17:49.280 ERROR neutron_lbaas.drivers.octavia.driver [req-ddf86912-f98a-4e2a-9a81-f669f889bd64 admin 91119519f30442c4a4f6ce0ca4047a05] args = None
2015-06-25 06:17:49.300 ERROR neutron_lbaas.drivers.octavia.driver [req-ddf86912-f98a-4e2a-9a81-f669f889bd64 admin 91119519f30442c4a4f6ce0ca4047a05] r = <Response [202]>
2015-06-25 06:17:49.312 DEBUG keystoneclient.session [req-4647d7f2-cff2-43c0-95d5-393f04378771 admin 771cf81118f8474587236cff34cff67d] REQ: curl -g -i --cacert "/opt/openstack/data/ca-bundle.pem" -X GET http://10.35.6.61:35357/v3/auth/tok
ens -H "X-Subject-Token: {SHA1}855d53ce953a93b88f8395b3b143c256d4f572d5" -H "User-Agent: python-keystoneclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}49513fda5c7e84b9725e957b1d343f6f53a01993" from (pid=5682) _http_log_requ
est /usr/lib/python2.7/site-packages/keystoneclient/session.py:195
2015-06-25 06:17:49.316 ERROR neutron_lbaas.services.loadbalancer.plugin [req-ddf86912-f98a-4e2a-9a81-f669f889bd64 admin 91119519f30442c4a4f6ce0ca4047a05] There was an error in the driver
2015-06-25 06:17:49.316 TRACE neutron_lbaas.services.loadbalancer.plugin Traceback (most recent call last):
2015-06-25 06:17:49.316 TRACE neutron_lbaas.services.loadbalancer.plugin File "/opt/openstack/neutron-lbaas/neutron_lbaas/services/loadbalancer/plugin.py", line 463, in _call_driver_operation
2015-06-25 06:17:49.316 TRACE neutron_lbaas.services.loadbalancer.plugin driver_method(context, db_entity)
2015-06-25 06:17:49.316 TRACE neutron_lbaas.services.loadbalancer.plugin File "/opt/openstack/neutron-lbaas/neutron_lbaas/drivers/driver_base.py", line 111, in func_wrapper
2015-06-25 06:17:49.316 TRACE neutron_lbaas.services.loadbalancer.plugin raise e
2015-06-25 06:17:49.316 TRACE neutron_lbaas.services.loadbalancer.plugin JSONDecodeError: Expecting value: line 1 column 1 (char 0)
2015-06-25 06:17:49.316 TRACE neutron_lbaas.services.loadbalancer.plugin
2015-06-25 06:17:49.322 ERROR neutron.api.v2.resource [req-ddf86912-f98a-4e2a-9a81-f669f889bd64 admin 91119519f30442c4a4f6ce0ca4047a05] delete failed
2015-06-25 06:17:49.322 TRACE neutron.api.v2.resource Traceback (most recent call last):
2015-06-25 06:17:49.322 TRACE neutron.api.v2.resource File "/opt/openstack/neutron/neutron/api/v2/resource.py", line 83, in resource
2015-06-25 06:17:49.322 TRACE neutron.api.v2.resource result = method(request=request, **args)
2015-06-25 06:17:49.322 TRACE neutron.api.v2.resource File "/opt/openstack/neutron/neutron/api/v2/base.py", line 491, in delete
2015-06-25 06:17:49.322 TRACE neutron.api.v2.resource obj_deleter(request.context, id, **kwargs)
2015-06-25 06:17:49.322 TRACE neutron.api.v2.resource File "/opt/openstack/neutron-lbaas/neutron_lbaas/services/loadbalancer/plugin.py", line 545, in delete_loadbalancer
2015-06-25 06:17:49.322 TRACE neutron.api.v2.resource context, driver.load_balancer.delete, db_lb)
2015-06-25 06:17:49.322 TRACE neutron.api.v2.resource File "/opt/openstack/neutron-lbaas/neutron_lbaas/services/loadbalancer/plugin.py", line 471, in _call_driver_operation
2015-06-25 06:17:49.322 TRACE neutron.api.v2.resource raise loadbalancerv2.DriverError()
2015-06-25 06:17:49.322 TRACE neutron.api.v2.resource DriverError: An error happened in the driver
In addition it notes:
RESP BODY: {"itemNotFound": {"message": "No instances found for any event", "code": 404}}
from (pid=5682) _http_log_response /usr/lib/python2.7/site-packages/keystoneclient/session.py:224
2015-06-25 06:17:51.865 WARNING neutron.notifiers.nova [-] Nova returned NotFound for event: [{'name': 'network-changed', 'server_uuid': u'a4a98717-8fff-4743-858c-386e6360c866'}]
Note: a4a98717-8fff-4743-858c-386e6360c866 used to be the amphora UUID, Which at this point not longer exists as it manage to delete it.
At that point the provisioning_status status of the loadbalancer switched from ACTIVE to an ERROR state.
Expected results:
=================
The lbaas-loadbalancer-delete should successfully delete the loadbalancer and it's respective amphora.
Additional Info:
================
1. A second attept to delete the very same loadbalancer succeed.
Same happens when attepting to delete a listener:
2015-06-25 06:50:42.441 ERROR neutron_
rs/7c3c2835-
2015-06-25 06:50:42.441 ERROR neutron_
2015-06-25 06:50:42.492 ERROR neutron_
2015-06-25 06:50:42.509 DEBUG neutron.wsgi [req-212e371f-
n/neutron/
2015-06-25 06:50:42.510 INFO neutron.wsgi [req-212e371f-
2015-06-25 06:50:42.549 ERROR neutron_
2015-06-25 06:50:42.549 TRACE neutron_
2015-06-25 06:50:42.549 TRACE neutron_
2015-06-25 06:50:42.549 TRACE neutron_
2015-06-25 06:50:42.549 TRACE neutron_
2015-06-25 06:50:42.549 TRACE neutron_
2015-06-25 06:50:42.549 TRACE neutron_
2015-06-25 06:50:42.549 TRACE neutron_
2015-06-25 06:50:42.556 ERROR neutron.
2015-06-25 06:50:42.556 TRACE neutron.
2015-06-25 06:50:42.556 TRACE neutron.
2015-06-25 06:50:42.556 TRACE neutron.
2015-06-25 06:50:42.556 TRACE neutron.
2015-06-25 06:50:42.556 TRACE neutron.
2015-06-25 06:50:42.556 TRACE neutron.
2015-06-25 06:50:42.556 TRACE neutron.
2015-06-25 06:50:4...