Can not delete listener from a loadbalancer in ERROR status

[li,chen]

For some reason, the loadbalancer is in ACTIVE in neutron database, but ERROR in Octavia database.

neutron lbaas-loadbalancer-list
+-------------------------+-------------------------+-------------+---------------------+----------+
| id | name | vip_address | provisioning_status | provider |
+-------------------------+-------------------------+-------------+---------------------+----------+
| 65b0e95d-41f0-4a90-9120 | bdd_loadbalancer_2016-0 | 192.168.2.3 | ACTIVE | octavia |
| -33e31e76eaf6 | 8-15-02-02-04 | | | |
+-------------------------+-------------------------+-------------+---------------------+----------+

MariaDB [octavia]> select * from load_balancer;
+----------------------------------+--------------------------------------+--------------------------------------+-------------+---------------------+------------------+---------+----------------+-----------------+---------------------+---------------------+
| project_id | id | name | description | provisioning_status | operating_status | enabled | topology | server_group_id | created_at | updated_at |
+----------------------------------+--------------------------------------+--------------------------------------+-------------+---------------------+------------------+---------+----------------+-----------------+---------------------+---------------------+
| 438c5c3a808048248785d5fe8b34f1f2 | 65b0e95d-41f0-4a90-9120-33e31e76eaf6 | bdd_loadbalancer_2016-08-15-02-02-04 | | ERROR | OFFLINE | 1 | ACTIVE_STANDBY | NULL | 2016-08-15 07:02:07 | 2016-08-15 07:18:15 |
+----------------------------------+--------------------------------------+--------------------------------------+-------------+---------------------+------------------+---------+----------------+-----------------+---------------------+---------------------+
1 row in set (0.00 sec)

Then we try to delete the loadbalancer:

neutron lbaas-loadbalancer-delete 65b0e95d-41f0-4a90-9120-33e31e76eaf6
listener d22f4df4-785f-4f59-9e66-c57cc08f5abc is using this loadbalancer
Neutron server returns request_ids: ['req-c2379ee1-a84b-4058-bebe-56c5e9031832']

Then we try to delete the listener:

neutron lbaas-listener-delete d22f4df4-785f-4f59-9e66-c57cc08f5abc
Deleted listener: d22f4df4-785f-4f59-9e66-c57cc08f5abc

List the listener again, the listener is still there:

neutron lbaas-listener-list
+-------------------+-------------------+-------------------+----------+---------------+----------------+
| id | default_pool_id | name | protocol | protocol_port | admin_state_up |
+-------------------+-------------------+-------------------+----------+---------------+----------------+
| d22f4df4-785f-4f5 | 1cb324d1-6fe4-4c2 | bdd_listener_2016 | TCP | 80 | True |
| 9-9e66-c57cc08f5a | c-8146-6c3bf926b9 | -08-15-02-04-48 | | | |
| bc | 1d | | | | |
+-------------------+-------------------+-------------------+----------+---------------+----------------+

In octavia-api log, we get:

2016-08-17 00:03:12.383 18654 INFO octavia.api.v1.controllers.listener [req-08d794cf-8755-4d41-8d56-9f1d248fa862 - - - - -] Load Balancer 65b0e95d-41f0-4a90-9120-33e31e76eaf6 is immutable.
2016-08-17 00:03:12.419 18654 DEBUG wsme.api [req-08d794cf-8755-4d41-8d56-9f1d248fa862 - - - - -] Client-side error: Load Balancer 65b0e95d-41f0-4a90-9120-33e31e76eaf6 is immutable and cannot be updated. format_exception /usr/lib/python2.7/site-packages/wsme/api.py:221
127.0.0.1 - - [17/Aug/2016 00:03:12] "DELETE /v1/loadbalancers/65b0e95d-41f0-4a90-9120-33e31e76eaf6/listeners/d22f4df4-785f-4f59-9e66-c57cc08f5abc HTTP/1.1" 409 147
127.0.0.1 - - [17/Aug/2016 00:03:12] "GET /v1/loadbalancers/65b0e95d-41f0-4a90-9120-33e31e76eaf6 HTTP/1.1" 200 459

[li,chen (chen-li)]

I don't think this is duplicate of Bug #1496646.

Because:
When we try to delete a listener, it will try to change the loadbalancer status to 'PENDING_UPDATE'.
An 'ERROR' loadbalancer can not be updated to 'PENDING_UPDATE'.
So, the listener can not be deleted.

This is not related to the db mismatch between neutron and octavia.

[Michael Johnson (johnsom)]

Ok, I have looked into this deeper and you are right this is still an issue and not related to the db mismatch as stated in the first line.

[Michael Johnson (johnsom)]

Patch https://review.openstack.org/#/c/372791 resolves this issue.

[Adam Harwell (adam-harwell)]

https://review.openstack.org/#/c/372791 merged quite some time ago. Can we mark this as resolved?

[He Qing (tsinghe-7)]

I can confirm this problem is still exist. Patch https://review.openstack.org/#/c/372791 did not fix this.That patch just set Objects to ERROR while reverting.
But the problem here is When LB is in ERROR state, it is not allowed to delete it's listeners(Or pools,members...).

When LB is in ERROR state, if we try to delete the LB with cascade API, it would be allowed. But if we try to delete objects belongs to the LB, we need to change the LB's state to PENDING_UPDATE first, which is not allowed.

[Michael Johnson (johnsom)]

Yeah, I see it. Some of the API controllers are not properly checking the target state. This needs a refactor for sure.

[Michael Johnson (johnsom)]

Just a note: Workaround is to use the cascade delete on the load balancer.

[alpha23 (alpha23)]

Cascade delete (--cascade) does not resolve the issue. Loadbalancer provisioning into PENDING_DELETE state and then reverts to ERROR state. Confirmed with Pike with octavia

[Michael Johnson (johnsom)]

All Octavia bugs have been moved to Storyboard. All OpenStack projects are moving from launchpad to Storyboard. This bug is considered closed in Launchpad and will not be updated.
Please see the current bug here:
https://storyboard.openstack.org/#!/story/1613956

[Gregory Thiemonge (gthiemonge)]

Abandoned after re-enabling the Octavia launchpad.