Loadbalancer provisioning_status is inconsistent between Octavia and Neutron Database during LB create failure

Bug #1606842 reported by KRISHNA MOULI. TANKALA
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
octavia
In Progress
Undecided
Stephen Balukoff

Bug Description

Loadbalancer provisioning_status is inconsistent between Octavia and Neutron Database during LB create failure.

On my devstack with master branch, when a load balancer is created, if amphora is unreachable to controller-worker, after the defined request_poll_timeout LB provisioning_status in neutron DB changed to ERROR.

where as in Octavia DB provisioning_status lies/stuck in PENDING_CREATE even after [haproxy_amphora] connection_max_retries got over.

I see provisioning_status inconsistency always, if we create LB in tenant (like private, public) network.

10.1.0.8 is in private-subnet.
172.24.4.7 is in public-subnet.

stack@MouliDev:~$ neutron lbaas-loadbalancer-list
+--------------------------------------+------+-------------+---------------------+----------+
| id | name | vip_address | provisioning_status | provider |
+--------------------------------------+------+-------------+---------------------+----------+
| 545c0327-f9c1-4156-8177-850cd1e77d7c | lb1 | 10.1.0.8 | ERROR | octavia |
| c920f9de-e6d1-49db-ad8c-62148dbcc32b | lb2 | 172.24.4.7 | ERROR | octavia |
+--------------------------------------+------+-------------+---------------------+----------+
stack@MouliDev:~$ mysql
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 196
Server version: 5.5.50-0ubuntu0.14.04.1 (Ubuntu)

Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> use octavia;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> select * from load_balancer;
+----------------------------------+--------------------------------------+------+-------------+---------------------+------------------+---------+----------+-----------------+---------------------+---------------------+
| project_id | id | name | description | provisioning_status | operating_status | enabled | topology | server_group_id | created_at | updated_at |
+----------------------------------+--------------------------------------+------+-------------+---------------------+------------------+---------+----------+-----------------+---------------------+---------------------+
| b16c6ba1f68345e0802f6fac49272ac1 | 545c0327-f9c1-4156-8177-850cd1e77d7c | lb1 | | PENDING_CREATE | ONLINE | 1 | NULL | NULL | 2016-07-25 12:01:13 | 2016-07-25 12:02:22 |
| b16c6ba1f68345e0802f6fac49272ac1 | c920f9de-e6d1-49db-ad8c-62148dbcc32b | lb2 | | PENDING_CREATE | ONLINE | 1 | NULL | NULL | 2016-07-25 13:39:38 | 2016-07-25 13:40:22 |
+----------------------------------+--------------------------------------+------+-------------+---------------------+------------------+---------+----------+-----------------+---------------------+---------------------+
2 rows in set (0.00 sec)

mysql>

Expected result:
1. Loadbalancer provisioning_status should be consistent between Octavia and Neutron Database during LB create.
2. If load balancer create is failed then Octavia and Neutron Database should show provisioning_status as ERROR.

Changed in octavia:
assignee: nobody → KRISHNA MOULI. TANKALA (mouli-tankala)
Revision history for this message
li,chen (chen-li) wrote :

I thought this issue should be fixed by https://review.openstack.org/#/c/332704/.
When the creation flow revert, mark the lb to ERROR in DB.

Am i miss anything ?

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to octavia (master)

Fix proposed to branch: master
Review: https://review.openstack.org/351106

Changed in octavia:
status: New → In Progress
Revision history for this message
KRISHNA MOULI. TANKALA (mouli-tankala) wrote :

Loadbalancer provisioning_status is inconsistent between Octavia and Neutron Database during LB create failure.

When amphora post VIP plug task failed, LB provisioning state not set to ERROR during revert if result is failure.
this is causing LB provisioning_status inconsistent between Octavia and Neutron Database during LB create failure.

As part of this fix, setting LB provisioning_status to ERROR during AmphoraPostVIPPlug task revert even if result is failure.

Submitted patch: https://review.openstack.org/#/c/351106/

Changed in octavia:
assignee: KRISHNA MOULI. TANKALA (mouli-tankala) → selvakumar (selvakumar-s2)
Changed in octavia:
assignee: selvakumar (selvakumar-s2) → Stephen Balukoff (sbalukoff)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on octavia (master)

Change abandoned by Michael Johnson (<email address hidden>) on branch: master
Review: https://review.openstack.org/351106
Reason: The root cause of the issue was resolved in https://review.openstack.org/#/c/370968/

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.