AdminDownPort doesn't work as expected

Bug #2033392 reported by Gregory Thiemonge
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
octavia
New
Undecided
Unassigned

Bug Description

During the failover of a STANDALONE IPv6 load balancer, the AdminDownPort task is executed but fails (silently).

https://opendev.org/openstack/octavia/src/commit/5ce85105c9b9211df9f2d88c8f1f543120eabaf4/octavia/controller/worker/v2/tasks/network_tasks.py#L1060-L1074

It sets 'admin_state_up' to False for the VRRP port and waits for its 'status' to be DOWN, but the status is always ACTIVE.
The function contains an active loop and it takes 15 sec to exit it, during these 15 seconds the VIP of the load balancer is not reachable.

Aug 29 06:16:51 gthiemon-devstack octavia-worker[1344575]: DEBUG octavia.common.base_taskflow [-] Task 'octavia-failover-loadbalancer-flow-admin-down-port' (70413951-e49b-425f-951e-ec1f43dc8065) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=1344575) _task_receiver /opt/stack/taskflow/taskflow/listeners/logging.py:190}}
Aug 29 06:16:51 gthiemon-devstack octavia-worker[1344575]: DEBUG octavia.network.drivers.neutron.base [-] Neutron extension security-group found enabled {{(pid=1344575) _check_extension_enabled /opt/stack/octavia/octavia/network/drivers/neutron/base.py:61}}
Aug 29 06:16:51 gthiemon-devstack octavia-worker[1344575]: DEBUG octavia.network.drivers.neutron.base [-] Neutron extension dns-integration found enabled {{(pid=1344575) _check_extension_enabled /opt/stack/octavia/octavia/network/drivers/neutron/base.py:61}}
Aug 29 06:16:51 gthiemon-devstack octavia-worker[1344575]: DEBUG octavia.network.drivers.neutron.base [-] Neutron extension qos found enabled {{(pid=1344575) _check_extension_enabled /opt/stack/octavia/octavia/network/drivers/neutron/base.py:61}}
Aug 29 06:16:51 gthiemon-devstack octavia-worker[1344575]: DEBUG octavia.network.drivers.neutron.base [-] Neutron extension allowed-address-pairs found enabled {{(pid=1344575) _check_extension_enabled /opt/stack/octavia/octavia/network/drivers/neutron/base.py:61}}
Aug 29 06:16:51 gthiemon-devstack octavia-worker[1344575]: DEBUG octavia.controller.worker.v2.tasks.network_tasks [-] Port ba7be21b-f782-4f10-98a2-cb9b1b8752b5 is ACTIVE instead of DOWN, waiting. {{(pid=1344575) execute /opt/stack/octavia/octavia/controller/worker/v2/tasks/network_tasks.py:1070}}
Aug 29 06:16:52 gthiemon-devstack octavia-worker[1344575]: DEBUG octavia.controller.worker.v2.tasks.network_tasks [-] Port ba7be21b-f782-4f10-98a2-cb9b1b8752b5 is ACTIVE instead of DOWN, waiting. {{(pid=1344575) execute /opt/stack/octavia/octavia/controller/worker/v2/tasks/network_tasks.py:1070}}
Aug 29 06:16:53 gthiemon-devstack octavia-worker[1344575]: DEBUG octavia.controller.worker.v2.tasks.network_tasks [-] Port ba7be21b-f782-4f10-98a2-cb9b1b8752b5 is ACTIVE instead of DOWN, waiting. {{(pid=1344575) execute /opt/stack/octavia/octavia/controller/worker/v2/tasks/network_tasks.py:1070}}
Aug 29 06:16:54 gthiemon-devstack octavia-worker[1344575]: DEBUG octavia.controller.worker.v2.tasks.network_tasks [-] Port ba7be21b-f782-4f10-98a2-cb9b1b8752b5 is ACTIVE instead of DOWN, waiting. {{(pid=1344575) execute /opt/stack/octavia/octavia/controller/worker/v2/tasks/network_tasks.py:1070}}
Aug 29 06:16:55 gthiemon-devstack octavia-worker[1344575]: DEBUG octavia.controller.worker.v2.tasks.network_tasks [-] Port ba7be21b-f782-4f10-98a2-cb9b1b8752b5 is ACTIVE instead of DOWN, waiting. {{(pid=1344575) execute /opt/stack/octavia/octavia/controller/worker/v2/tasks/network_tasks.py:1070}}
Aug 29 06:16:56 gthiemon-devstack octavia-worker[1344575]: DEBUG octavia.controller.worker.v2.tasks.network_tasks [-] Port ba7be21b-f782-4f10-98a2-cb9b1b8752b5 is ACTIVE instead of DOWN, waiting. {{(pid=1344575) execute /opt/stack/octavia/octavia/controller/worker/v2/tasks/network_tasks.py:1070}}
Aug 29 06:16:57 gthiemon-devstack octavia-worker[1344575]: DEBUG octavia.controller.worker.v2.tasks.network_tasks [-] Port ba7be21b-f782-4f10-98a2-cb9b1b8752b5 is ACTIVE instead of DOWN, waiting. {{(pid=1344575) execute /opt/stack/octavia/octavia/controller/worker/v2/tasks/network_tasks.py:1070}}
Aug 29 06:16:58 gthiemon-devstack octavia-worker[1344575]: DEBUG octavia.controller.worker.v2.tasks.network_tasks [-] Port ba7be21b-f782-4f10-98a2-cb9b1b8752b5 is ACTIVE instead of DOWN, waiting. {{(pid=1344575) execute /opt/stack/octavia/octavia/controller/worker/v2/tasks/network_tasks.py:1070}}
Aug 29 06:16:59 gthiemon-devstack octavia-worker[1344575]: DEBUG octavia.controller.worker.v2.tasks.network_tasks [-] Port ba7be21b-f782-4f10-98a2-cb9b1b8752b5 is ACTIVE instead of DOWN, waiting. {{(pid=1344575) execute /opt/stack/octavia/octavia/controller/worker/v2/tasks/network_tasks.py:1070}}
Aug 29 06:17:00 gthiemon-devstack octavia-worker[1344575]: DEBUG octavia.controller.worker.v2.tasks.network_tasks [-] Port ba7be21b-f782-4f10-98a2-cb9b1b8752b5 is ACTIVE instead of DOWN, waiting. {{(pid=1344575) execute /opt/stack/octavia/octavia/controller/worker/v2/tasks/network_tasks.py:1070}}
Aug 29 06:17:01 gthiemon-devstack octavia-worker[1344575]: DEBUG octavia.controller.worker.v2.tasks.network_tasks [-] Port ba7be21b-f782-4f10-98a2-cb9b1b8752b5 is ACTIVE instead of DOWN, waiting. {{(pid=1344575) execute /opt/stack/octavia/octavia/controller/worker/v2/tasks/network_tasks.py:1070}}
Aug 29 06:17:02 gthiemon-devstack octavia-worker[1344575]: DEBUG octavia.controller.worker.v2.tasks.network_tasks [-] Port ba7be21b-f782-4f10-98a2-cb9b1b8752b5 is ACTIVE instead of DOWN, waiting. {{(pid=1344575) execute /opt/stack/octavia/octavia/controller/worker/v2/tasks/network_tasks.py:1070}}
Aug 29 06:17:03 gthiemon-devstack octavia-worker[1344575]: DEBUG octavia.controller.worker.v2.tasks.network_tasks [-] Port ba7be21b-f782-4f10-98a2-cb9b1b8752b5 is ACTIVE instead of DOWN, waiting. {{(pid=1344575) execute /opt/stack/octavia/octavia/controller/worker/v2/tasks/network_tasks.py:1070}}
Aug 29 06:17:04 gthiemon-devstack octavia-worker[1344575]: DEBUG octavia.controller.worker.v2.tasks.network_tasks [-] Port ba7be21b-f782-4f10-98a2-cb9b1b8752b5 is ACTIVE instead of DOWN, waiting. {{(pid=1344575) execute /opt/stack/octavia/octavia/controller/worker/v2/tasks/network_tasks.py:1070}}
Aug 29 06:17:05 gthiemon-devstack octavia-worker[1344575]: DEBUG octavia.controller.worker.v2.tasks.network_tasks [-] Port ba7be21b-f782-4f10-98a2-cb9b1b8752b5 is ACTIVE instead of DOWN, waiting. {{(pid=1344575) execute /opt/stack/octavia/octavia/controller/worker/v2/tasks/network_tasks.py:1070}}
Aug 29 06:17:06 gthiemon-devstack octavia-worker[1344575]: ERROR octavia.controller.worker.v2.tasks.network_tasks [-] Port ba7be21b-f782-4f10-98a2-cb9b1b8752b5 failed to go DOWN. Port status is still ACTIVE. Ignoring and continuing.
Aug 29 06:17:06 gthiemon-devstack octavia-worker[1344575]: DEBUG octavia.common.base_taskflow [-] Task 'octavia-failover-loadbalancer-flow-admin-down-port' (70413951-e49b-425f-951e-ec1f43dc8065) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=1344575) _task_receiver /opt/stack/taskflow/taskflow/listeners/logging.py:178}}

After further investigations, the request is correctly sent but neutron doesn't update the status of the ports to DOWN when they are disabled (tested manually with the CLI).

description: updated
Revision history for this message
Gregory Thiemonge (gthiemonge) wrote :

This is a bug in ML2/OVN, reported to neutron: https://bugs.launchpad.net/neutron/+bug/2036705

Revision history for this message
Gregory Thiemonge (gthiemonge) wrote :
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.