Floatingip status is empty for an HA router.

Bug #1449049 reported by Sridhar Gaddam
14
This bug affects 3 people
Affects Status Importance Assigned to Milestone
neutron
Fix Released
High
Assaf Muller

Bug Description

When we associate a floatingip in an HA router setup, it is properly associated.
However, when we check the status of the Floating ip, it is shown as empty.

$neutron floatingip-show 2675730e-6e9f-438d-be8a-0d45c641cf7a
+---------------------+--------------------------------------+
| Field | Value |
+---------------------+--------------------------------------+
| fixed_ip_address | 10.0.0.3 |
| floating_ip_address | 172.24.4.3 |
| floating_network_id | 074aa016-0da2-4000-8493-7511bc3e1789 |
| id | 2675730e-6e9f-438d-be8a-0d45c641cf7a |
| port_id | 700c1977-98a8-4009-bdcc-bd4e8493b4bb |
| router_id | c82c091e-d4e8-4e8c-9d29-8eb80ed16224 |
| status | |
| tenant_id | 5bb32fcba8d141b7b5db9de07ef31d88 |
+---------------------+--------------------------------------+

Changed in neutron:
assignee: nobody → Sridhar Gaddam (sridhargaddam)
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.openstack.org/177777

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on neutron (master)

Change abandoned by Kyle Mestery (<email address hidden>) on branch: master
Review: https://review.openstack.org/177777
Reason: This review is > 4 weeks without comment, and failed Jenkins the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

Revision history for this message
Timothy Swanson (tiswanso) wrote :

Hi Sridhar,

Let me know if you mind if I propose a slightly tweaked fix. This issue blocks a bunch of tempest tests when run against an L3 HA legacy routing deployment.

The fix in https://review.openstack.org/177777 was bad because it was referencing neutron constants via "l3_constants" instead of "n_consts". The fix should be:

diff --git a/neutron/agent/l3/ha_router.py b/neutron/agent/l3/ha_router.py
index d3ef8a6..ce6147d 100644
--- a/neutron/agent/l3/ha_router.py
+++ b/neutron/agent/l3/ha_router.py
@@ -256,7 +256,7 @@ class HaRouter(router.RouterInfo):
         ip_cidr = common_utils.ip_to_cidr(fip_ip)
         self._add_vip(ip_cidr, interface_name)
         # TODO(Carl) Should this return status?
- # return l3_constants.FLOATINGIP_STATUS_ACTIVE
+ return n_consts.FLOATINGIP_STATUS_ACTIVE

     def remove_floating_ip(self, device, ip_cidr):
         self._remove_vip(ip_cidr)

The above works for making the status ACTIVE when FIP is associated in a 3-controller L3 HA = true testbed deployed via tripleO.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.openstack.org/249389

Changed in neutron:
assignee: Sridhar Gaddam (sridhargaddam) → Timothy Swanson (tiswanso)
Assaf Muller (amuller)
tags: added: l3-ha
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on neutron (master)

Change abandoned by Assaf Muller (<email address hidden>) on branch: master
Review: https://review.openstack.org/249389
Reason: Abandoned in favor of https://review.openstack.org/#/c/177777/.

Assaf Muller (amuller)
Changed in neutron:
importance: Undecided → High
Revision history for this message
Assaf Muller (amuller) wrote :

I've been doing some thinking about this issue. First, at the time the patch was submitted I wasn't aware that we had Tempest tests asserting the floating IP status changed. At a minimum this changes the priority of this bug. I pointed out an issue in the review that still stands: It's more difficult to represent the current status of a floating IP with HA routers. For example, if an HA router is configured successfully, the status of FIPs on that router will switch to ACTIVE. Afterwards a backup node dies, after which the main node dies. A failover won't occur and yet the status of the FIP will remain ACTIVE. On the other hand, this is the case with FIPs associated with legacy routers as well - We don't really maintain the status of the FIP correctly in certain conditions.

To conclude, I'm OK with just setting it to ACTIVE for now.

A possible follow up:
Right now the suggested approach is to simply set the status of the FIP to ACTIVE when the agent adds it to a local cache of FIPs (That is later configured in keepalived.conf). We could improve on this: Stop sending update_floatingip_statuses for HA routers. Instead, when the server receives an 'update_ha_routers_states' message, use that notification to also update any FIPs associated with any of the relevant routers to 'ACTIVE'. This is at the very least more performant, because we send update_ha_routers_states anyway. Might as well stop sending update_floatingip_statuses.

Revision history for this message
Carl Baldwin (carl-baldwin) wrote :

I'm OK with just setting it to ACTIVE for now, too.

Changed in neutron:
assignee: Timothy Swanson (tiswanso) → Assaf Muller (amuller)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.openstack.org/177777
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=30e048d222c23c8fa07f70fa2d6a820fa0c88e0c
Submitter: Jenkins
Branch: master

commit 30e048d222c23c8fa07f70fa2d6a820fa0c88e0c
Author: sridhargaddam <email address hidden>
Date: Mon Apr 27 14:06:59 2015 +0000

    Fix floatingip status for an HA router

    When we associate a floatingip in an HA router setup, it is properly
    associated. However, when we check the status of the Floating ip, it
    is shown as empty. The L3 agent ha_router implementation never sets
    the status of the floating IP when it's added. This behavior blocks
    tempest scenario tests that use FIPs when run against a deployment
    with L3 agent HA routers. This fix sets the FIP status to active
    when the FIP is added by an HaRouter instance.

    Co-Authored-By: Assaf Muller <email address hidden>
    Closes-Bug: #1449049
    Change-Id: I7a3de36b64132e483a927ce9fed30777e84df96a

Changed in neutron:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/liberty)

Fix proposed to branch: stable/liberty
Review: https://review.openstack.org/271741

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/kilo)

Fix proposed to branch: stable/kilo
Review: https://review.openstack.org/271742

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/liberty)

Reviewed: https://review.openstack.org/271741
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=8e685c85311000d632dc9284de3816c65ab738b0
Submitter: Jenkins
Branch: stable/liberty

commit 8e685c85311000d632dc9284de3816c65ab738b0
Author: sridhargaddam <email address hidden>
Date: Mon Apr 27 14:06:59 2015 +0000

    Fix floatingip status for an HA router

    When we associate a floatingip in an HA router setup, it is properly
    associated. However, when we check the status of the Floating ip, it
    is shown as empty. The L3 agent ha_router implementation never sets
    the status of the floating IP when it's added. This behavior blocks
    tempest scenario tests that use FIPs when run against a deployment
    with L3 agent HA routers. This fix sets the FIP status to active
    when the FIP is added by an HaRouter instance.

    Conflicts:
     neutron/tests/functional/agent/l3/framework.py
     neutron/tests/functional/agent/l3/test_dvr_router.py
     neutron/tests/functional/agent/l3/test_ha_router.py
     neutron/tests/functional/agent/l3/test_legacy_router.py

    Co-Authored-By: Assaf Muller <email address hidden>
    Closes-Bug: #1449049
    Change-Id: I7a3de36b64132e483a927ce9fed30777e84df96a
    (cherry picked from commit 30e048d222c23c8fa07f70fa2d6a820fa0c88e0c)

tags: added: in-stable-liberty
Revision history for this message
Doug Hellmann (doug-hellmann) wrote : Fix included in openstack/neutron 7.0.3

This issue was fixed in the openstack/neutron 7.0.3 release.

Revision history for this message
Thierry Carrez (ttx) wrote : Fix included in openstack/neutron 8.0.0.0b3

This issue was fixed in the openstack/neutron 8.0.0.0b3 development milestone.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on neutron (stable/kilo)

Change abandoned by Armando Migliaccio (<email address hidden>) on branch: stable/kilo
Review: https://review.openstack.org/271742
Reason: This review is > 4 weeks without comment, and failed Jenkins the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.