Dead L3 agent should reflect HA router states

[Assaf Muller]

The main use case of L3 HA is bouncing back from a machine (That is running a L3 agent) dying. In this case, with bp/report-ha-router-master merged, any active routers on that node will remain active in the Neutron DB (As the dead agent cannot update the server of anything). A backup node will pick up the routers previously active on the dead node and will update their status, resulting in the Neutron DB having the router 'active' on two different nodes. This can mess up l2pop as HA router interfaces will now be arbitrarily hosted on any of the 'active' hosts.

The solution would be that when a L3 agent is marked as dead, to go ahead and change the HA router states on that agent to from active to standby, and also to update the router ports 'host' value to point to the new active agent.

Note: This bug is at least partially coupled with https://bugs.launchpad.net/neutron/+bug/1365476. Ideally we could solve the two bugs in two separate patches with no dependencies.

[Ann Taraday (akamyshnikova)]

@Mike

Are you actually working on this or I can look into?

[Assaf Muller (amuller)]

Changing priority to low. We fixed L3 HA's interaction with l2pop. The impact of this issue (I would hesitant to use the word bug here) is that the admin sees an 'active' router on an agent marked as dead, which can be confusing - There is no operational impact otherwise.

[Mike Kolesnik (mkolesni)]

@Ann

Feel free to tackle this

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/246952

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/246952
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=fc514b8e71d6ba7a1541ff925c63becf8834f6f5
Submitter: Jenkins
Branch: master

commit fc514b8e71d6ba7a1541ff925c63becf8834f6f5
Author: Ann Kamyshnikova <email address hidden>
Date: Wed Nov 18 17:03:12 2015 +0300

    Update HA router state if agent is not active

    If L3 agent is marked dead it is still show as 'active' in the HA
    router states on that agent. Current change adds check for such
    case and updates HA router state from 'active' to 'standby' if
    agent is dead.

    Closes-bug: #1461148

    Change-Id: Ie02e787a5b30cbe01694081cdec924304ab2ef41

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/liberty
Review: https://review.openstack.org/271819

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/kilo
Review: https://review.openstack.org/271820

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/271819
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=b908c5521e0b15a7e5f6eb65fa6d675d00f1e77a
Submitter: Jenkins
Branch: stable/liberty

commit b908c5521e0b15a7e5f6eb65fa6d675d00f1e77a
Author: Ann Kamyshnikova <email address hidden>
Date: Wed Nov 18 17:03:12 2015 +0300

    Update HA router state if agent is not active

    If L3 agent is marked dead it is still show as 'active' in the HA
    router states on that agent. Current change adds check for such
    case and updates HA router state from 'active' to 'standby' if
    agent is dead.

    Closes-bug: #1461148

    Change-Id: Ie02e787a5b30cbe01694081cdec924304ab2ef41
    (cherry picked from commit fc514b8e71d6ba7a1541ff925c63becf8834f6f5)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/271820
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=759918634628ce8fbed722ec08abef0bfc3649d6
Submitter: Jenkins
Branch: stable/kilo

commit 759918634628ce8fbed722ec08abef0bfc3649d6
Author: Ann Kamyshnikova <email address hidden>
Date: Wed Nov 18 17:03:12 2015 +0300

    Update HA router state if agent is not active

    If L3 agent is marked dead it is still show as 'active' in the HA
    router states on that agent. Current change adds check for such
    case and updates HA router state from 'active' to 'standby' if
    agent is dead.

    Closes-bug: #1461148

    Change-Id: Ie02e787a5b30cbe01694081cdec924304ab2ef41
    (cherry picked from commit fc514b8e71d6ba7a1541ff925c63becf8834f6f5)

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/neutron 2015.1.4 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 2015.1.4 release.