DVR: cannot manually remove router from l3 agent

Bug #1593653 reported by Oleg Bondarev
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Fix Released
High
Hong Hui Xiao

Bug Description

This is a regression from this commit [1]. If there are dvr serviceable ports on the node with agent, server now will notify agent with router_updated rather than router_removed, however when updating router, agent will request router_info and that's where server will schedule router back to this l3 agent due to autoscheduling being enabled.

[1] https://review.openstack.org/#/q/c198710dc551bc0f79851a7801038b033088a8c2

The potential fix might be not scheduler router when agent request router_info.

Now there are 2 code path that will auto schedule routers to agent.

The first one is get_router_ids, which will bind all un-scheduled
routers to the agent. This will happen at agent startup, revive, and
update.

The second one is sync_routers, which will bind the specified routers
to agent. This will happen when agent requires some router info.

Since 'router_id' was removed in bug 1594711, get_router_ids
will always be called at agent startup, revive, and update, which can
cover all the cases that auto schedule needs. And no need for other
code path.

Revision history for this message
Oleg Bondarev (obondarev) wrote :

This will likely be fixed by https://review.openstack.org/#/c/317949/ which removes autoscheduling from sync_routers rpc handler.

It's unlikely it will be backported so we probably need some other fix for stable/mitaka or revert commit 1f444899ca225846ac54ed9e33e112a30cc8bc0f

description: updated
Revision history for this message
Hong Hui Xiao (xiaohhui) wrote :

I can reproduce it in my local env, I would like to look into it.

Changed in neutron:
status: New → Confirmed
assignee: nobody → Hong Hui Xiao (xiaohhui)
tags: added: l3-ipam-dhcp
Revision history for this message
Hong Hui Xiao (xiaohhui) wrote :

I think we can partially apply https://review.openstack.org/#/c/317949/ to fix this bug. That is removing code at [1].

But before that, we need to remove the deprecated option router_id, so that get_router_ids will always be called in a full sync, and the auto_schedule_routers can be handled there. Removing router_id is reported at bug 1594711

[1] https://github.com/openstack/neutron/blob/b59bb0fcfa41963c0e2f7bcbf34b7e4f4ff5cd08/neutron/api/rpc/handlers/l3_rpc.py#L96-L97

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.openstack.org/332102

Changed in neutron:
status: Confirmed → In Progress
description: updated
Hong Hui Xiao (xiaohhui)
description: updated
description: updated
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.openstack.org/332102
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=64726983f2161b32e516f3f12007e5f5a3d57e50
Submitter: Jenkins
Branch: master

commit 64726983f2161b32e516f3f12007e5f5a3d57e50
Author: Hong Hui Xiao <email address hidden>
Date: Tue Jun 21 10:56:45 2016 +0000

    Not auto schedule router when sync routers from agent

    In this patch, auto schedule router will be removed from sync_routers,
    so that the reported bug can be fixed. And potential race can be avoid
    accoridng to [1]

    The result of patch will make the l3 agent can't get the router info
    when the router is not bound to the l3 agent. And router in agent will
    be removed during the agent processing. This makes sense, since, in
    neutron server, the router is not tied to the agent. For DVR, if there
    are service port in the agent host, the router info will still be
    returned to l3 agent.

    [1] https://review.openstack.org/#/c/317949/

    Change-Id: Id0a8cf7537fefd626df06064f915d2de7c1680c6
    Co-Authored-By: John Schwarz <email address hidden>
    Closes-Bug: #1593653

Changed in neutron:
status: In Progress → Fix Released
tags: added: neutron-proactive-backport-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/mitaka)

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/340868

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/mitaka)

Reviewed: https://review.openstack.org/340868
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=33650bf1d1994a96eff993af0bfdaa62588f08a4
Submitter: Jenkins
Branch: stable/mitaka

commit 33650bf1d1994a96eff993af0bfdaa62588f08a4
Author: Hong Hui Xiao <email address hidden>
Date: Tue Jun 21 10:56:45 2016 +0000

    Not auto schedule router when sync routers from agent

    In this patch, auto schedule router will be removed from sync_routers,
    so that the reported bug can be fixed. And potential race can be avoid
    accoridng to [1]

    The result of patch will make the l3 agent can't get the router info
    when the router is not bound to the l3 agent. And router in agent will
    be removed during the agent processing. This makes sense, since, in
    neutron server, the router is not tied to the agent. For DVR, if there
    are service port in the agent host, the router info will still be
    returned to l3 agent.

    [1] https://review.openstack.org/#/c/317949/

    Change-Id: Id0a8cf7537fefd626df06064f915d2de7c1680c6
    Co-Authored-By: John Schwarz <email address hidden>
    Closes-Bug: #1593653
    (cherry picked from commit 64726983f2161b32e516f3f12007e5f5a3d57e50)

tags: added: in-stable-mitaka
Revision history for this message
Doug Hellmann (doug-hellmann) wrote : Fix included in openstack/neutron 9.0.0.0b2

This issue was fixed in the openstack/neutron 9.0.0.0b2 development milestone.

Revision history for this message
Doug Hellmann (doug-hellmann) wrote : Fix included in openstack/neutron 8.2.0

This issue was fixed in the openstack/neutron 8.2.0 release.

tags: removed: neutron-proactive-backport-potential
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.