Static route can get stuck in the router snat namespace

[Anton Kurbatov]

I ran into a problem where a static route just gets stuck in the snat namepsace, even when removing all static routes from a distributed router with ha enabled.
Here is a simple demo from my devstack setup:

[root@node0 ~]# openstack network create private
[root@node0 ~]# openstack subnet create private --network private --subnet-range 192.168.10.0/24 --dhcp --gateway 192.168.10.1
[root@node0 ~]# openstack router create r1 --external-gateway public --distributed --ha
[root@node0 ~]# openstack router add subnet r1 private
[root@node0 ~]# openstack router set r1 --route destination=8.8.8.0/24,gateway=192.168.10.100 --route destination=8.8.8.0/24,gateway=192.168.10.200

After multipath route was added, snat-ns routes look like this:

[root@node0 ~]# ip netns exec snat-dcbec74b-2003-4447-8854-524d918260ac ip r
default via 10.136.16.1 dev qg-94c43336-56 proto keepalived
8.8.8.0/24 via 192.168.10.200 dev sg-dcf4a20b-8a proto keepalived
8.8.8.0/24 via 192.168.10.100 dev sg-dcf4a20b-8a proto keepalived
8.8.8.0/24 via 192.168.10.100 dev sg-dcf4a20b-8a proto static
10.136.16.0/20 dev qg-94c43336-56 proto kernel scope link src 10.136.17.171
169.254.0.0/24 dev ha-11b5b7d3-4e proto kernel scope link src 169.254.0.21
169.254.192.0/18 dev ha-11b5b7d3-4e proto kernel scope link src 169.254.195.228
192.168.10.0/24 dev sg-dcf4a20b-8a proto kernel scope link src 192.168.10.228
[root@node0 ~]#

Note that there is only one 'static' route added by neutron and no multipath route.
And two routes with 'proto keepalived' that have been added by keepalived process.
Now delete all routes and check the routes inside snat-ns, the route is still there:

[root@node0 ~]# openstack router set r1 --no-route
[root@node0 ~]# ip netns exec snat-dcbec74b-2003-4447-8854-524d918260ac ip r
default via 10.136.16.1 dev qg-94c43336-56 proto keepalived
8.8.8.0/24 via 192.168.10.100 dev sg-dcf4a20b-8a proto static
10.136.16.0/20 dev qg-94c43336-56 proto kernel scope link src 10.136.17.171
169.254.0.0/24 dev ha-11b5b7d3-4e proto kernel scope link src 169.254.0.21
169.254.192.0/18 dev ha-11b5b7d3-4e proto kernel scope link src 169.254.195.228
192.168.10.0/24 dev sg-dcf4a20b-8a proto kernel scope link src 192.168.10.228
[root@node0 ~]#

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/867678

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Slawek Kaplonski <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/867678
Reason: This review is > 4 weeks without comment, and failed Zuul jobs the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/867678
Committed: https://opendev.org/openstack/neutron/commit/f3c743d0901ba3405a4fc1586f45d2cc434fcdd4
Submitter: "Zuul (22348)"
Branch: master

commit f3c743d0901ba3405a4fc1586f45d2cc434fcdd4
Author: Anton Kurbatov <email address hidden>
Date: Wed Dec 14 16:50:06 2022 +0000

    Do not update static routes in snat-ns for dvr router with ha

    If a router is distributed with ha enabled, then the keepalived service
    is responsible for setting static routes. This patch adds a check if
    the router ha is disabled before adding routes. Otherwise, there are
    duplicate routes and the issue when this route needs to be removed.
    In addition this patch fixes multipath route in the snat-ns if no HA is
    enabled.

    Closes-Bug: #1999678
    Signed-off-by: Anton Kurbatov <email address hidden>
    Change-Id: I8f1004b3fe2cad79cb61aa942b257c1508d18b68

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 23.0.0.0b3 development milestone.