[dvr][fast-exit] incorrect policy rules get deleted when a distributed router has ports on multiple tenant networks
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Ubuntu Cloud Archive |
Fix Released
|
Medium
|
Unassigned | ||
Pike |
Fix Released
|
Medium
|
Unassigned | ||
Queens |
Fix Released
|
Medium
|
Unassigned | ||
neutron |
Fix Released
|
Medium
|
Dmitrii Shcherbakov | ||
neutron (Ubuntu) |
Fix Released
|
Medium
|
Unassigned | ||
Artful |
Won't Fix
|
Medium
|
Unassigned | ||
Bionic |
Fix Released
|
Medium
|
Unassigned |
Bug Description
Ubuntu SRU details
------------------
[Impact]
See Original Description below.
[Test Case]
See Original Description below.
[Regression Potential]
Low. All patches have landed upstream in corresponding stable branches.
Original Description
-------
TL;DR: ip -4 rule del priority <priority> table <table-id> type unicast will delete the first matching rule it encounters: if there are two rules with the same priority it will just kill the first one it finds.
The original setup is described here:
https:/
OpenStack Queens from UCA (xenial, GA kernel, deployed via OpenStack charms), 2 external subnets (one routed provider network), 2 tenant subnets all in the same address scope to trigger "fast exit".
2 tenant networks attached (subnets 192.168.100.0/24 and 192.168.200.0/24) to a DVR:
# 2 rules as expected
ip netns exec qrouter-
0: from all lookup local
32766: from all lookup main
32767: from all lookup default
80000: from 192.168.100.0/24 lookup 16
80000: from 192.168.200.0/24 lookup 16
# remove 192.168.200.0/24 sometimes deletes an incorrect policy rule
openstack router remove subnet pubrouter othertenantsubnet
# ip route del contains the cidr
2018-03-29 20:09:52.946 2083594 DEBUG neutron.
tns', 'exec', 'fip-d0f008fc-
] create_process /usr/lib/
# ip rule delete is not that specific
2018-03-29 20:09:53.195 2083594 DEBUG neutron.
ocess /usr/lib/
2018-03-29 20:15:59.210 2083594 DEBUG neutron.
2018-03-29 20:15:59.455 2083594 DEBUG neutron.
~~~~
ip netns exec qrouter-
0: from all lookup local
32766: from all lookup main
32767: from all lookup default
80000: from 192.168.100.0/24 lookup 16
80000: from 192.168.200.0/24 lookup 16
# try to delete a rule manually to see what is going on
ip netns exec qrouter-
0: from all lookup local
32766: from all lookup main
32767: from all lookup default
80000: from 192.168.100.0/24 lookup 16
80000: from 192.168.200.0/24 lookup 16
0: from all lookup local
32766: from all lookup main
32767: from all lookup default
80000: from 192.168.200.0/24 lookup 16
# ^^ 192.168.100.0/24 rule got deleted instead of 192.168.200.0/24
# add the rule back manually
ip netns exec qrouter-
# different order now - 192.168.200.0/24 is first
ip netns exec qrouter-
0: from all lookup local
32766: from all lookup main
32767: from all lookup default
80000: from 192.168.200.0/24 lookup 16
80000: from 192.168.100.0/24 lookup 16
# now 192.168.200.0/24 got deleted because it was first to match
ip netns exec qrouter-
0: from all lookup local
32766: from all lookup main
32767: from all lookup default
80000: from 192.168.200.0/24 lookup 16
80000: from 192.168.100.0/24 lookup 16
0: from all lookup local
32766: from all lookup main
32767: from all lookup default
80000: from 192.168.100.0/24 lookup 16
Code:
_dvr_internal_
https:/
_delete_
https:/
ip_rule = ip_lib.
for subnet in router_
IpRuleCommand
https:/
# TODO(Carl) ip ignored in delete, okay in general?
He-he, experience shows that definitely not.
We need to use the most specific rule description to avoid ordering issues.
ip -4 rule del from 192.168.200.0/24 priority 80000 table 16 type unicast
With a fix it looks like this:
2018-03-29 20:58:57.023 192084 DEBUG neutron.
tags: | added: l3-dvr-backlog |
Changed in neutron: | |
importance: | Undecided → Critical |
importance: | Critical → Medium |
Changed in neutron (Ubuntu Artful): | |
status: | New → Triaged |
importance: | Undecided → Medium |
Changed in neutron (Ubuntu Bionic): | |
status: | Confirmed → Triaged |
importance: | Undecided → Medium |
description: | updated |
Fix proposed to branch: master /review. openstack. org/557836
Review: https:/