OVN deployment with DVR environment incorrectly routes FIP traffic through Controller/Chassis-GW
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
neutron |
Fix Released
|
Undecided
|
Lucas Alvares Gomes | ||
tripleo |
Fix Released
|
Undecided
|
Unassigned |
Bug Description
TripleO Stein. OVN deployment with DVR environment file incorrectly routes FIP traffic through Controller/
Steps to reproduce
===========
1. Deployed overcloud enabling ovn with DVR.
The following neutron environment files were used (in additional to network isolation using bonded VLAN and other customizations)
-e $TD/environment
-e $TD/environment
-e $TD/environment
2. After overcloud deployment confirmed that the neutron conf files and chassis settings are correct.
neutron.conf -> enable_dvr=True
ml2_conf.ini -> enable_
bridge_mapping on compute chassis -> ovn-bridge-
3. Deployed instance with Geneve Tenant network with floating IP on VLAN external ‘datacentre’ network.
Expected Result
=============
FIP traffic is routed through the same compute node as instance via a local NAT rule.
Actual Result
============
FIP is operational but traffic routed through the Controller/
The matching NAT entry for the FIP shows that the external_mac is Null and logical port was not set, so there is no local NAT routing occurring as observed.
Environment
===========
1. Tripleo Stein using the latest current-tripleo-rdo container images with standard Compute role plus OvsDpdk and SR-IOV roles.
2. Ceph and Pure Storage
3. OVN networking (default in Stein) with the following neutron environment
-e $TD/environment
-e $TD/environment
-e $TD/environment
(in additional to network isolation using bonded VLAN and other customizations)
Confirmed that after deployment
• neutron.conf -> enable_dvr=True
• ml2_conf.ini -> enable_
• bridge_mapping on compute chassis -> ovn-bridge-
Logs & Configs
===========
neutron.conf -> enable_dvr=True
ml2_conf.ini -> enable_
bridge_mapping on compute chassis -> ovn-bridge-
ovn-nbctl lr-nat-list neutron-
TYPE EXTERNAL_IP LOGICAL_IP EXTERNAL_MAC LOGICAL_PORT
dnat_and_snat 10.3.27.20 192.168.0.18
snat 10.3.25.207 192.168.0.0/24
Changed in tripleo: | |
assignee: | nobody → Lucas Alvares Gomes (lucasagomes) |
milestone: | none → ussuri-1 |
Changed in tripleo: | |
milestone: | ussuri-1 → ussuri-2 |
Changed in tripleo: | |
assignee: | Lucas Alvares Gomes (lucasagomes) → nobody |
tags: | added: ovn |
Changed in neutron: | |
assignee: | nobody → Lucas Alvares Gomes (lucasagomes) |
Changed in tripleo: | |
milestone: | ussuri-2 → ussuri-3 |
Changed in tripleo: | |
milestone: | ussuri-3 → ussuri-rc3 |
tags: | added: neutron-proactive-backport-potential |
Changed in tripleo: | |
milestone: | ussuri-rc3 → victoria-1 |
Changed in tripleo: | |
milestone: | victoria-1 → victoria-3 |
Changed in tripleo: | |
milestone: | victoria-3 → wallaby-1 |
Changed in tripleo: | |
milestone: | wallaby-1 → wallaby-2 |
I ran into the same issue downstream Red Hat OSP 13 in 2 different, brand new environments. I'm going to add further details from my troubleshooting.
First of all, a workaround or way to "fix" this is to reboot the instance. The external_mac field then will be populated. However, if one detaches and reattaches the VIP, one can easily reproduce the issue.
From my lab, here are steps to reproduce and verify this:
Create server without FIP: ------- ------- ------- ------- ----+-- ------- ---+--- -----+- ------- ----+-- ------- ----+-- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- --+ ------- ------- ------- ------- ----+-- ------- ---+--- -----+- ------- ----+-- ------- ----+-- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- --+ c22e-4e5b- be90-f2bc79b30c a4 | rhel-test1 | ACTIVE | - | Running | private1= 2000:192: 168:0:f816: 3eff:fe3f: 9ee4, 192.168.0.101, 172.31.0.201 | b39a-4231- b2f1-46dae48744 b4 | rhel-test2 | ACTIVE | - | Running | private1= 2000:192: 168:0:f816: 3eff:fe00: bcd6, 192.168.0.103, 172.31.0.217 | 9951-4bff- 8b97-3a6e02b794 67 | rhel-test3 | ACTIVE | - | Running | private2= 192.168. 1.101, 2000:192: 168:1:f816: 3eff:feb7: 5d7d | ------- ------- ------- ------- ----+-- ------- ---+--- -----+- ------- ----+-- ------- ----+-- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- --+
~~~
(overcloud) [stack@undercloud-0 ~]$ nova list
+------
| ID | Name | Status | Task State | Power State | Networks |
+------
| ac73af43-
| f8dccb8d-
| 1cd9d144-
+------
~~~
Check NAT rules: -controller- 0 ~]# export SB=$(sudo ovs-vsctl get open . external_ ids:ovn- remote | sed -e 's/\"//g') -controller- 0 ~]# export NB=$(sudo ovs-vsctl get open . external_ ids:ovn- remote | sed -e 's/\"//g' | sed -e 's/6642/6641/g') -controller- 0 ~]# alias ovn-sbctl='sudo docker exec ovn_controller ovn-sbctl --db=$SB' -controller- 0 ~]# alias ovn-nbctl='sudo docker exec ovn_controller ovn-nbctl --db=$NB' -controller- 0 ~]# alias ovn-trace='sudo docker exec ovn_controller ovn-trace --db=$SB' -controller- 0 ~]# ovn-nbctl find NAT type=dnat_and_snat 1c06-4c22- b0e5-50ef64813c c8 fip_external_ mac"="fa: 16:3e:ee: 4b:3b", "neutron: fip_id" ="b0c03864- 8c79-43b3- 9240-370cfbcde9 04", "neutron: fip_port_ id"="b7e9fb53- db4e-4b3e- a6e4-2acbbc4ba9 ba", "neutron: revision_ number" ="2", "neutron: router_ name"=" neutron- e75c5fb4- 29ee-450d- 840f-a911c98962 56"} db4e-4b3e- a6e4-2acbbc4ba9 ba"
~~~
[root@overcloud
[root@overcloud
[root@overcloud
[root@overcloud
[root@overcloud
[root@overcloud
_uuid : 61b357c0-
external_ids : {"neutron:
external_ip : "172.31.0.201"
external_mac : "fa:16:3e:ee:4b:3b"
logical_ip : "192.168.0.101"
logical_port : "b7e9fb53-
type : dnat_and_snat
_uuid : b0782824- 00e9-4b66- bc4b-9d54cf8ec7 37 fip_external_ mac"="fa: 16:3e:8b: 04:4f", "neutron: fip_id" ="4fc2b87a- a188-4468- ad40-8b03ed26b5 6d", "neutron: fip_port_ id"="581de757- 7b2e-4995- aa84-3c4bc58edd 4d", "neutron: revision_ number" ="2", "neutron: router_ name"=" neutron- e75c5fb4- 29ee-45. ..
external_ids : {"neutron: