[ovn] fip assignment to instance via router with snat disabled is broken

[James Page]

Ubuntu: 20.04
OpenStack: Ussuri
Networking: OVN (20.03.x)

Network topology:

Geneve overlay network for project networks, router has snat disabled and the project network and the external network are all in the same address scope and subnet pool. OVN routers are simply acting as L3 routers and instances on the project network can be directly accessed by the address assigned to their port (with appropriate route configuration in the outside of openstack world).

Issue:

Its possible to create and then associate a floating IP on the external network with an instance attached to the project network - however this does not work - access to the instance via the FIP is broken, as is access to its fixed IP (when this worked OK before).

Thoughts:

The concept of a FIP is very much NAT centric, and in the described configuration NAT is very much disabled. This idea seems to have worked way back in icehouse, however does not work at Ussuri. If this is not a supported network model, the association of the FIP to the instance should error with an appropriate message that NAT is not supported to the in-path router to the external network.

[Bence Romsics (bence-romsics)]

As far as I can remember the enable_snat bit in the external_gateway_info of a router should only control the generic SNAT-ting of all traffic from internal networks to the external. Support for floating IPs should be orthogonal to the enable_snat bit.

Could you please elaborate on the connectivity test you did? From where and what exactly? Does access to the fixed IP break only when a floating IP is created is it broken when enable_snat is updated to False?

First I need to build an ovn environment to reproduce this.

[Bence Romsics (bence-romsics)]

I built a master ovn devstack (first using vlan networks) to reproduce this but I could not, this is what I did:

# physical bridge, bridge mappings, hostname
sudo ovs-vsctl --may-exist add-br br-physnet0
sudo ovs-vsctl set Open_vSwitch . external_ids:ovn-bridge-mappings=public:br-ex,physnet0:br-physnet0
sudo ovs-vsctl set Open_vSwitch . external_ids:hostname=$(hostname)

# target IP on the external net
sudo ip link set up dev br-physnet0
sudo ip address add 10.0.0.2/24 dev br-physnet0

# random image for testing
openstack image create --disk-format qcow2 --public --file ~/ubuntu-20.04-server-cloudimg-amd64-disk-kvm-root-password.img u2004

openstack address scope create scope0
openstack subnet pool create --address-scope scope0 --pool-prefix 10.0.0.0/8 --default-prefix-length 22 pool0

# external net
openstack network create net-physnet0 --external --provider-network-type flat --provider-physical-network physnet0
openstack subnet create subnet-physnet0 --network net-physnet0 --subnet-pool pool0 --subnet-range 10.0.0.0/24 --gateway 10.0.0.1 --no-dhcp

# internal net in the same address scope
openstack network create net0 --provider-network-type vlan --provider-physical-network physnet0 --provider-segment 100
openstack subnet create subnet0 --network net0 --subnet-pool pool0 --subnet-range 10.0.1.0/24 --gateway 10.0.1.1

# router in disable_snat mode
openstack router create router0
openstack router set --external-gateway net-physnet0 --disable-snat router0
openstack router add subnet router0 subnet0

# add route for external system to know where the internal net is
sudo ip route add 10.0.1.0/24 via $( openstack router show router0 -f yaml -c external_gateway_info | awk '/ip_address:/ { print $3 }' )

openstack server create --flavor ds1G --image u2004 --nic net-id=net0 --wait vm0

# from internal to external
sudo virsh console "$( openstack server show vm0 -f value -c OS-EXT-SRV-ATTR:instance_name )"
ping 10.0.0.2 # works

# from external to internal fixed IP
ping 10.0.1.70 # works

openstack floating ip create --port "$( openstack port list --device-id "$( openstack server show vm0 -f value -c id )" -f value -c id | head -1 )" net-physnet0

# from external to floating IP mapped to internal fixed IP
ping 10.0.0.41 # works

I want to configure my environment to work with geneve networks instead of vlan networks and try that way too. But also please tell me what else you're doing differently compared to above.

[Bence Romsics (bence-romsics)]

Sorry for the slow followup.

I did reconfigure my ovn environment so I could change the internal network to a geneve network and with that did the same test as above. I was able to ping 10.0.0.2 from vm0 both before and after the floating IP creation. So currently I'm not able to reproduce the bug reported.

Please let us know the exact steps to reproduce the error you see.

[Launchpad Janitor (janitor)]

[Expired for neutron because there has been no activity for 60 days.]

[Launchpad Janitor (janitor)]

[Expired for neutron (Ubuntu) because there has been no activity for 60 days.]