[ovn] instance in a shared network owned by another project is unreachable via floating IP

[Debasis]

In my openstack environment, whenever I create an instance in a shared network owned by another project, the instance becomes unreachable even via floating IP. I'm using the latest kolla ansible with ovn network. I cannot even ssh into the server because it does not run the cloud-inti to set the password
(and also because public network unreachable). Spice brings the login prompt but cannot login due to lack of credentials.

 Any suggestion on how to solve the issue?

[Debasis (debamondal)]

I created a shared network (and subnet) in the admin project. I also created a router and connected it with floating IP and the subnet. I created a new user and a project for the user. In this new project, if I create a server in this shared net, I cannot associate the floating Ip because for some reason, the network ports created the in subnet are not reachable by the floating IP!

To solve that problem, I created a new router in this new project and attached with the public network and the subnet(under shared network ). This allows me to attach the floating IP with the servers in the shared network.

However, for some reason, it does not run cloud init and the floating IP still remains unreachable!

[Debasis (debamondal)]

I was able to fix this. This issue arises with the network type vxlan. Geneve works perfectly with ovn.

[Bence Romsics (bence-romsics)]

Hi,

Thanks for your report!

I believe I see some buggy behavior here, though not the exact same thing as you described. Can you please cross-check if what I'm doing below covers the bug you intended to report?

(I'm using devstack, not kolla, but I believe that should not matter.)

# Make sure we enable both geneve and vxlan type networks.
source ~/src/os/openstack/devstack/inc/ini-config
iniset /etc/neutron/plugins/ml2/ml2_conf.ini ml2 tenant_network_types geneve,vxlan,vlan
iniset /etc/neutron/plugins/ml2/ml2_conf.ini ml2 type_drivers local,flat,vlan,vxlan,geneve
sudo systemctl restart devstack@q-svc

# Configure external connectivity for floating IPs.
sudo ovs-vsctl set Open_vSwitch . external_ids:ovn-bridge-mappings=public:br-ex
sudo ip address add "$( openstack subnet show -f value -c gateway_ip public-subnet )/$( openstack subnet show -f value -c cidr public-subnet | cut -d/ -f2 )" dev br-ex
sudo ip link set up dev br-ex
sudo ip route add "$( openstack subnet show public-subnet -f value -c cidr )" dev br-ex

# Create and connect the shared networks as project #1 (= admin).
source openrc admin admin

openstack net create shared-geneve --provider-network-type geneve --share
openstack net create shared-vxlan --provider-network-type vxlan --share

openstack subnet create shared-geneve-subnet --ip-version 4 --network shared-geneve --subnet-range 10.0.4.0/24
openstack subnet create shared-vxlan-subnet --ip-version 4 --network shared-vxlan --subnet-range 10.0.5.0/24

openstack router set --external-gateway public router1

openstack router add subnet router1 shared-geneve-subnet
openstack router add subnet router1 shared-vxlan-subnet

# From here on we have two scenarios: create the servers and floating IPs either as the same project (= admin) or as a different project (= demo).
source openrc admin admin # scenario: same project
source openrc demo demo # scenario: different project

openstack server create --flavor cirros256 --image cirros-0.6.2-x86_64-disk --nic net-id=shared-geneve vm-geneve --wait
openstack server create --flavor cirros256 --image cirros-0.6.2-x86_64-disk --nic net-id=shared-vxlan vm-vxlan --wait

port_geneve="$( openstack port list --server vm-geneve -f value -c id )"
port_vxlan="$( openstack port list --server vm-vxlan -f value -c id )"

ip_geneve="$( openstack floating ip create public --port "$port_geneve" -f value -c 'floating_ip_address' )"
ip_vxlan="$( openstack floating ip create public --port "$port_vxlan" -f value -c 'floating_ip_address' )"

# wait until the guest kernel starts its network stack

ping -c3 "$ip_geneve"
ping -c3 "$ip_vxlan"

At this point what I see is that the tenant network type does not matter. The ping works for both types, in the same project scenario. It fails for both types in the different project scenario.

The latter seems buggy to me.

[Bence Romsics (bence-romsics)]

By the way I believe the cloud-init problems you have are unrelated and should be checked independently.

[Rodolfo Alonso (rodolfo-alonso-hernandez)]

Hello:

About the bug description. I can't reproduce the issue reported. I've deployed master branch and I can use VLAN, VXLAN or Geneve as tenant network without any issue. The metadata agent creates the corresponding namespaces and provides the metadata to the cloud-init requests without any issue. IMO, but this is just a surmise, the problem could be in the OVN metadata agent you have deployed. Please check the logs and report them here if there is any issue.

As Bence commented, if the issue is in the OS cloud-init, then you need further investigation on the OS itself but not related to Neutron.

About c#3 comment, I can't neither reproduce that. I've used Geneve and VXLAN networks, shared from a project to another. The VMs created from the project owner and the other one, are both working when connected to a floating IP. The only missing step in c#3 I would comment is the SG rule needed to accept ping packets from an external device (I'm pinging from the host, that is an IP that doesn't belong to the SG remote group):
  # openstack security group rule create --ethertype IPv4 --protocol icmp --ingress $sg

Once added the SG rule, I can ping to the floating IP of the VMs from both projects (owner of the network, project using the shared network).

I'll keep this bug as "status=invalid" unless new logs/evidences/reproducers are reported.

Regards.