IPv6 not working with iptables

[Junien F]

Hi,

Running rocky on Ubuntu 18.04 deployed by juju, using ML2, ovs, iptables. IPv6 appears to be broken because of missing MARK-related rules in the qrouter netns.

The iptables and ip6tables rules generated by neutron are https://pastebin.ubuntu.com/p/S32TQcmTzX/

For egress (traffic leaving an instance) to work, the following additional rule is needed :
sudo ip6tables -t mangle -I neutron-l3-agent-POSTROUTING -o qg-45ba891c-4c -m connmark --mark 0x0/0xffff0000 -j CONNMARK --save-mark --nfmask 0xffff0000 --ctmask 0xffff0000

The following patch should fix the problem : https://pastebin.ubuntu.com/p/RpbYBjCVnp/ (sorry, I don't have time right now to update the tests for a proper merge request)

For ingress, the following is needed :
sudo ip6tables -t mangle -A neutron-l3-agent-scope -i qg-45ba891c-4c -j MARK --set-xmark 0x4000000/0xffff0000

Haven't had the time to dig out in the code where exactly the bug is.

Is IPv6 working for anyone with this setup ? Are these commands the right fix ? (I'm just mimicking what IPv4 does)

I've looked at unit tests for my patch above, and IPv6 testing is extremely limited.

My IPv6 subnet got created with :
$ openstack subnet create --network net_instances --ip-version 6 --ipv6-address-mode=slaac --ipv6-ra-mode=slaac --allocation-pool start=<ipv6start>,end=<ipv6end> --subnet-range <ipv6net>::/64 --gateway <ipv6gateway> subnet_instances_v6

Thanks

[Brian Haley (brian-haley)]

I think that rule will get added when a prefix-delegated subnet is added to the external interface, or when you add a prefix from a configured address pool. I don't think modifying that code is exactly correct at first glance.

Maybe you can describe more details about your config so we can re-create it?

[Junien F (axino)]

Sure, I'll provide that.

However, note that for the first case, I just couldn't find a code path that would add such a rule (didn't dig in the second case yet).

[Junien F (axino)]

Setup without IPv6 : https://pastebin.ubuntu.com/p/Nqr33HQygG/

Then to add IPv6, I just did the following :
$ openstack subnet create --network net_instances --ip-version 6 --ipv6-address-mode=slaac --ipv6-ra-mode=slaac --allocation-pool start=<ipv6start>,end=<ipv6end> --subnet-range <ipv6net>::/64 --gateway <ipv6gateway> subnet_instances_v6

$ openstack router add subnet router_instances subnet_instances_v6

Let me know if you have further questions.

Thanks

[Brian Haley (brian-haley)]

The prefix delegation code I mentioned is in _process_pd_iptables_rules(), and it's called when a port with an address scope is added to a router.

In your case you are not adding such a port, instead you are just adding an IPv6 subnet to the private network, then adding an interface for the router in the subnet. That will not explicitly make that subnet routable since the internal "address scope" is different than the external address scope (on the external network).

Here are a couple of links to both address scopes and IPv6 configuration:

https://docs.openstack.org/neutron/rocky/admin/config-address-scopes.html
https://docs.openstack.org/neutron/rocky/admin/config-ipv6.html

In the second there is a section on Prefix Delegation, which is what I would recommend if your infrastructure supports it (i.e. upstream router running a PD server). If not then you should be able to use address scopes and subnet pools such that all the IPv6 subnets are allocated from the same pool (internal and external), and therefore have the same address scope. This should cause the missing rule to get added when the router is added to the internal subnet.

[Launchpad Janitor (janitor)]

[Expired for neutron because there has been no activity for 60 days.]