[ipv6] when slaas is setup as ipv6_address_mode ipv6-icmp packets are rejected by iptables

[Andrey Grebennikov]

Mitaka and Newton

Setting up subnet with ipv6 addressing for provider network (baremetal external router providing RA).
Expecting the advertising packets to be able to reach out to instance so that the instance can pick the subnet from the external router.

What happens:
Iptables on the compute node is only set up to allow certain types of ipv6-icmp:

-A neutron-linuxbri-i4d4602ea-3 -p ipv6-icmp -m icmp6 --icmpv6-type 130 -j RETURN
-A neutron-linuxbri-i4d4602ea-3 -p ipv6-icmp -m icmp6 --icmpv6-type 131 -j RETURN
-A neutron-linuxbri-i4d4602ea-3 -p ipv6-icmp -m icmp6 --icmpv6-type 132 -j RETURN
-A neutron-linuxbri-i4d4602ea-3 -p ipv6-icmp -m icmp6 --icmpv6-type 135 -j RETURN
-A neutron-linuxbri-i4d4602ea-3 -p ipv6-icmp -m icmp6 --icmpv6-type 136 -j RETURN

while RA type is 134.
The list of available types most likely has to be extended in the Neutron constants or some deeper logic has to be implemented.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/431238

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/431379

[John Davidge (john-davidge)]

Hi Andrey, what is your subnet ipv6_ra_mode set to? And what scope is your subnet gateway_ip?

Thanks

[Andrey Grebennikov (agrebennikov)]

John,
ipv6_ra_mode was either empty or slaas, it didn't make any change.
Gateway IP was ::1 from the subnet.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/432506

[John Davidge (john-davidge)]

Thanks Andrey, ipv6_ra_mode needs to be empty, and the subnet gateway ip needs to be the link-local address from which the external router is sending its RAs. Could you try that and see if it works? Thanks.

[Andrey Grebennikov (agrebennikov)]

John,
Let me clarify.
As I mentioned before - ipv6_ra_mode was EITHER empty or "slaas" (I mean I tried both).
Subnet gateway IP was <ipv6 subnet>::1.
And no, this didn't work.

[Kevin Benton (kevinbenton)]

I suspect the problem is that it's not a link-local address on the subnet and Neutron doesn't have any way to actually know what the link local address is that a provider router will use.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Kevin Benton (<email address hidden>) on branch: master
Review: https://review.openstack.org/431379
Reason: Let's proceed with https://review.openstack.org/432506

[John Davidge (john-davidge)]

Andrey,
Please try the following:

Make sure ipv6_ra_mode is empty
Set the subnet gateway IP to the LLA of the provider router issuing the RAs.

Your instances should receive RAs with the above configuration. Please let me know if that fixes the problem, thanks.

[Andrey Grebennikov (agrebennikov)]

John,
As I mentioned in the previous comment:
>>> As I mentioned before - ipv6_ra_mode was EITHER empty or "slaas" (I mean I tried both).
>>> Subnet gateway IP was <ipv6 subnet>::1 (Meaning that was set to the ipv6 of the baremetal router).
Just to triple confirm for you - no, it doesn't work. It only started to work when I applied dirty hack to the code and added the new port to iptables in order to allow RA packets to come in.

[John Davidge (john-davidge)]

Andrey, it will not work with the subnet configured as you describe.

ipv6_ra_mode MUST BE empty. slaac will not work.

gateway_ip MUST BE the link-local address of the router, i.e. from the fe80::/64 prefix. <ipv6_subnet>::1 will not work.

If you're able to try with the described subnet settings it will help us a lot with diagnosing the bug. Thanks.

[Kevin Benton (kevinbenton)]

This bug has had a related patch abandoned and has been automatically un-assigned due to inactivity. Please re-assign yourself if you are continuing work or adjust the state as appropriate if it is no longer valid.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Kevin Benton (<email address hidden>) on branch: stable/mitaka
Review: https://review.openstack.org/431238
Reason: This review is > 4 weeks without comment and currently blocked by a core reviewer with a -2. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and contacting the reviewer with the -2 on this review to ensure you address their concerns.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/432506
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=ae9d1160bdaedc23dd6a1a0a85c09b8e65422a13
Submitter: Jenkins
Branch: master

commit ae9d1160bdaedc23dd6a1a0a85c09b8e65422a13
Author: Kevin Benton <email address hidden>
Date: Thu Feb 9 00:27:20 2017 -0800

    Stop making IP-specific provider rules in SG code

    Setting up rules to allow DHCPv6, DHCP, and RAs from specific
    IP addresses based on Neutron resources has a few issues:

    1. It violates separation of concerns. We are implementing logic to
       calculate where an IPv6 RA advertisement or DHCP advertisement
       should be coming from in the security group code. This code should
       not be trying to guess IPv6 LLAs, know about subnet modes, DHCP server
       implementations, or the type of L3 plugin being used. Currently all
       of these assumptions are baked into code that should only be
       filtering, which makes it very rigid and brittle when it comes to
       other implementations for DHCP and/or RAs.
    2. It has scaling issues on large networks. Every time one of these
       provider rules is updated, it triggers every L2 agent to refresh
       all of the security group rules for ports in that network, which puts
       significant load on the server.
    3. It's main purpose: preventing spoofing of RA[1,2] and DHCP packets,
       has long been superceded by preventing VMs from acting as DHCP/RA
       servers[3][4].

    This patch completely removes all of this logic and just returns
    static provider rules to the agents that allow all DHCP server
    and RA traffic ingress to the client. This addresses the issues
    highlighted above since the code is significantly simplified and
    the provider rules don't require refreshes on the agents.

    Now that the provider rules never change, the RPC notification
    listener on the agent-side for 'notify_provider_updated' is now
    just a NOOP that doesn't trigger any refreshes. The notification
    was left in place on the server side for older version agents
    that have stale IP-specific provider rules. The entire notification
    can be removed in the future.

    The one open concern with this approach is that VMs will now be
    able to receive DHCP offers from other DHCP servers on the same
    network that aren't being filtered (e.g. a VM with port security
    disabled or another device on a provider network). In order to
    address this for DHCP, this patch adds two rules that only allow
    DHCP offers targeted to either the broadcast or the correct client
    IP. This prevents incorrect offers from ever reaching the client.
    For RAs, this patch just allows all RAs so we may pick up
    advertisements from other v6 routers attached to a network;
    however, the instance won't actually be allowed to use bad addresses.

    1. https://bugs.launchpad.net/neutron/+bug/1262759
    2. I1d5c7aaa8e4cf057204eb746c0faab2c70409a94
    3. Ice1c9dd349864da28806c5053e38ef86f43b7771
    4. https://git.openstack.org/cgit/openstack/neutron/tree/
       neutron/agent/linux/iptables_firewall.py
       ?h=521b1074...

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/456745

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 11.0.0.0b1 development milestone.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/456745
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=c5fa966e8fe8e770ee9a367428df4be698e3edc0
Submitter: Jenkins
Branch: stable/ocata

commit c5fa966e8fe8e770ee9a367428df4be698e3edc0
Author: Kevin Benton <email address hidden>
Date: Thu Feb 9 00:27:20 2017 -0800

    Stop making IP-specific provider rules in SG code

    Setting up rules to allow DHCPv6, DHCP, and RAs from specific
    IP addresses based on Neutron resources has a few issues:

    1. It violates separation of concerns. We are implementing logic to
       calculate where an IPv6 RA advertisement or DHCP advertisement
       should be coming from in the security group code. This code should
       not be trying to guess IPv6 LLAs, know about subnet modes, DHCP server
       implementations, or the type of L3 plugin being used. Currently all
       of these assumptions are baked into code that should only be
       filtering, which makes it very rigid and brittle when it comes to
       other implementations for DHCP and/or RAs.
    2. It has scaling issues on large networks. Every time one of these
       provider rules is updated, it triggers every L2 agent to refresh
       all of the security group rules for ports in that network, which puts
       significant load on the server.
    3. It's main purpose: preventing spoofing of RA[1,2] and DHCP packets,
       has long been superceded by preventing VMs from acting as DHCP/RA
       servers[3][4].

    This patch completely removes all of this logic and just returns
    static provider rules to the agents that allow all DHCP server
    and RA traffic ingress to the client. This addresses the issues
    highlighted above since the code is significantly simplified and
    the provider rules don't require refreshes on the agents.

    Now that the provider rules never change, the RPC notification
    listener on the agent-side for 'notify_provider_updated' is now
    just a NOOP that doesn't trigger any refreshes. The notification
    was left in place on the server side for older version agents
    that have stale IP-specific provider rules. The entire notification
    can be removed in the future.

    The one open concern with this approach is that VMs will now be
    able to receive DHCP offers from other DHCP servers on the same
    network that aren't being filtered (e.g. a VM with port security
    disabled or another device on a provider network). In order to
    address this for DHCP, this patch adds two rules that only allow
    DHCP offers targeted to either the broadcast or the correct client
    IP. This prevents incorrect offers from ever reaching the client.
    For RAs, this patch just allows all RAs so we may pick up
    advertisements from other v6 routers attached to a network;
    however, the instance won't actually be allowed to use bad addresses.

    1. https://bugs.launchpad.net/neutron/+bug/1262759
    2. I1d5c7aaa8e4cf057204eb746c0faab2c70409a94
    3. Ice1c9dd349864da28806c5053e38ef86f43b7771
    4. https://git.openstack.org/cgit/openstack/neutron/tree/
       neutron/agent/linux/iptables_firewall.py
       ?h=52...

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 10.0.2 release.