[OSSA-2016-009] ICMPv6 anti-spoofing rules are too permissive (CVE-2015-8914)

[xens]

ICMPv6 default firewall rules are too permissive on the hypervisors leaving VMs able to do ICMPv6 source address spoofing.

Pre-condition:

- having a provider-network providing IPv6 connectivity to the VMs
- in my case the controllers are providing statefull DHCPv6 and my physical router provides the default gateway using Router Advertisements.

How to reproduce:

- spin a VM and attach to it an IPv6 enabled network
- obtain an IPv6 address using #dhclient -6
- try to ping6 an IPv6 enabled host
- remove your IPv6 address from the interface: #sudo ip addr del 2001:0DB8::100/32 dev eth0
- add a forged IPv6 address to your interface, into the same subnet of the original IPv6 address: #sudo ip addr add 2001:0DB8::200/32 dev eth0
- try to ping6 the previous IPv6 enabled host, it will still work
- try to assign another IPv6 address to your NIC, completely outside your IPv6 assignment: sudo ip addr add 2001:dead:beef::1/64 dev eth0
- try to ping6 the previous IPv6 enabled host -> the destination will still receive your echo requests with your forget address but you won't receive answers, they won't be router back to you.

Expected behavior:

- VMs should not be able to spoof their IPv6 address and issue forged ICMPv6 packets. The firewall rules on the hypervisor should restrict ICMPv6 egress to the VMs link-local and global-unicast addresses.

Affected versions:

- I saw the issue into OpenStack Juno, under Ubuntu 14.04. But according to the upstream code, the issue is still present into the master branch, into; neutron/agent/linux/iptables_firewall.py, into line 385:

ipv6_rules += [comment_rule('-p icmpv6 -j RETURN', comment=ic.IPV6_ICMP_ALLOW)]

[Jeremy Stanley (fungi)]

Since this report concerns a possible security risk, an incomplete security advisory task has been added while the core security reviewers for the affected project or projects confirm the bug and discuss the scope of any vulnerability along with potential solutions.

[Salvatore Orlando (salvatore-orlando)]

In this reports 2 different levels of address forging are discussed:

1 - forging with another address from the same subnet
2 - forging with an address outside the assigned subnet

In case #2 even if potentially there is a bug to investigate, there is not a security vulnerability imho.
For Case #1 I need to verify the behaviour independently (or get someone to do that), since it is a minor security vulnerability on shared networks as one tenant might impersonate another tenant's VM. Please note that this would happen if the operator did not assign each tenant a distinct security group, which is quite unlikely.

I'm inclined to say that this should not be granted an OSSA, but this is pending further verification on the behaviour reported.

[xens (r-aviolat)]

Salvatore,

In our setup we have a public, shared provider-network where all tenants can obtain public-IP resources, both v4 and v6 subnets are setup on top of that network. For IPv6, users on multiple tenants share the same subnet (/64).

Case #1; each tenant have their own security groups in my setup but they are on the same shared v6 subnet; yes they can impersonate another tenant's VM.
Case #2; a malicious user could also impersonate or perturb any others tenant's VMs it's ihmo more critical than Case #1 from a usability point-of-view.

[Tristan Cacqueray (tristan-cacqueray)]

Unless someone complains about it, let's make this bug report public by the end of the week.

[Travis McPeak (travis-mcpeak)]

I'm having trouble wrapping my head around this one. Can somebody knowledgable about this please help me understand:

1) The potential impact in above mentioned scenarios?
2) What would a fix look like? Specifically which component(s) should be updated and how?

[Robert Clark (robert-clark)]

repudiation probably, ICMPv6 can be used in some DDoS scenarios, a cloud provider that cannot identify where ICMP is originating from within it's network (wether the DOS is outbound or targeted at something else internal) will probably have a bad time.

I don't think it's super critical though and can probably be fixed in the open

[Travis McPeak (travis-mcpeak)]

Are there issues aside from the one which Rob mentioned?

If not we I agree, let's fix this in the open.

[Salvatore Orlando (salvatore-orlando)]

We have a related bug report filed here: https://bugs.launchpad.net/ossa/+bug/1515444

Tristan mentioned it could a duplicated, and indeed it might be.
If the underlying root cause is that we fail to apply the baseline DENY ALL security rule in the v6 case, then the extent of this issue might go well beyond DOS, and therefore should be granted an OSSA.

I think in the other bug report Brian Haley mentioned that he will do some tests to repro the issue and therefore I'd wait for his results before making a call on this bug.

[Tristan Cacqueray (tristan-cacqueray)]

Salvatore, turns out the related issue have been invalidated. At least DVR doesn't seems to be the culprit here.
Is someone still trying to reproduce this ?

[Tristan Cacqueray (tristan-cacqueray)]

If it's ok with reporter and neutron-coresec team, we'd like to open this bug report for public review by the end of the week.

[xens (r-aviolat)]

It's ok for me

[Salvatore Orlando (salvatore-orlando)]

Tristan, as far as I am aware there is no activity going on this bug atm.
IMHO this can be opened up just as it was done for bug 1515444

[Tristan Cacqueray (tristan-cacqueray)]

Is this happening with vlan/vxlan ? I wonder if it's not related to bug 1534652...

[Armando Migliaccio (armando-migliaccio)]

failed to triage

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/300233

[Tristan Cacqueray (tristan-cacqueray)]

bug 1558658 points this bug as being the IPv6 case of the same flaw, but only ICMPv6 is discussed here.
Are DHCP discovery message also not protected in IPv6 ?

So would it makes sense to issue a single advisory for both bugs or are they different beast ?

[Dustin Lundquist (dlundquist)]

@Tristan, the antispoofing rules (prior to 299021 which just merged) has permitted a number of types of spoofing attacks:
1. Source MAC spoofing -- this could allow an attacker to cause physical and virtual switches to learn an incorrect destination of MAC addresses allowing an attacker to intercept traffic destine to another instance on the local Neutron network. In the case of a shared multi-tenant network this allowed cross tenant traffic inception.
2. Source IP spoofing in IPv4 DHCP requests -- I'm not aware of an attack which would permit accessing another instances traffic, but could be used to mask the source of a DoS attack.
3. Source IP spoofing of IPv6 DHCP requests -- same as #2.
4. Source IP spoofing of ICMPv6 traffic -- this could allow an attacker to participate in neighbor discovery on the behalf of any other host on the local network, allowing it to receive traffic destine to another instance on the local Neutron network. Same multi tenant concerns as #1.

[xens (r-aviolat)]

Tristan,

this bug was a bit different originally as it was concerning only the ICMPv6 traffic, but it looks like that Dustin also enforced the DHCP traffic on the same occasion so it make sense for me to make a single advisory.

[Dustin Lundquist (dlundquist)]

I don' think an additional advisory is necessary. , but the ICMPv6 Neighbor Discovery aspect of this bug is the IPv6 case of bug 1274034 rather than bug 1558658.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/301977

[Tristan Cacqueray (tristan-cacqueray)]

Just to be clear, the ICMPv6 source address spoof isn't addressed by bug 1558658 patch (I39dc0e23fc118ede19ef2d986b29fc5a8e48ff78).

Since both issues abuse the same fundamental flaw, it seems like a good opportunity to bundle both fix in a single advisory.

However, because we need different patch, this will likely requires 2 different CVE numbers...

Does it makes sense ?

[xens (r-aviolat)]

yes it does

[Simon Leinen (simon-leinen)]

I would love to see this backported to Liberty and Kilo. Other than the security issues, it creates a dataplane/usability issue for guest OSes that use RFC 4941 privacy addresses by default, such as Windows.

The bug has the following effect: Normally, IPv6 privacy addresses should not work, because they cannot be known to Nova/Neutron, and thus they should be caught by the anti-spoofing filters.

But because of the bug, the anti-spoofing filters let all ICMPv6 through. So for example, on an instance running standard Windows 2012 Server, "ping6 ipv6.google.com" will work, but one cannot surf to http://ipv6.google.com. The built-in Windows network check will misdiagnose this as "server is online but refuses to respond to HTTP connections".

Of course it is true that OSes that use privacy addresses by default aren't suitable for out-of-the-box use under OpenStack with IPv6—privacy addresses must be disabled to get usable IPv6 connectivity. But legacy images exist, and because of the special ICMP behavior, it is hard to find the cause of the problem.

[Dustin Lundquist (dlundquist)]

@Simon

The IPv6 networking guide calls out RFC 4941 privacy extensions as being not supported. With only ICMP being permitted from privacy addresses is it really a regression because Microsoft tries to provide an automated network diagnostic? I don't think it is worth while permitted source IP spoofing to improve the automated error text provided my Microsoft's products.

[Simon Leinen (simon-leinen)]

@dlundquist

Sorry for being unclear. I don't ask for spoofing to be permitted. I want anti-spoofing to work for all protocols including ICMP. Then Microsoft's network diagnostics won't be confused anymore. (And also things will be a bit more secure.)

[Dustin Lundquist (dlundquist)]

@Simon, that is what the patch does: it permits very limited use of the unspecified address as source to enable DAD, then enforces anti-spoofing rules and then permits ICMP once anti-spoofing chain has been applied. What, if any, documentation do you think is required to alert users changed behavior of IPv6 privacy address (previously permitting only ICMP, and now blocking all traffic from privacy addresses)?

Existing documentation indicating privacy addresses are not supported: http://docs.openstack.org/liberty/networking-guide/adv_config_ipv6.html (under Security Considerations). I meant to include this link in my previous post.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/300233
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=a8a9d225d8496c044db7057552394afd6c950a8e
Submitter: Jenkins
Branch: master

commit a8a9d225d8496c044db7057552394afd6c950a8e
Author: Dustin Lundquist <email address hidden>
Date: Thu Mar 31 15:19:04 2016 -0700

    IPtables firewall prevent ICMPv6 spoofing

    IPv6 includes the concept of link-local addresses. There are address
    within the fe80::/64 prefix which are used only within the local layer 2
    network. They should never be routed. DHCPv6 is one of several protocols
    which utilize link-local addresses.

    Previously the blanket permit DHCPv6 rule permitted DHCPv6 requests from
    a link-local source, before the source address was validated.

    The structure of the IPtables egress firewall is:

      a. fixed rules for special traffic
      b. validate source address
      c. fixed rules necessary for host to function
      d. user rules defined by security groups

    This change restricts the special traffic permitted in part (a) to only
    that traffic which utilizes the "unspecified address" (::), by moving
    the fixed permit ICMPv6 and DHCPv6 rules to part (c), so they are
    applied after the source address has been validated. In order to enable
    DHCPv6 and other protocols utilizing link-local addresses, the
    link-local address corresponding to each MAC address are included in the
    permitted source addresses. After the source address is verified, the
    fixed rules permit ICMPv6 and DHCPv6, then the user defined security
    group rules are applied.

    In the existing implementation ICMPv6 and DHCPv6 rules in the fixed
    ip6tables firewall rules are too permissive: they permit ICMPv6 and
    DHCPv6 traffic, regardless of source MAC or IPv6 address. These rules
    where intended to allow a host to acquire an IPv6 address, but
    inadvertently allowed a malicious or compromised host to spoof another's
    MAC or IPv6 address.

    A host acquiring an IPv6 address should preform DAD (duplicate address
    detection). To preform this the host must join the multicast group
    corresponding to the tentative IPv6 address and the all nodes multicast
    group. To join these groups the host sends ICMP MLD (multicast listener
    discovery) report messages before it has an IPv6 address assigned, so
    the unspecified address is used as the source address. To complete DAD,
    ICMP neighbor solicitation messages are sent to solicit if any nodes
    using that address. This should be the only use of the unspecified IPv6
    address as a source address. The IPv4 case is similar the unspecified
    address is used for DHCP discovery and request messages.

    To summarize, this patch permits only ICMPv6 traffic from the unspecified
    address which is used for duplicate address detection. Then it enforces
    the source IPv6 and MAC addresses and finally, allows only ICMPv6 traffic
    which has passed this source address validation.

    In addition this patch permits traffic from all link-local addresses
    associated with each MAC address assigned t...

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/310648

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/liberty
Review: https://review.openstack.org/310652

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/315828

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/310652
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=1d1159bb2b57f0b4193f8666f53736f05bf7eac9
Submitter: Jenkins
Branch: stable/liberty

commit 1d1159bb2b57f0b4193f8666f53736f05bf7eac9
Author: Dustin Lundquist <email address hidden>
Date: Thu Mar 31 15:19:04 2016 -0700

    IPtables firewall prevent ICMPv6 spoofing

    IPv6 includes the concept of link-local addresses. There are address
    within the fe80::/64 prefix which are used only within the local layer 2
    network. They should never be routed. DHCPv6 is one of several protocols
    which utilize link-local addresses.

    Previously the blanket permit DHCPv6 rule permitted DHCPv6 requests from
    a link-local source, before the source address was validated.

    The structure of the IPtables egress firewall is:

      a. fixed rules for special traffic
      b. validate source address
      c. fixed rules necessary for host to function
      d. user rules defined by security groups

    This change restricts the special traffic permitted in part (a) to only
    that traffic which utilizes the "unspecified address" (::), by moving
    the fixed permit ICMPv6 and DHCPv6 rules to part (c), so they are
    applied after the source address has been validated. In order to enable
    DHCPv6 and other protocols utilizing link-local addresses, the
    link-local address corresponding to each MAC address are included in the
    permitted source addresses. After the source address is verified, the
    fixed rules permit ICMPv6 and DHCPv6, then the user defined security
    group rules are applied.

    In the existing implementation ICMPv6 and DHCPv6 rules in the fixed
    ip6tables firewall rules are too permissive: they permit ICMPv6 and
    DHCPv6 traffic, regardless of source MAC or IPv6 address. These rules
    where intended to allow a host to acquire an IPv6 address, but
    inadvertently allowed a malicious or compromised host to spoof another's
    MAC or IPv6 address.

    A host acquiring an IPv6 address should preform DAD (duplicate address
    detection). To preform this the host must join the multicast group
    corresponding to the tentative IPv6 address and the all nodes multicast
    group. To join these groups the host sends ICMP MLD (multicast listener
    discovery) report messages before it has an IPv6 address assigned, so
    the unspecified address is used as the source address. To complete DAD,
    ICMP neighbor solicitation messages are sent to solicit if any nodes
    using that address. This should be the only use of the unspecified IPv6
    address as a source address. The IPv4 case is similar the unspecified
    address is used for DHCP discovery and request messages.

    To summarize, this patch permits only ICMPv6 traffic from the unspecified
    address which is used for duplicate address detection. Then it enforces
    the source IPv6 and MAC addresses and finally, allows only ICMPv6 traffic
    which has passed this source address validation.

    In addition this patch permits traffic from all link-local addresses
    associated with each MAC address as...

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/310648
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=3e66b1a87544d7a127abceec13bfeacb8f18f7e1
Submitter: Jenkins
Branch: stable/mitaka

commit 3e66b1a87544d7a127abceec13bfeacb8f18f7e1
Author: Dustin Lundquist <email address hidden>
Date: Thu Mar 31 15:19:04 2016 -0700

    IPtables firewall prevent ICMPv6 spoofing

    IPv6 includes the concept of link-local addresses. There are address
    within the fe80::/64 prefix which are used only within the local layer 2
    network. They should never be routed. DHCPv6 is one of several protocols
    which utilize link-local addresses.

    Previously the blanket permit DHCPv6 rule permitted DHCPv6 requests from
    a link-local source, before the source address was validated.

    The structure of the IPtables egress firewall is:

      a. fixed rules for special traffic
      b. validate source address
      c. fixed rules necessary for host to function
      d. user rules defined by security groups

    This change restricts the special traffic permitted in part (a) to only
    that traffic which utilizes the "unspecified address" (::), by moving
    the fixed permit ICMPv6 and DHCPv6 rules to part (c), so they are
    applied after the source address has been validated. In order to enable
    DHCPv6 and other protocols utilizing link-local addresses, the
    link-local address corresponding to each MAC address are included in the
    permitted source addresses. After the source address is verified, the
    fixed rules permit ICMPv6 and DHCPv6, then the user defined security
    group rules are applied.

    In the existing implementation ICMPv6 and DHCPv6 rules in the fixed
    ip6tables firewall rules are too permissive: they permit ICMPv6 and
    DHCPv6 traffic, regardless of source MAC or IPv6 address. These rules
    where intended to allow a host to acquire an IPv6 address, but
    inadvertently allowed a malicious or compromised host to spoof another's
    MAC or IPv6 address.

    A host acquiring an IPv6 address should preform DAD (duplicate address
    detection). To preform this the host must join the multicast group
    corresponding to the tentative IPv6 address and the all nodes multicast
    group. To join these groups the host sends ICMP MLD (multicast listener
    discovery) report messages before it has an IPv6 address assigned, so
    the unspecified address is used as the source address. To complete DAD,
    ICMP neighbor solicitation messages are sent to solicit if any nodes
    using that address. This should be the only use of the unspecified IPv6
    address as a source address. The IPv4 case is similar the unspecified
    address is used for DHCP discovery and request messages.

    To summarize, this patch permits only ICMPv6 traffic from the unspecified
    address which is used for duplicate address detection. Then it enforces
    the source IPv6 and MAC addresses and finally, allows only ICMPv6 traffic
    which has passed this source address validation.

    In addition this patch permits traffic from all link-local addresses
    associated with each MAC address ass...

[Thierry Carrez (ttx)]

This issue was fixed in the openstack/neutron 7.1.0 release.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/301977
Committed: https://git.openstack.org/cgit/openstack/ossa/commit/?id=756540a726aa7309fcd7916c5f22ede66badeb1e
Submitter: Jenkins
Branch: master

commit 756540a726aa7309fcd7916c5f22ede66badeb1e
Author: Tristan Cacqueray <email address hidden>
Date: Tue Apr 5 19:41:31 2016 -0400

    Adds OSSA 2016-009 (CVE-2016-5362, CVE-2016-5363 and CVE-2015-8914)

    Change-Id: Iad029108209fc631da286c777e8485106cea7f53
    Related-Bug: #1502933
    Related-Bug: #1558658

[OpenStack Infra (hudson-openstack)]

Change abandoned by Armando Migliaccio (<email address hidden>) on branch: master
Review: https://review.openstack.org/315828
Reason: This review is > 4 weeks without comment, and failed Jenkins the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

[Armando Migliaccio (armando-migliaccio)]

ping?

[Tristan Cacqueray (tristan-cacqueray)]

@Armando it seems like this bug report has been addressed for IPTable firewall, perhaps the OVS issue could be address with a new bug number ?