On new port, traffic flow is allowed before security groups are programmed

[Mickey Spiegel]

Description:
During the creation of a neutron port, in the ovs_neutron_agent, traffic flow is enabled shortly before security groups are programmed.

File: neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py
Funtion: process_network_ports

Step-by-step:
During the creation of a neutron port, the following calls are made:
- treat_devices_added_or_updated
- sg_agent.setup_port_filters
- _bind_devices

Before early November, process_network_ports called sg_agent.setup_port_filters before it called _bind_devices. This meant that security groups were programmed before traffic flow is enabled by _bind_devices, which sets the port-lvm mapping in br-int.

Bug #1512636 reversed this order of operation, so that _bind_devices is called before sg_agent.setup_port_filters. This opens up a brief security hole, allowing traffic to flow for a short time before security groups are applied.

Proposed solution:
Revert bug# 1512636

[Tristan Cacqueray (tristan-cacqueray)]

Since this report concerns a possible security risk, an incomplete security advisory task has been added while the core security reviewers for the affected project or projects confirm the bug and discuss the scope of any vulnerability along with potential solutions.

How big is the window when port is not filtered ?

[Salvatore Orlando (salvatore-orlando)]

The bug report identifies correctly an issue with the fix for bug #1512636
Several patches in the past attempted to invert the process of securing and wiring a port, but they were fixed in the review process.
This one unfortunately slipped through the net.

While there is a security vulnerability it is worth noting that:
- there is a very small amount of time in which the port is left wired but unsecured.
- the default neutron configuration won't allow anything to boot on that port anyway.
- even if the VIF plug notification mechanism is disabled, is quite unlikely that an instance is already booted or in advanced stage in the boot process when this stage is reached.

A potential serious security implication is that if in the future a bug introduced in the code that sets up security groups causes the process to fail there is a slight chance of having a port wired but unsecured.

Considering the above, my opinion is that the possibilities of conducting a successful attack using this vulnerability are very small. Even if such vector is found the ability of perpetrating any actual damage is still questionable considering the very short time interval where a port is left unsecured.

I would suggest opening up this bug.

[Kevin Benton (kevinbenton)]

I'm setting the priority to low for this one because even though this allows a small window where an attacker could spoof IP traffic using any arbitrary IP, the ARP spoofing protection will prevent it from responding to ARP requests for any other IPs than its own so no other devices will inadvertently send traffic to it.

[Kevin Benton (kevinbenton)]

I will wait to post the revert patch here until the decision is made to open up the bug or not. If it's opened, I'll just use gerrit.

[Tristan Cacqueray (tristan-cacqueray)]

I believe this bug would be better taken care of publicly since the knowledge of this flaw doesn't give new abilities to a malicious user.

[Jeremy Stanley (fungi)]

I agree this is probably class C1 in our taxonomy ("not considered a practical vulnerability" https://security.openstack.org/vmt-process.html#incident-report-taxonomy ) and so would not get a security advisory. As such there's not much reason to maintain a secret embargo. Mickey, do you object to us making this report public prior to fixing?

[Kevin Benton (kevinbenton)]

To answer the earlier question about how big the window is, in my testing it was less than <500ms so I had to add a sleep statement to see what was possible. In an extremely heavily loaded compute server, the calculation of iptables rules and the call to iptables could increase this delay to a couple of seconds, but it would be highly unlikely for it to exceed 10 seconds.

[Mickey Spiegel (emspiege)]

I agree with all of the comments above. It is a small window during which the port is not secured. No objections on my side to making this report public

[Tristan Cacqueray (tristan-cacqueray)]

I've removed the privacy settings and put the OSSA tasks as Won't Fix since it's a C1 type of bug (according to VMT taxonomy https://security.openstack.org/vmt-process.html#incident-report-taxonomy ), This can be put back to incomplete if the situation changes.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/268192

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/liberty
Review: https://review.openstack.org/268194

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/268192
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=29dffc084164c29b4726dc08bb986f7961e9fa9f
Submitter: Jenkins
Branch: master

commit 29dffc084164c29b4726dc08bb986f7961e9fa9f
Author: Kevin Benton <email address hidden>
Date: Fri Jan 15 15:22:03 2016 +0000

    Revert "Change function call order in ovs_neutron_agent."

    This reverts commit 9c72bac0ea37971b2d5430246295c5e8b859b4ed.

    Change-Id: I9b6b588b68f63f6688749d011dc8b20ef80edadc
    Closes-Bug: #1534322

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/268194
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=ca193d023285e5b5b1b07a5f673a9864e75a2d8d
Submitter: Jenkins
Branch: stable/liberty

commit ca193d023285e5b5b1b07a5f673a9864e75a2d8d
Author: Kevin Benton <email address hidden>
Date: Fri Jan 15 15:22:43 2016 +0000

    Revert "Change function call order in ovs_neutron_agent."

    This reverts commit c5629d59e3699cbee0adbbd3c4b296431294d307.

    Change-Id: Ie8779fae4210e71e8a0416ecd35b97fe35d4276e
    Closes-Bug: #1534322

[Thierry Carrez (ttx)]

This issue was fixed in the openstack/neutron 8.0.0.0b2 development milestone.

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/neutron 7.0.2 release.