Security groups don't work with LibvirtGenericVIFDriver driver

[Simon Pasquier]

Security groups on master branch using Neutron and OVS plugin are broken. No problem to create/delete security group rules but even though iptables configuration is updated, traffic to my instances is never filtered [0].

I'm running DevStack on 2 nodes (1 controller + 1 compute):
- OS: Ubuntu 12.04.3 (LTS) with the Havana cloud archive repository.
- Open vSwitch package version: 1.10.2-0ubuntu2~cloud0
- libvirt package version: 1.1.1-0ubuntu8~cloud2
- localrc, nova.conf, neutron.conf and ovs_neutron_plugin.ini files pasted at [1] (I didn't modify any of these files after the DevStack run)

According to [2], [3] and [4], iptables is not compatible with TAP devices connected directly to Open vSwitch ports, this is why there used to be the additional veth + bridge interfaces [5]. But in my setup, this is not the case anymore as shown in [6] ('ovs-vsctl show' + 'iptables-save' ouptut). I've also pasted the libvirt XML configuration [7] that shows that the instance is directly connected to the Open vSwitch.

[0] http://paste.openstack.org/show/50490/
[1] http://paste.openstack.org/show/50448/
[2] http://www.spinics.net/linux/fedora/libvirt-users/msg05384.html
[3] http://openvswitch.org/pipermail/discuss/2013-October/011461.html
[4] http://docs.openstack.org/havana/configreference/content/under_the_hood_openvswitch.html
[5] http://docs.openstack.org/havana/config-reference/content/figures/7/a/a/common/figures/under-the-hood-scenario-2-ovs-compute.png
[6] http://paste.openstack.org/show/50486/
[7] http://paste.openstack.org/show/50487/

[Simon Pasquier (simon-pasquier)]

The LibvirtHybridOVSBridgeDriver driver is gone on the master branch ([0]). Joe Gordon asked the Neutron devs about it few weeks ago [1] but no answer and in another review [2], the conclusion was that the Tempest tests passed with Neutron. However I don't see anywhere in the tests ([3], [4]) that we check if the security rules allow/block traffic.

[0] https://review.openstack.org/#/c/49660/
[1] http://lists.openstack.org/pipermail/openstack-dev/2013-October/016886.html
[2] https://review.openstack.org/#/c/44349
[3] https://github.com/openstack/tempest/blob/master/tempest/api/network/test_security_groups.py
[4] https://github.com/openstack/tempest/blob/master/tempest/api/network/test_security_groups_negative.py

[Yaguang Tang (heut2008)]

you need to config firewall_driver = nova.virt.libvirt.firewall.IptablesFirewallDriver, unfortunately a default devstack install now config it as firewall_driver="nova.virt.firewall.NoopFirewallDriver when using Neutron. so this may be a devstack bug.

[Salvatore Orlando (salvatore-orlando)]

Actually - running the Noop driver when neutron is enabled is intended, as nova would let Neutron configure security groups.
I guess that Simon's configuration was working before the switch to the new generic drivers.

It will be good to check what the port binding extension is returning for your neutron ports. It should instruct the generic driver to used the 'hybrid' mode (chaining a LB bridge onto the OVS integration bridge). If it just uses OVS integration bridge, sec groups won't be enforced at all in gre mode, and enforced only at the uplink in vlan mode.

Moving to incomplete waiting for more input.

[Salvatore Orlando (salvatore-orlando)]

Simon, as usual I have been lazy and did not check all your attachments.
The hybrid mode is definetely not being picked up; I will assign it to myself for further triage.

[Simon Pasquier (simon-pasquier)]

@Salvatore

Here is the output for the port where my instance is connected:

$ neutron port-show bb94ecec-6b2f-4c50-9d9c-4c1c8cb98795
+-----------------------+---------------------------------------------------------------------------------+
| Field | Value |
+-----------------------+---------------------------------------------------------------------------------+
| admin_state_up | True |
| allowed_address_pairs | |
| binding:capabilities | {"port_filter": true} |
| binding:host_id | dev-simon-oscomputegroup-43mug5ielckr-lbgl4depwimu-l232bg66dyl4 |
| binding:vif_type | ovs |
| device_id | 2bdd7563-0c97-4f1a-8795-f72667d9dbed |
| device_owner | compute:None |
| extra_dhcp_opts | |
| fixed_ips | {"subnet_id": "42a98132-6b8a-48fc-8ef5-54ee08504e30", "ip_address": "10.0.0.3"} |
| id | bb94ecec-6b2f-4c50-9d9c-4c1c8cb98795 |
| mac_address | fa:16:3e:3b:a0:d4 |
| name | |
| network_id | cba1a035-54c9-44e0-b77e-6439bd0c51df |
| security_groups | ba464270-93f6-4e51-912d-d5ffc3eea610 |
| status | ACTIVE |
| tenant_id | 633480e028b3482bb8e004e4b1514c39 |
+-----------------------+---------------------------------------------------------------------------------+

[Salvatore Orlando (salvatore-orlando)]

The output from the neutron side is correct.
Actually Yaguang correctly pointed out that the setting for the firewall driver should be different from NoOp, otherwise nova won't apply the hybrid vif plugging strategy.

In my opinion this might still be a nova bug - get_firewall_required should return true if the driver is != noop or security_group_api=neutron; I don't think it is really intuitive to require specifying a firewall driver in nova if then the sec group is implemented by neutron.

Another way of looking at it is that neutron should mandate when it needs an hybrid strategy.

But since this has impact also on nova-network, we need input from some expert on the nova side.

[Simon Pasquier (simon-pasquier)]

Note that the Havana documentation clearly states that firewall_driver in nova.conf should be noop when using Neutron security groups => http://docs.openstack.org/havana/config-reference/content/nova_config_security_groups.html

[Yoshihiro Kaneko (ykaneko0929)]

Is this related to the following bug?
  https://bugs.launchpad.net/neutron/+bug/1112912
There are non-active patches...
  https://review.openstack.org/#/c/44596/
  https://review.openstack.org/#/c/21946/

[Simon Pasquier (simon-pasquier)]

@Yoshiro, yes I think it is related.

[Jeremy Deininger (jeremydei)]

looks like this change just released in the lastest package. We had started using LibvirtHybridOVSBridgeDriver recently to address a bug where iptables were not working. Using this driver enabled iptables with libvirt, but now the package has taken it away. Suggestions? We may rollback the package but I want to know the right solution.

nova and neutron 1:2013.2.1-0ubuntu1~cloud0

[Jeremy Deininger (jeremydei)]

oops, disregard previous comment. I thought the driver was gone because there was an error about it missing, however moving the config line to the very top of the default section of nova.conf fixed the problem (from the bottom of the default section previously). Strange.