Issue with security groups and floating ips on the same host

[Bartosz Kupidura]

If we have many VMs on single compute-node, security groups with floating-ip as CIDR does not work properly.

VM1:
* fixed_ip = 10.10.10.1
* floating_ip = 192.168.10.1

VM2:
* fixed_ip = 10.10.10.2
* floating_ip = 192.168.10.2

Sec group:
$ nova secgroup-add-rule default tcp 22 22 192.168.10.1
$ nova secgroup-add-rule default tcp 22 22 192.168.10.2

Traffic from VM1 to VM2 float IP is dropped.

Security group chain is created in FORWARD iptables chain, and source nat (fixed_ip->float_ip) is created in postrouting - http://www.linuxhomenetworking.com/wiki/index.php/File:Iptables.gif

If VMs are on different compute-nodes everythings works perfect.

[Vish Ishaya (vishvananda)]

Traffic from vm to vm should be going across the fixed ips always, so I'm not surprised this doesn't work. Any ideas on how we could construct the table differently to make this work? Do we need to add an output rule in addition to a FORWARD rule?

[Vish Ishaya (vishvananda)]

I'm going to try to verify this one

[Vish Ishaya (vishvananda)]

So this is actually very difficult to fix. It definitely won't be done for folsom and may not be done at all. I'm going to put this in the Folsom release notes as a known issue with the following workaround:

The recommendation is to use fixed ip cidrs or source groups. Source groups should work with floating ips as long as you specify a dmz_cidr=x.x.x.x/x for each of your floating ranges in your conf file. This will make sure that vm -> vm traffic is not snatted and the source groups should work