Openstack multi-homing fails when using port security

[Joseph Phillips]

Initial support for allowing Juju to work with multiple O7k networks was added in this patch:
https://github.com/juju/juju/pull/14848

As of this change, using multiple networks is only possible with port security disabled. The reasons for this are described here:
https://superuser.openstack.org/articles/managing-port-level-security-openstack/

In order to work with port security, we need enhancement to allow host NIC MAC/address pairs to pass through the public IP port.

[Matteo Panella (mpanella)]

The article on Superuser only applies when an instance needs to use different IP/MAC addresses on a single port, though, in which case the solution is to modify the allowed_address_pairs attribute of the port to whitelist the additional addresses. When using multiple networks like Juju does in multihoming setups, machines gets one neutron port for each network being referenced in the model, so modifying allowed_address_pairs on the ports is not useful.

The real issue is that Juju does *not* add security groups as it does when using a single network, all ports get created only with the default security group which (unless modified by the user) does not allow any kind of ingress traffic from external networks.

The fact that disabling port security makes traffic flow again is not related to the anti-spoofing rules at all, it's just that disabling port security _also_ disables security group processing altogether.