should not add default security group to quantum unless api-request had it

[Aaron Rosen]

when booting an instance nova-api automatically adds a default security group if one is not specified, though we shouldn't be doing this and instead quantum should handle be handing this. This actually causes an issue for plugins that implement the port_security_api and have port_security_enabled=False on a network.

https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/contrib/security_groups.py#L498
https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/contrib/security_groups.py#L511

[Joshua Hesketh (joshua.hesketh)]

Setting to in progress since assigned.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/59578

[Bhuvan Arumugam (bhuvan)]

Aaron, the below error occur when creating a server using nova or neutron+nova when port_security_enabled=False for the network.
SecurityGroupCannotBeApplied: Network requires port_security_enabled and subnet associated in order to apply security groups.

The error don't occur with the patch https://review.openstack.org/59578. With this patch we could create a server using nova or neutron+nova with port_security_enabled=False.

* neutorn+nova: the port is created directly using neutron and port id is passed to nova to create the server.

Hopefully the patch is merged soon.

[Bhuvan Arumugam (bhuvan)]

Aaron, any luck with revising the patch to validate using api?
https://review.openstack.org/59578

[Bhuvan Arumugam (bhuvan)]

Aaron, FYI. With patch set 12 in https://review.openstack.org/#/c/59578/, the following unit tests fail. This is in our environment.

Traceback (most recent call last):
 File "nova/tests/network/test_neutronv2.py", line 1331, in test_validate_networks_port_no_subnet_id_psec_enabled_false
   api.validate_networks(self.context, requested_networks, 1)
 File "nova/network/neutronv2/api.py", line 664, in validate_networks
   neutron=neutron)
 File "nova/network/neutronv2/api.py", line 120, in _get_available_networks
   nets = neutron.list_networks(**search_opts).get('networks', [])
 File "<http://ci.isg.apple.com/jenkins/job/csi-nova-merge/ws/.tox/py26/lib/python2.6/site-packages/mox.py",> line 1002, in __call__
   expected_method = self._VerifyMethodCall()
 File "<http://ci.isg.apple.com/jenkins/job/csi-nova-merge/ws/.tox/py26/lib/python2.6/site-packages/mox.py",> line 1049, in _VerifyMethodCall
   expected = self._PopNextMethod()
 File "<http://ci.isg.apple.com/jenkins/job/csi-nova-merge/ws/.tox/py26/lib/python2.6/site-packages/mox.py",> line 1035, in _PopNextMethod
   raise UnexpectedMethodCallError(self, None)
UnexpectedMethodCallError: Unexpected method call Client.list_networks(id=['my_netid1']) -> None

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/99181

[OpenStack Infra (hudson-openstack)]

Change abandoned by Aaron Rosen (<email address hidden>) on branch: master
Review: https://review.openstack.org/99181

[OpenStack Infra (hudson-openstack)]

Change abandoned by Aaron Rosen (<email address hidden>) on branch: master
Review: https://review.openstack.org/59578

[OpenStack Infra (hudson-openstack)]

Change abandoned by Sean Dague (<email address hidden>) on branch: master
Review: https://review.openstack.org/59578
Reason: This review is > 4 weeks without comment, and failed Jenkins the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

[Matt Riedemann (mriedem)]

Note a similar issue here: https://review.openstack.org/#/c/151184/

Oleg made a change which broke nova boot with neutron if you specified a port_id because the default security group was always added by the nova API. Not adding the default security group is the correct change, but the fix from arosen has been abandoned and not worked on.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/173204

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/200046

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/278369

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/284095

[OpenStack Infra (hudson-openstack)]

Change abandoned by Matt Riedemann (<email address hidden>) on branch: master
Review: https://review.openstack.org/173204
Reason: We're going with this fix:

https://review.openstack.org/#/c/284095/

[OpenStack Infra (hudson-openstack)]

Change abandoned by Matt Riedemann (<email address hidden>) on branch: master
Review: https://review.openstack.org/278369
Reason: This is a duplicate of https://review.openstack.org/#/c/173204/

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/306470

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/284095
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=ee7a01982611cdf8012a308fa49722146c51497f
Submitter: Jenkins
Branch: master

commit ee7a01982611cdf8012a308fa49722146c51497f
Author: Sahid Orentino Ferdjaoui <email address hidden>
Date: Wed Feb 24 06:55:30 2016 -0500

    network: make nova to handle port_security_enabled=False

    In somes cases we need to have network without any security rules
    applied, unfortunatly nova does not provide way to remove l3
    assignements and always at least expose the default security group.
    This commit updates code to clear security groups applied to the
    network when option port_security_enabled=False is activated on the
    network.

    Change-Id: I630008a9733624a9d9b59b7aa3b8b2a3f8985d61
    Closes-Bug: #1460630
    Related-Bug: #1175464

[OpenStack Infra (hudson-openstack)]

Change abandoned by Michael Still (<email address hidden>) on branch: master
Review: https://review.openstack.org/200046
Reason: This patch has been idle for a long time, so I am abandoning it to keep the review clean sane. If you're interested in still working on this patch, then please unabandon it and upload a new patchset.

[Gabriel Assis Bezerra (gabriel-bezerra)]

For those interested in having the backport of this bugfix for Kilo, find it at https://github.com/gabriel-bezerra/nova/tree/kilo-bug/1175464

https://github.com/gabriel-bezerra/nova/commit/d4cf5f180728abce809326f10278dacef5b3b6c1

Best regards,
Gabriel.

[Gabriel Assis Bezerra (gabriel-bezerra)]

This is patch is related: https://review.openstack.org/310920

I added that to the kilo back port as well: https://github.com/gabriel-bezerra/nova/tree/kilo-bug/1175464

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/306470
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=84d5697c9e614c2bf299e213f5398e4ecf160400
Submitter: Jenkins
Branch: stable/mitaka

commit 84d5697c9e614c2bf299e213f5398e4ecf160400
Author: Sahid Orentino Ferdjaoui <email address hidden>
Date: Wed Feb 24 06:55:30 2016 -0500

    network: make nova to handle port_security_enabled=False

    In somes cases we need to have network without any security rules
    applied, unfortunatly nova does not provide way to remove l3
    assignements and always at least expose the default security group.
    This commit updates code to clear security groups applied to the
    network when option port_security_enabled=False is activated on the
    network.

    Change-Id: I630008a9733624a9d9b59b7aa3b8b2a3f8985d61
    Closes-Bug: #1460630
    Related-Bug: #1175464

[Siva Kollipara (skollipa)]

On a related note, I am trying the following combinations: (where the port is allocated from that network and SG is short for '--security-group SG' )

(a) network.portsecurity=T, nova-boot-port.portsecurity=T/SG, nova-attach-port.portsecurity=F: ok

(b) network.portsecurity=T, nova-boot-port.portsecurity=F, nova-attach-port.portsecurity=T/SG: ok

(c) network.portsecurity=F, nova-boot-port.portsecurity=F (requires 306470, 310920), nova-attach-port.portsecurity=T/SG: ok

(d) network.portsecurity=F, nova-boot-port.portsecurity=T/SG: *fails* with SecurityGroupCannotBeApplied exception.

(e) network.portsecurity=F, nova-boot-net.id=network.portsecurity=F: ok

(f) network.portsecurity=F, nova-boot-net.id=network.portsecurity=F/SG: as expected fails with SecurityGroupCannotBeApplied

(g) network.portsecurity=T, nova-boot-port.portsecurity=F/SG: ok, SG was ignored for the port

This is from the function _create_ports_for_instance() at network/neutronv2/api.py:680-700:
                if port_security_enabled:
                    <snip>
                else:
                    if security_group_ids:
                        raise exception.SecurityGroupCannotBeApplied()

I am wondering if the interface-attach of a port in (c) worked fine, then why should not the bootup using (d)?

the nova-boot in (c) is similar as (e) and behaving as such.
the nova-boot in (d) is *not* similar as (f) but behaving as such.

I suppose the check should be done only if the port is not provided in the API.
                if port_security_enabled:
                    <snip>
                else:
- if security_group_ids:
+ if security_group_ids and not request.port_id:
                        raise exception.SecurityGroupCannotBeApplied()

This change specifically allows (d) to succeed without impacting (f) or anything else.

Comments? Acceptable?

[Siva Kollipara (skollipa)]

adding the prior diff as an attachment

[Sean Dague (sdague)]

Fixed in https://review.openstack.org/#/c/306470/

[Matt Riedemann (mriedem)]

This was fixed in https://review.openstack.org/#/q/I630008a9733624a9d9b59b7aa3b8b2a3f8985d61,n,z

[Matt Riedemann (mriedem)]

If there are new issues let's open a new bug for this.