VM boot is broken with providing port-id from Neutron

[Valeriy Ponomaryov]

Commit https://review.openstack.org/#/c/124059/ has introduced bug, where Nova can not boot VM.

Steps to reproduce:

1) Create port in Neutron
2) Boot Vm without security group, but with port:

nova --debug boot tt --image=25a15f92-6bbe-43d6-8da5-b015966a4bd1 --flavor=100 --nic port-id=01e02c22-6ea3-4fe6-8cfe-407a06b634a0

...

REQ: curl -i 'http://172.18.198.52:8774/v2/35b86f321c03497fbfa1c0fdf98a3426/servers' -X POST -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Project-Id: demo" -H "X-Auth-Token: {SHA1}696ac31a35c12934a64485459b0a95a48a9ab4dd" -d '{"server": {"name": "tt", "imageRef": "25a15f92-6bbe-43d6-8da5-b015966a4bd1", "flavorRef": "100", "max_count": 1, "min_count": 1, "networks": [{"port": "01e02c22-6ea3-4fe6-8cfe-407a06b634a0"}]}}'

...

Trace as a result:

2015-01-29 12:14:03.338 ERROR nova.compute.manager [-] Instance failed network setup after 1 attempt(s)
2015-01-29 12:14:03.338 TRACE nova.compute.manager Traceback (most recent call last):
2015-01-29 12:14:03.338 TRACE nova.compute.manager File "/opt/stack/nova/nova/compute/manager.py", line 1677, in _allocate_network_async
2015-01-29 12:14:03.338 TRACE nova.compute.manager dhcp_options=dhcp_options)
2015-01-29 12:14:03.338 TRACE nova.compute.manager File "/opt/stack/nova/nova/network/neutronv2/api.py", line 457, in allocate_for_instance
2015-01-29 12:14:03.338 TRACE nova.compute.manager raise exception.SecurityGroupNotAllowedTogetherWithPort()
2015-01-29 12:14:03.338 TRACE nova.compute.manager SecurityGroupNotAllowedTogetherWithPort: It's not allowed to specify security groups if port_id is provided on instance boot. Neutron should be used to configure security groups on port.
2015-01-29 12:14:03.338 TRACE nova.compute.manager
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/poll.py", line 115, in wait
    listener.cb(fileno)
  File "/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py", line 214, in main
    result = function(*args, **kwargs)
  File "/opt/stack/nova/nova/compute/manager.py", line 1677, in _allocate_network_async
    dhcp_options=dhcp_options)
  File "/opt/stack/nova/nova/network/neutronv2/api.py", line 457, in allocate_for_instance
    raise exception.SecurityGroupNotAllowedTogetherWithPort()
SecurityGroupNotAllowedTogetherWithPort: It's not allowed to specify security groups if port_id is provided on instance boot. Neutron should be used to configure security groups on port.
Removing descriptor: 19
2015-01-29 12:14:03.529 DEB

2015-01-29 12:14:03.710 INFO nova.virt.libvirt.driver [-] [instance: c4892579-e32b-44ca-b8c7-72f3e04c6800] Using config drive
2015-01-29 12:14:03.763 ERROR nova.compute.manager [-] [instance: c4892579-e32b-44ca-b8c7-72f3e04c6800] Instance failed to spawn
2015-01-29 12:14:03.763 TRACE nova.compute.manager [instance: c4892579-e32b-44ca-b8c7-72f3e04c6800] Traceback (most recent call last):
2015-01-29 12:14:03.763 TRACE nova.compute.manager [instance: c4892579-e32b-44ca-b8c7-72f3e04c6800] File "/opt/stack/nova/nova/compute/manager.py", line 2303, in _build_resources
2015-01-29 12:14:03.763 TRACE nova.compute.manager [instance: c4892579-e32b-44ca-b8c7-72f3e04c6800] yield resources
2015-01-29 12:14:03.763 TRACE nova.compute.manager [instance: c4892579-e32b-44ca-b8c7-72f3e04c6800] File "/opt/stack/nova/nova/compute/manager.py", line 2173, in _build_and_run_instance
2015-01-29 12:14:03.763 TRACE nova.compute.manager [instance: c4892579-e32b-44ca-b8c7-72f3e04c6800] flavor=flavor)
2015-01-29 12:14:03.763 TRACE nova.compute.manager [instance: c4892579-e32b-44ca-b8c7-72f3e04c6800] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 2309, in spawn
2015-01-29 12:14:03.763 TRACE nova.compute.manager [instance: c4892579-e32b-44ca-b8c7-72f3e04c6800] admin_pass=admin_password)
2015-01-29 12:14:03.763 TRACE nova.compute.manager [instance: c4892579-e32b-44ca-b8c7-72f3e04c6800] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 2783, in _create_image
2015-01-29 12:14:03.763 TRACE nova.compute.manager [instance: c4892579-e32b-44ca-b8c7-72f3e04c6800] content=files, extra_md=extra_md, network_info=network_info)
2015-01-29 12:14:03.763 TRACE nova.compute.manager [instance: c4892579-e32b-44ca-b8c7-72f3e04c6800] File "/opt/stack/nova/nova/api/metadata/base.py", line 159, in __init__
2015-01-29 12:14:03.763 TRACE nova.compute.manager [instance: c4892579-e32b-44ca-b8c7-72f3e04c6800] ec2utils.get_ip_info_for_instance_from_nw_info(network_info)
2015-01-29 12:14:03.763 TRACE nova.compute.manager [instance: c4892579-e32b-44ca-b8c7-72f3e04c6800] File "/opt/stack/nova/nova/api/ec2/ec2utils.py", line 152, in get_ip_info_for_instance_from_nw_info
2015-01-29 12:14:03.763 TRACE nova.compute.manager [instance: c4892579-e32b-44ca-b8c7-72f3e04c6800] fixed_ips = nw_info.fixed_ips()
2015-01-29 12:14:03.763 TRACE nova.compute.manager [instance: c4892579-e32b-44ca-b8c7-72f3e04c6800] File "/opt/stack/nova/nova/network/model.py", line 450, in _sync_wrapper
2015-01-29 12:14:03.763 TRACE nova.compute.manager [instance: c4892579-e32b-44ca-b8c7-72f3e04c6800] self.wait()
2015-01-29 12:14:03.763 TRACE nova.compute.manager [instance: c4892579-e32b-44ca-b8c7-72f3e04c6800] File "/opt/stack/nova/nova/network/model.py", line 482, in wait
2015-01-29 12:14:03.763 TRACE nova.compute.manager [instance: c4892579-e32b-44ca-b8c7-72f3e04c6800] self[:] = self._gt.wait()
2015-01-29 12:14:03.763 TRACE nova.compute.manager [instance: c4892579-e32b-44ca-b8c7-72f3e04c6800] File "/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py", line 175, in wait
2015-01-29 12:14:03.763 TRACE nova.compute.manager [instance: c4892579-e32b-44ca-b8c7-72f3e04c6800] return self._exit_event.wait()
2015-01-29 12:14:03.763 TRACE nova.compute.manager [instance: c4892579-e32b-44ca-b8c7-72f3e04c6800] File "/usr/local/lib/python2.7/dist-packages/eventlet/event.py", line 125, in wait
2015-01-29 12:14:03.763 TRACE nova.compute.manager [instance: c4892579-e32b-44ca-b8c7-72f3e04c6800] current.throw(*self._exc)
2015-01-29 12:14:03.763 TRACE nova.compute.manager [instance: c4892579-e32b-44ca-b8c7-72f3e04c6800] File "/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py", line 214, in main
2015-01-29 12:14:03.763 TRACE nova.compute.manager [instance: c4892579-e32b-44ca-b8c7-72f3e04c6800] result = function(*args, **kwargs)
2015-01-29 12:14:03.763 TRACE nova.compute.manager [instance: c4892579-e32b-44ca-b8c7-72f3e04c6800] File "/opt/stack/nova/nova/compute/manager.py", line 1677, in _allocate_network_async
2015-01-29 12:14:03.763 TRACE nova.compute.manager [instance: c4892579-e32b-44ca-b8c7-72f3e04c6800] dhcp_options=dhcp_options)
2015-01-29 12:14:03.763 TRACE nova.compute.manager [instance: c4892579-e32b-44ca-b8c7-72f3e04c6800] File "/opt/stack/nova/nova/network/neutronv2/api.py", line 457, in allocate_for_instance
2015-01-29 12:14:03.763 TRACE nova.compute.manager [instance: c4892579-e32b-44ca-b8c7-72f3e04c6800] raise exception.SecurityGroupNotAllowedTogetherWithPort()
2015-01-29 12:14:03.763 TRACE nova.compute.manager [instance: c4892579-e32b-44ca-b8c7-72f3e04c6800] SecurityGroupNotAllowedTogetherWithPort: It's not allowed to specify security groups if port_id is provided on instance boot. Neutron should be used to configure security groups on port.
2015-01-29 12:14:03.763 TRACE nova.compute.manager [instance: c4892579-e32b-44ca-b8c7-72f3e04c6800]

Reason of bug - Nova raises error in case security group is provided. And it is always provided, because Nova pushes in "default" security group in case None is provided with request, see:

https://github.com/openstack/nova/blob/37af08116249dee64a7d688b06fff509422911ac/nova/api/openstack/compute/servers.py#L515

[Matthew Booth (mbooth-9)]

Quick review: in nova/compute/api.py:_create_instance we do:

        security_groups = security_groups or ['default']

This value seems to propagate unmolested to the code in question, suggesting that, when called from this path at least, security_groups will never be empty.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/151184

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/151318

[Matt Riedemann (mriedem)]

Discussion on Tempest coverage:

(2:38:57 PM) mtreinish: mriedem: hmm, there might be a neutron api test that does that, but I doubt a compute test would
(2:39:32 PM) mriedem: mtreinish: neutron api test that boots a server with a port id?
(2:39:41 PM) mriedem: i thought neutron api tests for the most part just tested neutron apis
(2:40:07 PM) mtreinish: mriedem: I did say might :) Most likely there is a scenario test doing that
(2:40:12 PM) mtreinish: let me check quickly
(2:42:39 PM) mtreinish: mriedem: yeah check test_network_basic_ops.py there is a test in there that does what you're looking for
(2:45:17 PM) openstackgerrit: Adolfo Duarte proposed openstack/tempest: Adding api tests for dvr routers https://review.openstack.org/133317
(2:53:10 PM) mriedem: mtreinish: weird....wonder why we didn't explode on oleg's change then...
(2:53:23 PM) mriedem: i'll have to dig into it at some point, or just update review so i don't forget
(2:54:00 PM) mtreinish: mriedem: do you have links? I can take a quick look
(2:54:17 PM) mriedem: https://bugs.launchpad.net/nova/+bug/1415835
(2:55:40 PM) mriedem: mtreinish: http://git.openstack.org/cgit/openstack/tempest/tree/tempest/scenario/test_network_basic_ops.py#n103
(2:55:50 PM) mriedem: so the test creates a security group under a given tenant
(2:55:55 PM) mriedem: creates a netework
(2:56:03 PM) mriedem: then boots the server to that network
(2:56:12 PM) mriedem: but doesn't boot the server with a specific port at create time
(2:56:13 PM) mriedem: which is the bug
(2:56:39 PM) mriedem: so the port exists in neutron before the server is created, etc
(2:56:59 PM) mtreinish: mriedem: ah, ok. You can probably add that case somewhere if you want, I wouldn't complain
(2:57:40 PM) mriedem: yeah that's the plan at some point

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/151184
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=6d8f31349d2c6f2d13479a066eb18d4a65147067
Submitter: Jenkins
Branch: master

commit 6d8f31349d2c6f2d13479a066eb18d4a65147067
Author: Matthew Booth <email address hidden>
Date: Thu Jan 29 11:08:37 2015 +0000

    Revert "Raise if sec-groups and port id are provided on boot"

    This reverts commit d8cafb9a6efb95b78609aca8fcb3532c33a1788f.

    The above commit assumes that security_groups can be empty. However,
    it will always have had the default security_group added to it,
    meaning that with this patch it is never possible to specify a port
    it.

    Closes-bug: #1415835
    Change-Id: I01b561ac8b9b29cc4721d88567678a42a4fd1f6b

[OpenStack Infra (hudson-openstack)]

Change abandoned by Matt Riedemann (<email address hidden>) on branch: master
Review: https://review.openstack.org/151318
Reason: The revert https://review.openstack.org/#/c/151184/ was merged.

[Matt Riedemann (mriedem)]

Note bug 1175464, that's for not adding the default security group to the neutron request unless the user specifies it, which is probably the more correct thing to do.

There is also an old abandoned change from arosen for that:

https://review.openstack.org/#/c/59578/