OpenStack Compute (nova)

Instance create fails with NetworkNotFoundClient Error

Bug #1815417 reported by Derek Miller
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Expired
Undecided
Unassigned

Bug Description

Description
===========
Attempting to launch the first instance in PoC cluster after installation (https://docs.openstack.org/install-guide/launch-instance-selfservice.html) fails with error:

2019-02-10 18:18:02.975 16910 INFO nova.api.openstack.wsgi [req-576448be-8191-4361-867f-f0dbe076bc25 572a94b58e7c476b87027a8e1efe81d3 2ded1e03b20f4866bf0a590e9a21e353 - default default] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.

Steps to reproduce
==================

- New cluster with a demo user:

openstack server create --flavor m1.nano --image cirros --nic net-id=ec3c94d7-50fb-41cc-843e-e55074dbdfc4 --security-group default --key-name regular-key selfservice-instance

- A network with the specified net-id existed, but using the id and its name both resulted in the same error.

- This error only occurs when attempting to create instance with self-service networks

Environment
===========
1. Exact version of OpenStack you are running. See the following
  list for all releases: http://docs.openstack.org/releases/

OpenStack Rocky

# rpm -qa | grep nova
openstack-nova-conductor-18.1.0-1.el7.noarch
openstack-nova-console-18.1.0-1.el7.noarch
openstack-nova-api-18.1.0-1.el7.noarch
openstack-nova-novncproxy-18.1.0-1.el7.noarch
openstack-nova-common-18.1.0-1.el7.noarch
openstack-nova-scheduler-18.1.0-1.el7.noarch
python2-novaclient-11.0.0-1.el7.noarch
python-nova-18.1.0-1.el7.noarch
openstack-nova-placement-api-18.1.0-1.el7.noarch

2. Which hypervisor did you use?
   (For example: Libvirt + KVM, Libvirt + XEN, Hyper-V, PowerKVM, ...)
   What's the version of that?
+----+---------------------+-----------------+---------------+-------+
| ID | Hypervisor Hostname | Hypervisor Type | Host IP | State |
+----+---------------------+-----------------+---------------+-------+
| 1 | compute01 | QEMU | 192.168.10.20 | up |
| 2 | compute02 | QEMU | 192.168.10.30 | up |
+----+---------------------+-----------------+---------------+-------+

3. Which networking type did you use?
   - Neutron w/Linux Bridging (Self-Service)
   - 3 Networks Mgmt, Overlay, Provider (and Data, but that isn't implemented yet)

Logs & Configs
==============

$ openstack network agent list
+--------------------------------------+--------------------+--------------+-------------------+-------+-------+---------------------------+
| ID | Agent Type | Host | Availability Zone | Alive | State | Binary |
+--------------------------------------+--------------------+--------------+-------------------+-------+-------+---------------------------+
| 1bd03d48-ea95-4c44-ab9f-60b510192220 | Linux bridge agent | controller01 | None | :-) | UP | neutron-linuxbridge-agent |
| 46ededd4-efe7-414a-b902-df65c7b8ddc2 | Metadata agent | controller01 | None | :-) | UP | neutron-metadata-agent |
| 6152ed73-4b22-4889-bbaf-dfa6ff2ba501 | DHCP agent | controller01 | nova | :-) | UP | neutron-dhcp-agent |
| be311766-0713-4e33-be71-0015a0650fe3 | Linux bridge agent | compute02 | None | :-) | UP | neutron-linuxbridge-agent |
| be87fc1b-becd-4b43-b512-1e463cfc8d44 | L3 agent | controller01 | nova | :-) | UP | neutron-l3-agent |
| d1a44b38-4c03-45fe-b1cf-f34b8b2770e7 | Linux bridge agent | compute01 | None | :-) | UP | neutron-linuxbridge-agent |
+--------------------------------------+--------------------+--------------+-------------------+-------+-------+---------------------------+

$ openstack compute service list
+----+------------------+--------------+----------+---------+-------+----------------------------+
| ID | Binary | Host | Zone | Status | State | Updated At |
+----+------------------+--------------+----------+---------+-------+----------------------------+
| 1 | nova-conductor | controller01 | internal | enabled | up | 2019-02-11T07:03:52.000000 |
| 4 | nova-consoleauth | controller01 | internal | enabled | up | 2019-02-11T07:03:54.000000 |
| 5 | nova-scheduler | controller01 | internal | enabled | up | 2019-02-11T07:03:51.000000 |
| 10 | nova-compute | compute01 | nova | enabled | up | 2019-02-11T07:03:46.000000 |
| 11 | nova-compute | compute02 | nova | enabled | up | 2019-02-11T07:03:50.000000 |
+----+------------------+--------------+----------+---------+-------+----------------------------+

2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi [req-16947722-f8ed-41db-8431-421a17c4abf7 fa1c0a14a9bc40c181f02ce2c5665b6f a110f982d44345f1b4bdcf66663435f8 - default default] Unexpected exception in API method: NetworkNotFoundClient: Network ec3c94d7-50fb-41cc-843e-e55074dbdfc4 could not be found.
Neutron server returns request_ids: ['req-7fa89fa1-4458-4687-a5dd-9a6cb052db0f']
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi Traceback (most recent call last):
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py", line 801, in wrapped
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi return f(*args, **kwargs)
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 110, in wrapper
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi return func(*args, **kwargs)
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 110, in wrapper
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi return func(*args, **kwargs)
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 110, in wrapper
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi return func(*args, **kwargs)
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 110, in wrapper
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi return func(*args, **kwargs)
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 110, in wrapper
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi return func(*args, **kwargs)
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 110, in wrapper
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi return func(*args, **kwargs)
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 110, in wrapper
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi return func(*args, **kwargs)
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 110, in wrapper
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi return func(*args, **kwargs)
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 110, in wrapper
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi return func(*args, **kwargs)
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 110, in wrapper
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi return func(*args, **kwargs)
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/api/openstack/compute/servers.py", line 576, in create
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi **create_kwargs)
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/hooks.py", line 154, in inner
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi rv = f(*args, **kwargs)
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 1714, in create
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi trusted_certs=trusted_certs)
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 1161, in _create_instance
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi reservation_id, max_count)
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 818, in _validate_and_build_base_options
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi context, requested_networks, pci_request_info)
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 1908, in create_resource_requests
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi context, neutron, network_id)
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 1806, in _get_physnet_tunneled_info
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi fields='segments').get('network')
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 126, in wrapper
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs)
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 813, in show_network
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi return self.get(self.network_path % (network), params=_params)
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 126, in wrapper
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs)
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 354, in get
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi headers=headers, params=params)
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 126, in wrapper
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs)
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 331, in retry_request
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi headers=headers, params=params)
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 126, in wrapper
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs)
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 294, in do_request
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp)
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 126, in wrapper
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs)
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 269, in _handle_fault_response
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body)
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 93, in exception_handler_v20
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi request_ids=request_ids)
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi NetworkNotFoundClient: Network ec3c94d7-50fb-41cc-843e-e55074dbdfc4 could not be found.
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-7fa89fa1-4458-4687-a5dd-9a6cb052db0f']
2019-02-10 15:51:38.826 16909 ERROR nova.api.openstack.wsgi
2019-02-10 15:51:38.840 16909 INFO nova.api.openstack.wsgi [req-16947722-f8ed-41db-8431-421a17c4abf7 fa1c0a14a9bc40c181f02ce2c5665b6f a110f982d44345f1b4bdcf66663435f8 - default default] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'neutronclient.common.exceptions.NetworkNotFoundClient'>

controller linuxbridge_agent.ini
# cat /etc/neutron/plugins/ml2/linuxbridge_agent.ini
[linux_bridge]
physical_interface_mappings = provider:enp4s0

[securitygroup]
enable_security_group = true
firewall_driver = neutron.agent.linux.iptables_firewall.IptablesFirewallDriver

[vxlan]
enable_vxlan = true
local_ip = 192.168.20.10
l2_population = true

compute01 linuxbridge_agent.ini:
# cat /etc/neutron/plugins/ml2/linuxbridge_agent.ini
[linux_bridge]
physical_interface_mappings = provider:enp7s0.40

[securitygroup]
enable_security_group = true
firewall_driver = neutron.agent.linux.iptables_firewall.IptablesFirewallDriver

[vxlan]
enable_vxlan = true
local_ip = 192.168.20.20
l2_population = true

IP addrs for reference:
controller01
============
Mgmt - 192.168.10.10
Overlay - 192.168.20.10
Provider (enp4s0) - no ip

compute01
=========
Mgmt - 192.168.10.20
Overlay - 192.168.20.20
Provider - no ip

compute02
=========
Mgmt - 192.168.10.30
Overlay - 192.168.20.30
Provider - no ip

Hope this isn't too much output. Any help would be appreciated. Let me know if more information is needed.

See original description
Takashi Natsume (natsume-takashi)
tags: added: neutron
Derek Miller (mevry240)
description: updated
Derek Miller (mevry240)
description: updated
description: updated
tags: added: neutronclient
Revision history for this message
Balazs Gibizer (balazs-gibizer) wrote :

From the information in the report it is not clear that
* the network ec3c94d7-50fb-41cc-843e-e55074dbdfc4 exists in neutron and
* visible for the demo user

Could you try to create a neutron port in ec3c94d7-50fb-41cc-843e-e55074dbdfc4 network with the demo user?

Revision history for this message
Balazs Gibizer (balazs-gibizer) wrote :

I set the bug to Incomplete as I have questions above. When you answer it please set the bug back to New state.

Changed in nova:
status: New → Incomplete
Revision history for this message
Launchpad Janitor (janitor) wrote :

[Expired for OpenStack Compute (nova) because there has been no activity for 60 days.]

Changed in nova:
status: Incomplete → Expired
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.