controller nodes:
con1 con2 con3
compute nodes:
compute01
oopenstack version: train
There are no problems in the all-in-one test，Change to multi-node point to create a virtual machine and report the following error? The linuxbridge-agent on comput01 can create the vm network normally, but nova did not receive the message that the linuxbridge-agent creation is complete, resulting in a timeout error. I can also ignore this error by changing the nova.conf parameters. But this is not the right way to solve the problem
[root@compute01 ~]# tail -f /var/log/nova/nova-compute.log
"normal", "profile": {}, "preserve_on_delete": false, "meta": {}} nova_to_osvif_vif /usr/local/lib/python3.7/site-packages/nova/network/os_vif_util.py:516
2020-08-24 09:36:09.246 321929 DEBUG nova.network.os_vif_util [req-eeee017f-3487-4f3d-a324-d91478b7e6da a5c870c4977a48bba3c89646b1b96b36 08f60b7919a04ad1803a2899721a6f76 - default default] Converted object VIFBridge(active=False,address=fa:16:3e:48:4d:32,bridge_name='brqb5ea24c8-97',has_traffic_filtering=True,id=091f0cdf-9e7d-4f74-b76c-306d2a580f25,network=Network(b5ea24c8-971d-4213-a258-27aeb022461d),plugin='linux_bridge',port_profile=<?>,preserve_on_delete=False,vif_name='tap091f0cdf-9e') nova_to_osvif_vif /usr/local/lib/python3.7/site-packages/nova/network/os_vif_util.py:553
2020-08-24 09:36:09.247 321929 DEBUG os_vif [req-eeee017f-3487-4f3d-a324-d91478b7e6da a5c870c4977a48bba3c89646b1b96b36 08f60b7919a04ad1803a2899721a6f76 - default default] Unplugging vif VIFBridge(active=False,address=fa:16:3e:48:4d:32,bridge_name='brqb5ea24c8-97',has_traffic_filtering=True,id=091f0cdf-9e7d-4f74-b76c-306d2a580f25,network=Network(b5ea24c8-971d-4213-a258-27aeb022461d),plugin='linux_bridge',port_profile=<?>,preserve_on_delete=False,vif_name='tap091f0cdf-9e') unplug /usr/local/lib/python3.7/site-packages/os_vif/__init__.py:109
2020-08-24 09:36:09.247 321929 INFO os_vif [req-eeee017f-3487-4f3d-a324-d91478b7e6da a5c870c4977a48bba3c89646b1b96b36 08f60b7919a04ad1803a2899721a6f76 - default default] Successfully unplugged vif VIFBridge(active=False,address=fa:16:3e:48:4d:32,bridge_name='brqb5ea24c8-97',has_traffic_filtering=True,id=091f0cdf-9e7d-4f74-b76c-306d2a580f25,network=Network(b5ea24c8-971d-4213-a258-27aeb022461d),plugin='linux_bridge',port_profile=<?>,preserve_on_delete=False,vif_name='tap091f0cdf-9e')
2020-08-24 09:36:09.248 321929 INFO nova.virt.libvirt.driver [req-eeee017f-3487-4f3d-a324-d91478b7e6da a5c870c4977a48bba3c89646b1b96b36 08f60b7919a04ad1803a2899721a6f76 - default default] [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] Deleting instance files /var/lib/nova/instances/bca80c49-7c0d-4cf4-a1cc-5afa16f7470a_del
2020-08-24 09:36:09.248 321929 INFO nova.virt.libvirt.driver [req-eeee017f-3487-4f3d-a324-d91478b7e6da a5c870c4977a48bba3c89646b1b96b36 08f60b7919a04ad1803a2899721a6f76 - default default] [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] Deletion of /var/lib/nova/instances/bca80c49-7c0d-4cf4-a1cc-5afa16f7470a_del complete
2020-08-24 09:36:09.356 321929 ERROR nova.compute.manager [req-eeee017f-3487-4f3d-a324-d91478b7e6da a5c870c4977a48bba3c89646b1b96b36 08f60b7919a04ad1803a2899721a6f76 - default default] [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] Instance failed to spawn: nova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed
2020-08-24 09:36:09.356 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] Traceback (most recent call last):
2020-08-24 09:36:09.356 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] File "/usr/local/lib/python3.7/site-packages/nova/virt/libvirt/driver.py", line 6325, in _create_domain_and_network
2020-08-24 09:36:09.356 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] network_info)
2020-08-24 09:36:09.356 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] File "/usr/lib64/python3.7/contextlib.py", line 119, in __exit__
2020-08-24 09:36:09.356 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] next(self.gen)
2020-08-24 09:36:09.356 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] File "/usr/local/lib/python3.7/site-packages/nova/compute/manager.py", line 478, in wait_for_instance_event
2020-08-24 09:36:09.356 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] actual_event = event.wait()
2020-08-24 09:36:09.356 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] File "/usr/local/lib/python3.7/site-packages/eventlet/event.py", line 125, in wait
2020-08-24 09:36:09.356 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] result = hub.switch()
2020-08-24 09:36:09.356 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] File "/usr/local/lib/python3.7/site-packages/eventlet/hubs/hub.py", line 298, in switch
2020-08-24 09:36:09.356 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] return self.greenlet.switch()
2020-08-24 09:36:09.356 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] eventlet.timeout.Timeout: 300 seconds
2020-08-24 09:36:09.356 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a]
2020-08-24 09:36:09.356 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] During handling of the above exception, another exception occurred:
2020-08-24 09:36:09.356 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a]
2020-08-24 09:36:09.356 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] Traceback (most recent call last):
2020-08-24 09:36:09.356 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] File "/usr/local/lib/python3.7/site-packages/nova/compute/manager.py", line 2663, in _build_resources
2020-08-24 09:36:09.356 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] yield resources
2020-08-24 09:36:09.356 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] File "/usr/local/lib/python3.7/site-packages/nova/compute/manager.py", line 2437, in _build_and_run_instance
2020-08-24 09:36:09.356 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] block_device_info=block_device_info)
2020-08-24 09:36:09.356 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] File "/usr/local/lib/python3.7/site-packages/nova/virt/libvirt/driver.py", line 3530, in spawn
2020-08-24 09:36:09.356 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] cleanup_instance_disks=created_disks)
2020-08-24 09:36:09.356 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] File "/usr/local/lib/python3.7/site-packages/nova/virt/libvirt/driver.py", line 6348, in _create_domain_and_network
2020-08-24 09:36:09.356 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] raise exception.VirtualInterfaceCreateException()
2020-08-24 09:36:09.356 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] nova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed
2020-08-24 09:36:09.356 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a]
2020-08-24 09:36:09.357 321929 INFO nova.compute.manager [req-eeee017f-3487-4f3d-a324-d91478b7e6da a5c870c4977a48bba3c89646b1b96b36 08f60b7919a04ad1803a2899721a6f76 - default default] [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] Terminating instance
2020-08-24 09:36:09.358 321929 DEBUG nova.compute.manager [req-eeee017f-3487-4f3d-a324-d91478b7e6da a5c870c4977a48bba3c89646b1b96b36 08f60b7919a04ad1803a2899721a6f76 - default default] [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] Start destroying the instance on the hypervisor. _shutdown_instance /usr/local/lib/python3.7/site-packages/nova/compute/manager.py:2845
2020-08-24 09:36:09.360 321929 DEBUG nova.virt.libvirt.driver [-] [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] During wait destroy, instance disappeared. _wait_for_destroy /usr/local/lib/python3.7/site-packages/nova/virt/libvirt/driver.py:1279
2020-08-24 09:36:09.360 321929 INFO nova.virt.libvirt.driver [-] [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] Instance destroyed successfully.
2020-08-24 09:36:09.361 321929 DEBUG nova.virt.libvirt.vif [req-eeee017f-3487-4f3d-a324-d91478b7e6da a5c870c4977a48bba3c89646b1b96b36 08f60b7919a04ad1803a2899721a6f76 - default default] vif_type=bridge instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=True,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2020-08-24T01:31:04Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='sssss',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(1),hidden=False,host='compute01',hostname='sssss',id=16,image_ref='9d3e7945-1eeb-4226-884f-428fdfd27427',info_cache=InstanceInfoCache,instance_type_id=1,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='compute01',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='compute01',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='08f60b7919a04ad1803a2899721a6f76',ramdisk_id='',reservation_id='r-p3tehlc7',resources=None,root_device_name='/dev/vda',root_gb=10,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='member,reader,admin',clean_attempts='1',image_base_image_ref='9d3e7945-1eeb-4226-884f-428fdfd27427',image_container_format='bare',image_disk_format='qcow2',image_min_disk='10',image_min_ram='0',network_allocated='True',owner_project_name='admin',owner_user_name='admin'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2020-08-24T01:31:06Z,user_data=None,user_id='a5c870c4977a48bba3c89646b1b96b36',uuid=bca80c49-7c0d-4cf4-a1cc-5afa16f7470a,vcpu_model=VirtCPUModel,vcpus=2,vm_mode=None,vm_state='building') vif={"id": "091f0cdf-9e7d-4f74-b76c-306d2a580f25", "address": "fa:16:3e:48:4d:32", "network": {"id": "b5ea24c8-971d-4213-a258-27aeb022461d", "bridge": "brqb5ea24c8-97", "label": "ssss", "subnets": [{"cidr": "192.168.20.0/24", "dns": [], "gateway": {"address": null, "type": "gateway", "version": null, "meta": {}}, "ips": [{"address": "192.168.20.246", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.20.1"}}], "meta": {"injected": false, "tenant_id": "08f60b7919a04ad1803a2899721a6f76", "mtu": 1500, "physical_network": "provider", "tunneled": false, "should_create_bridge": true}}, "type": "bridge", "details": {"connectivity": "l2", "port_filter": true}, "devname": "tap091f0cdf-9e", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "meta": {}} unplug /usr/local/lib/python3.7/site-packages/nova/virt/libvirt/vif.py:834
2020-08-24 09:36:09.361 321929 DEBUG nova.network.os_vif_util [req-eeee017f-3487-4f3d-a324-d91478b7e6da a5c870c4977a48bba3c89646b1b96b36 08f60b7919a04ad1803a2899721a6f76 - default default] Converting VIF {"id": "091f0cdf-9e7d-4f74-b76c-306d2a580f25", "address": "fa:16:3e:48:4d:32", "network": {"id": "b5ea24c8-971d-4213-a258-27aeb022461d", "bridge": "brqb5ea24c8-97", "label": "ssss", "subnets": [{"cidr": "192.168.20.0/24", "dns": [], "gateway": {"address": null, "type": "gateway", "version": null, "meta": {}}, "ips": [{"address": "192.168.20.246", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.20.1"}}], "meta": {"injected": false, "tenant_id": "08f60b7919a04ad1803a2899721a6f76", "mtu": 1500, "physical_network": "provider", "tunneled": false, "should_create_bridge": true}}, "type": "bridge", "details": {"connectivity": "l2", "port_filter": true}, "devname": "tap091f0cdf-9e", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "meta": {}} nova_to_osvif_vif /usr/local/lib/python3.7/site-packages/nova/network/os_vif_util.py:516
2020-08-24 09:36:09.362 321929 DEBUG nova.network.os_vif_util [req-eeee017f-3487-4f3d-a324-d91478b7e6da a5c870c4977a48bba3c89646b1b96b36 08f60b7919a04ad1803a2899721a6f76 - default default] Converted object VIFBridge(active=False,address=fa:16:3e:48:4d:32,bridge_name='brqb5ea24c8-97',has_traffic_filtering=True,id=091f0cdf-9e7d-4f74-b76c-306d2a580f25,network=Network(b5ea24c8-971d-4213-a258-27aeb022461d),plugin='linux_bridge',port_profile=<?>,preserve_on_delete=False,vif_name='tap091f0cdf-9e') nova_to_osvif_vif /usr/local/lib/python3.7/site-packages/nova/network/os_vif_util.py:553
2020-08-24 09:36:09.362 321929 DEBUG os_vif [req-eeee017f-3487-4f3d-a324-d91478b7e6da a5c870c4977a48bba3c89646b1b96b36 08f60b7919a04ad1803a2899721a6f76 - default default] Unplugging vif VIFBridge(active=False,address=fa:16:3e:48:4d:32,bridge_name='brqb5ea24c8-97',has_traffic_filtering=True,id=091f0cdf-9e7d-4f74-b76c-306d2a580f25,network=Network(b5ea24c8-971d-4213-a258-27aeb022461d),plugin='linux_bridge',port_profile=<?>,preserve_on_delete=False,vif_name='tap091f0cdf-9e') unplug /usr/local/lib/python3.7/site-packages/os_vif/__init__.py:109
2020-08-24 09:36:09.363 321929 INFO os_vif [req-eeee017f-3487-4f3d-a324-d91478b7e6da a5c870c4977a48bba3c89646b1b96b36 08f60b7919a04ad1803a2899721a6f76 - default default] Successfully unplugged vif VIFBridge(active=False,address=fa:16:3e:48:4d:32,bridge_name='brqb5ea24c8-97',has_traffic_filtering=True,id=091f0cdf-9e7d-4f74-b76c-306d2a580f25,network=Network(b5ea24c8-971d-4213-a258-27aeb022461d),plugin='linux_bridge',port_profile=<?>,preserve_on_delete=False,vif_name='tap091f0cdf-9e')
2020-08-24 09:36:09.363 321929 INFO nova.virt.libvirt.driver [req-eeee017f-3487-4f3d-a324-d91478b7e6da a5c870c4977a48bba3c89646b1b96b36 08f60b7919a04ad1803a2899721a6f76 - default default] [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] Deletion of /var/lib/nova/instances/bca80c49-7c0d-4cf4-a1cc-5afa16f7470a_del complete
2020-08-24 09:36:09.463 321929 INFO nova.compute.manager [req-eeee017f-3487-4f3d-a324-d91478b7e6da a5c870c4977a48bba3c89646b1b96b36 08f60b7919a04ad1803a2899721a6f76 - default default] [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] Took 0.10 seconds to destroy the instance on the hypervisor.
2020-08-24 09:36:09.464 321929 DEBUG nova.compute.claims [req-eeee017f-3487-4f3d-a324-d91478b7e6da a5c870c4977a48bba3c89646b1b96b36 08f60b7919a04ad1803a2899721a6f76 - default default] [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] Aborting claim: <nova.compute.claims.Claim object at 0xfffd0c707850> abort /usr/local/lib/python3.7/site-packages/nova/compute/claims.py:82
2020-08-24 09:36:09.464 321929 DEBUG oslo_concurrency.lockutils [req-eeee017f-3487-4f3d-a324-d91478b7e6da a5c870c4977a48bba3c89646b1b96b36 08f60b7919a04ad1803a2899721a6f76 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner /usr/local/lib/python3.7/site-packages/oslo_concurrency/lockutils.py:359
2020-08-24 09:36:09.600 321929 DEBUG nova.compute.provider_tree [req-eeee017f-3487-4f3d-a324-d91478b7e6da a5c870c4977a48bba3c89646b1b96b36 08f60b7919a04ad1803a2899721a6f76 - default default] Inventory has not changed in ProviderTree for provider: 3021e4a9-483c-486e-83c4-9885645276d1 update_inventory /usr/local/lib/python3.7/site-packages/nova/compute/provider_tree.py:181
2020-08-24 09:36:09.600 321929 DEBUG nova.virt.libvirt.driver [req-eeee017f-3487-4f3d-a324-d91478b7e6da a5c870c4977a48bba3c89646b1b96b36 08f60b7919a04ad1803a2899721a6f76 - default default] CPU mode 'host-model' models '' was chosen, with extra flags: '' _get_guest_cpu_model_config /usr/local/lib/python3.7/site-packages/nova/virt/libvirt/driver.py:4352
2020-08-24 09:36:09.615 321929 DEBUG nova.scheduler.client.report [req-eeee017f-3487-4f3d-a324-d91478b7e6da a5c870c4977a48bba3c89646b1b96b36 08f60b7919a04ad1803a2899721a6f76 - default default] Inventory has not changed for provider 3021e4a9-483c-486e-83c4-9885645276d1 based on inventory data: {'VCPU': {'total': 96, 'reserved': 0, 'min_unit': 1, 'max_unit': 96, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 260990, 'reserved': 32768, 'min_unit': 1, 'max_unit': 260990, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 2977, 'reserved': 0, 'min_unit': 1, 'max_unit': 2977, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider /usr/local/lib/python3.7/site-packages/nova/scheduler/client/report.py:898
2020-08-24 09:36:09.615 321929 DEBUG oslo_concurrency.lockutils [req-eeee017f-3487-4f3d-a324-d91478b7e6da a5c870c4977a48bba3c89646b1b96b36 08f60b7919a04ad1803a2899721a6f76 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.152s inner /usr/local/lib/python3.7/site-packages/oslo_concurrency/lockutils.py:371
2020-08-24 09:36:09.616 321929 ERROR nova.compute.manager [req-eeee017f-3487-4f3d-a324-d91478b7e6da a5c870c4977a48bba3c89646b1b96b36 08f60b7919a04ad1803a2899721a6f76 - default default] [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] Failed to allocate network(s): nova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed
2020-08-24 09:36:09.616 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] Traceback (most recent call last):
2020-08-24 09:36:09.616 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] File "/usr/local/lib/python3.7/site-packages/nova/virt/libvirt/driver.py", line 6325, in _create_domain_and_network
2020-08-24 09:36:09.616 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] network_info)
2020-08-24 09:36:09.616 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] File "/usr/lib64/python3.7/contextlib.py", line 119, in __exit__
2020-08-24 09:36:09.616 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] next(self.gen)
2020-08-24 09:36:09.616 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] File "/usr/local/lib/python3.7/site-packages/nova/compute/manager.py", line 478, in wait_for_instance_event
2020-08-24 09:36:09.616 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] actual_event = event.wait()
2020-08-24 09:36:09.616 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] File "/usr/local/lib/python3.7/site-packages/eventlet/event.py", line 125, in wait
2020-08-24 09:36:09.616 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] result = hub.switch()
2020-08-24 09:36:09.616 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] File "/usr/local/lib/python3.7/site-packages/eventlet/hubs/hub.py", line 298, in switch
2020-08-24 09:36:09.616 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] return self.greenlet.switch()
2020-08-24 09:36:09.616 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] eventlet.timeout.Timeout: 300 seconds
2020-08-24 09:36:09.616 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a]
2020-08-24 09:36:09.616 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] During handling of the above exception, another exception occurred:
2020-08-24 09:36:09.616 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a]
2020-08-24 09:36:09.616 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] Traceback (most recent call last):
2020-08-24 09:36:09.616 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] File "/usr/local/lib/python3.7/site-packages/nova/compute/manager.py", line 2437, in _build_and_run_instance
2020-08-24 09:36:09.616 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] block_device_info=block_device_info)
2020-08-24 09:36:09.616 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] File "/usr/local/lib/python3.7/site-packages/nova/virt/libvirt/driver.py", line 3530, in spawn
2020-08-24 09:36:09.616 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] cleanup_instance_disks=created_disks)
2020-08-24 09:36:09.616 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] File "/usr/local/lib/python3.7/site-packages/nova/virt/libvirt/driver.py", line 6348, in _create_domain_and_network
2020-08-24 09:36:09.616 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] raise exception.VirtualInterfaceCreateException()
2020-08-24 09:36:09.616 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] nova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed
2020-08-24 09:36:09.616 321929 ERROR nova.compute.manager [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a]
2020-08-24 09:36:09.617 321929 DEBUG nova.compute.utils [req-eeee017f-3487-4f3d-a324-d91478b7e6da a5c870c4977a48bba3c89646b1b96b36 08f60b7919a04ad1803a2899721a6f76 - default default] [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] Virtual Interface creation failed notify_about_instance_usage /usr/local/lib/python3.7/site-packages/nova/compute/utils.py:431
2020-08-24 09:36:09.618 321929 ERROR nova.compute.manager [req-eeee017f-3487-4f3d-a324-d91478b7e6da a5c870c4977a48bba3c89646b1b96b36 08f60b7919a04ad1803a2899721a6f76 - default default] [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] Build of instance bca80c49-7c0d-4cf4-a1cc-5afa16f7470a aborted: Failed to allocate the network(s), not rescheduling.: nova.exception.BuildAbortException: Build of instance bca80c49-7c0d-4cf4-a1cc-5afa16f7470a aborted: Failed to allocate the network(s), not rescheduling.
2020-08-24 09:36:09.618 321929 DEBUG nova.compute.manager [req-eeee017f-3487-4f3d-a324-d91478b7e6da a5c870c4977a48bba3c89646b1b96b36 08f60b7919a04ad1803a2899721a6f76 - default default] [instance: bca80c49-7c0d-4cf4-a1cc-5afa16f7470a] Unplugging VIFs for instance _cleanup_allocated_networks /usr/local/lib/python3.7/site-packages/nova/compute/manager.py:2700
Could you please attach the following logs:
* the neutron agent log showing that the vif plug succeeded
* the nova-api log around these time to see if the os-server-