Juniper Openstack

Vcenter-as-compute: R4.1 ocata VM launch failure on Esxi host

  1. Series r4.0
  2. Bug #1733273
Bug #1733273 reported by Sarath
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Juniper Openstack
Status tracked in Trunk
R3.2
Fix Committed
Critical
ram yadav
Juniper Openstack r3.2.8.0
R4.0
Fix Committed
Critical
ram yadav
Juniper Openstack r4.0.3.0
R4.1
Fix Committed
Critical
ram yadav
Juniper Openstack r4.1.0.0-fcs "r4.1"
Trunk
Fix Committed
Critical
ram yadav
Juniper Openstack r5.0.0

Bug Description

VM launch works fine when KVM hypervisor and issue seen only when vcenter-compute hypervisor

Sachin did triaging on problem setup and please find below email thread for details,

From: Sarathbabu Narasimhan
Sent: Saturday, November 18, 2017 6:01 PM
To: Sachchidanand Vaidya <email address hidden>; Kamlesh Parmar <email address hidden>; Sachin Bansal <email address hidden>; Ram Yadav <email address hidden>
Cc: Jeba Paulaiyan <email address hidden>; Amudha R <email address hidden>; Sarathbabu Narasimhan <email address hidden>
Subject: RE: R4.1 ocata not launching VM on Vcenter-compute

Sorry for delayed response, I had email client issue morning and only received this email now,

Yes, Sachin and 5a10s26 is vcenter-compute and it shows neutron errors on logs during vm launch failure,

2017-11-18 17:52:06.509 26958 INFO nova.compute.resource_tracker [req-21e11b8b-7007-45de-bec7-f9402e66a55c - - - - -] Final resource view: name=domain-c5356.bb23c43b-e398-49fa-9c2e-fe7262544d25 phys_ram=252884MB used_ram=512MB phys_disk=923GB used_disk=0GB total_vcpus=40 used_vcpus=0 pci_stats=[]
2017-11-18 17:52:13.988 26958 INFO nova.compute.claims [req-8a3dfab9-5459-46ba-8e00-92926a83fa22 9380ba1286de4db2822e1170ea2f4e0d 0bdeb9f9f30b4b2fa9fc6d99fcec91e8 - - -] [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] Attempting claim: memory 1024 MB, disk 10 GB, vcpus 1 CPU
2017-11-18 17:52:13.988 26958 INFO nova.compute.claims [req-8a3dfab9-5459-46ba-8e00-92926a83fa22 9380ba1286de4db2822e1170ea2f4e0d 0bdeb9f9f30b4b2fa9fc6d99fcec91e8 - - -] [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] Total memory: 252884 MB, used: 512.00 MB
2017-11-18 17:52:13.989 26958 INFO nova.compute.claims [req-8a3dfab9-5459-46ba-8e00-92926a83fa22 9380ba1286de4db2822e1170ea2f4e0d 0bdeb9f9f30b4b2fa9fc6d99fcec91e8 - - -] [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] memory limit: 379326.00 MB, free: 378814.00 MB
2017-11-18 17:52:13.989 26958 INFO nova.compute.claims [req-8a3dfab9-5459-46ba-8e00-92926a83fa22 9380ba1286de4db2822e1170ea2f4e0d 0bdeb9f9f30b4b2fa9fc6d99fcec91e8 - - -] [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] Total disk: 923 GB, used: 0.00 GB
2017-11-18 17:52:13.990 26958 INFO nova.compute.claims [req-8a3dfab9-5459-46ba-8e00-92926a83fa22 9380ba1286de4db2822e1170ea2f4e0d 0bdeb9f9f30b4b2fa9fc6d99fcec91e8 - - -] [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] disk limit: 923.00 GB, free: 923.00 GB
2017-11-18 17:52:13.990 26958 INFO nova.compute.claims [req-8a3dfab9-5459-46ba-8e00-92926a83fa22 9380ba1286de4db2822e1170ea2f4e0d 0bdeb9f9f30b4b2fa9fc6d99fcec91e8 - - -] [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] Total vcpu: 40 VCPU, used: 0.00 VCPU
2017-11-18 17:52:13.991 26958 INFO nova.compute.claims [req-8a3dfab9-5459-46ba-8e00-92926a83fa22 9380ba1286de4db2822e1170ea2f4e0d 0bdeb9f9f30b4b2fa9fc6d99fcec91e8 - - -] [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] vcpu limit not specified, defaulting to unlimited
2017-11-18 17:52:13.991 26958 INFO nova.compute.claims [req-8a3dfab9-5459-46ba-8e00-92926a83fa22 9380ba1286de4db2822e1170ea2f4e0d 0bdeb9f9f30b4b2fa9fc6d99fcec91e8 - - -] [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] Claim successful
2017-11-18 17:52:14.225 26958 INFO nova.scheduler.client.report [req-8a3dfab9-5459-46ba-8e00-92926a83fa22 9380ba1286de4db2822e1170ea2f4e0d 0bdeb9f9f30b4b2fa9fc6d99fcec91e8 - - -] [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] Submitted allocation for instance
2017-11-18 17:52:15.595 26958 ERROR nova.compute.manager [req-8a3dfab9-5459-46ba-8e00-92926a83fa22 9380ba1286de4db2822e1170ea2f4e0d 0bdeb9f9f30b4b2fa9fc6d99fcec91e8 - - -] Instance failed network setup after 1 attempt(s)
2017-11-18 17:52:15.595 26958 ERROR nova.compute.manager Traceback (most recent call last):
2017-11-18 17:52:15.595 26958 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1399, in _allocate_network_async
2017-11-18 17:52:15.595 26958 ERROR nova.compute.manager bind_host_id=bind_host_id)
2017-11-18 17:52:15.595 26958 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 873, in allocate_for_instance
2017-11-18 17:52:15.595 26958 ERROR nova.compute.manager bind_host_id, dhcp_options, available_macs)
2017-11-18 17:52:15.595 26958 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 992, in _update_ports_for_instance
2017-11-18 17:52:15.595 26958 ERROR nova.compute.manager vif.destroy()
2017-11-18 17:52:15.595 26958 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-11-18 17:52:15.595 26958 ERROR nova.compute.manager self.force_reraise()
2017-11-18 17:52:15.595 26958 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-11-18 17:52:15.595 26958 ERROR nova.compute.manager six.reraise(self.type_, self.value, self.tb)
2017-11-18 17:52:15.595 26958 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 962, in _update_ports_for_instance
2017-11-18 17:52:15.595 26958 ERROR nova.compute.manager port_client, instance, port_id, port_req_body)
2017-11-18 17:52:15.595 26958 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 427, in _update_port
2017-11-18 17:52:15.595 26958 ERROR nova.compute.manager port_response = port_client.update_port(port_id, port_req_body)
2017-11-18 17:52:15.595 26958 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 99, in wrapper
2017-11-18 17:52:15.595 26958 ERROR nova.compute.manager ret = obj(*args, **kwargs)
2017-11-18 17:52:15.595 26958 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 776, in update_port
2017-11-18 17:52:15.595 26958 ERROR nova.compute.manager return self.put(self.port_path % (port), body=body)
2017-11-18 17:52:15.595 26958 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 99, in wrapper
2017-11-18 17:52:15.595 26958 ERROR nova.compute.manager ret = obj(*args, **kwargs)
2017-11-18 17:52:15.595 26958 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 370, in put
2017-11-18 17:52:15.595 26958 ERROR nova.compute.manager headers=headers, params=params)
2017-11-18 17:52:15.595 26958 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 99, in wrapper
2017-11-18 17:52:15.595 26958 ERROR nova.compute.manager ret = obj(*args, **kwargs)
2017-11-18 17:52:15.595 26958 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 338, in retry_request
2017-11-18 17:52:15.595 26958 ERROR nova.compute.manager headers=headers, params=params)
2017-11-18 17:52:15.595 26958 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 99, in wrapper
2017-11-18 17:52:15.595 26958 ERROR nova.compute.manager ret = obj(*args, **kwargs)
2017-11-18 17:52:15.595 26958 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 301, in do_request
2017-11-18 17:52:15.595 26958 ERROR nova.compute.manager self._handle_fault_response(status_code, replybody, resp)
2017-11-18 17:52:15.595 26958 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 99, in wrapper
2017-11-18 17:52:15.595 26958 ERROR nova.compute.manager ret = obj(*args, **kwargs)
2017-11-18 17:52:15.595 26958 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 276, in _handle_fault_response
2017-11-18 17:52:15.595 26958 ERROR nova.compute.manager exception_handler_v20(status_code, error_body)
2017-11-18 17:52:15.595 26958 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 92, in exception_handler_v20
2017-11-18 17:52:15.595 26958 ERROR nova.compute.manager request_ids=request_ids)
2017-11-18 17:52:15.595 26958 ERROR nova.compute.manager BadRequest: Bad port request: Internal error : virtual router default-global-system-config:5a10s26 not found.
2017-11-18 17:52:15.595 26958 ERROR nova.compute.manager Neutron server returns request_ids: ['req-318dc0ca-c90c-4485-9f1a-861d8202ea89']
2017-11-18 17:52:15.595 26958 ERROR nova.compute.manager
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [req-8a3dfab9-5459-46ba-8e00-92926a83fa22 9380ba1286de4db2822e1170ea2f4e0d 0bdeb9f9f30b4b2fa9fc6d99fcec91e8 - - -] [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] Instance failed to spawn
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] Traceback (most recent call last):
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2124, in _build_resources
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] yield resources
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1930, in _build_and_run_instance
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] block_device_info=block_device_info)
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] File "/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/contrailVC.py", line 148, in spawn
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] if network_info:
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] File "/usr/lib/python2.7/dist-packages/nova/network/model.py", line 540, in __len__
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] return self._sync_wrapper(fn, *args, **kwargs)
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] File "/usr/lib/python2.7/dist-packages/nova/network/model.py", line 527, in _sync_wrapper
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] self.wait()
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] File "/usr/lib/python2.7/dist-packages/nova/network/model.py", line 565, in wait
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] self[:] = self._gt.wait()
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 175, in wait
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] return self._exit_event.wait()
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] File "/usr/lib/python2.7/dist-packages/eventlet/event.py", line 121, in wait
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] return hubs.get_hub().switch()
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] File "/usr/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 294, in switch
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] return self.greenlet.switch()
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 214, in main
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] result = function(*args, **kwargs)
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] File "/usr/lib/python2.7/dist-packages/nova/utils.py", line 1087, in context_wrapper
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] return func(*args, **kwargs)
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1416, in _allocate_network_async
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] six.reraise(*exc_info)
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1399, in _allocate_network_async
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] bind_host_id=bind_host_id)
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 873, in allocate_for_instance
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] bind_host_id, dhcp_options, available_macs)
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 992, in _update_ports_for_instance
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] vif.destroy()
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] self.force_reraise()
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] six.reraise(self.type_, self.value, self.tb)
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 962, in _update_ports_for_instance
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] port_client, instance, port_id, port_req_body)
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 427, in _update_port
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] port_response = port_client.update_port(port_id, port_req_body)
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 99, in wrapper
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] ret = obj(*args, **kwargs)
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 776, in update_port
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] return self.put(self.port_path % (port), body=body)
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 99, in wrapper
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] ret = obj(*args, **kwargs)
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 370, in put
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] headers=headers, params=params)
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 99, in wrapper
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] ret = obj(*args, **kwargs)
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 338, in retry_request
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] headers=headers, params=params)
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 99, in wrapper
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] ret = obj(*args, **kwargs)
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 301, in do_request
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] self._handle_fault_response(status_code, replybody, resp)
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 99, in wrapper
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] ret = obj(*args, **kwargs)
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 276, in _handle_fault_response
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] exception_handler_v20(status_code, error_body)
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 92, in exception_handler_v20
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] request_ids=request_ids)
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] BadRequest: Bad port request: Internal error : virtual router default-global-system-config:5a10s26 not found.
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] Neutron server returns request_ids: ['req-318dc0ca-c90c-4485-9f1a-861d8202ea89']
2017-11-18 17:52:15.597 26958 ERROR nova.compute.manager [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5]
2017-11-18 17:52:15.599 26958 INFO nova.compute.manager [req-8a3dfab9-5459-46ba-8e00-92926a83fa22 9380ba1286de4db2822e1170ea2f4e0d 0bdeb9f9f30b4b2fa9fc6d99fcec91e8 - - -] [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] Terminating instance
2017-11-18 17:52:15.745 26958 WARNING nova.virt.vmwareapi.vmops [req-8a3dfab9-5459-46ba-8e00-92926a83fa22 9380ba1286de4db2822e1170ea2f4e0d 0bdeb9f9f30b4b2fa9fc6d99fcec91e8 - - -] [instance: 3b2d4a85-286d-465f-ae32-bb76afdcc0d5] Instance does not exist on backend
2017-11-18 17:52:15.745 26958

From: Sachchidanand Vaidya
Sent: Saturday, November 18, 2017 12:11 PM
To: Sarathbabu Narasimhan <email address hidden>; Kamlesh Parmar <email address hidden>; Sachin Bansal <email address hidden>; Ram Yadav <email address hidden>
Cc: Jeba Paulaiyan <email address hidden>; Amudha R <email address hidden>
Subject: Re: R4.1 ocata not launching VM on Vcenter-compute

Hi Sarath,
  Can you paste nova-compute backTrace ? Is there any error from Neutron.
Is ‘5a10s26’ nova-compute node ?

>>Bad port request: Internal error : virtual router default-global-system-config:5a10s26 not found.

Vcenter-as-compute is a special case as there is no vrouter where nova-compute is running.

Thanks,
Sachin
From: Sarathbabu Narasimhan <email address hidden>
Date: Saturday, November 18, 2017 at 1:12 AM
To: Kamlesh Parmar <email address hidden>
Cc: Jeba Paulaiyan <email address hidden>, Sachchidanand Vaidya <email address hidden>, Amudha R <email address hidden>, Sarathbabu Narasimhan <email address hidden>
Subject: R4.1 ocata not launching VM on Vcenter-compute

Hi Kamlesh,

This is latest R4.1 ocata #2 build and after you patched, provisioining success and the issue seen during sanity tests.

Based on my initial triaging,
>> when vm launched on KVM it works good and issue seen only when Esxi/vcenter-compute

>> Vcenter-compute hypervisor shown in openstack good and VM launch errors with below message seen on horizon UI,
Message
Bad port request: Internal error : virtual router default-global-system-config:5a10s26 not found. Neutron server returns request_ids: ['req-71513c50-f6a6-4c0f-83f1-8758bf00d930']
Code
500
Details
File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1780, in _do_build_and_run_instance filter_properties) File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2016, in _build_and_run_instance instance_uuid=instance.uuid, reason=six.text_type(e))
Created
Nov. 18, 2017, 8:53 a.m.

>> when checking vcenter-plugin there was no message when launching VM and found API hostname showing possibly incorrect as “10.87.36.15” and there is no api-server on this host and so changed this to 10.87.36.10 and restarted
     Plugin to try and it shown old but still new VM launch errors not seen any.

Please find the setup in problem state,
  Due to workaround for other bug#1716297, this topology SM-lite is on “LB/vcplugin” node which is 10.87.36.15 and you can find JSON file at /root/ which give you complete details of topology.

Please call me as required for this issue.

Thanks
*Sarath

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R4.1

Review in progress for https://review.opencontrail.org/37740
Submitter: ryadav (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/37740
Committed: http://github.com/Juniper/contrail-controller/commit/8a85d3ed6bf3d694031a7b716deb4941ca89ff55
Submitter: Zuul (<email address hidden>)
Branch: R4.1

commit 8a85d3ed6bf3d694031a7b716deb4941ca89ff55
Author: ram-yadav <email address hidden>
Date: Tue Nov 21 11:33:03 2017 -0800

Ocata VM launch failure on Esxi host and guest VM not getting DHCP IP.

NoIdError exception suppressed and treated as non dpdk case to since
vcenter plugin cannot handle/expect the exception to be generated.
Resolved some serialization issues related to vif_details.

Closes-Bug:#1733273
Closes-Bug:#1733458

Change-Id: I89b1a76361a6be3d93c9312f3f6007cf3983621e

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] master

Review in progress for https://review.opencontrail.org/37795
Submitter: ryadav (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R4.0

Review in progress for https://review.opencontrail.org/37796
Submitter: ryadav (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R3.2

Review in progress for https://review.opencontrail.org/37797
Submitter: ryadav (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R4.0

Review in progress for https://review.opencontrail.org/37796
Submitter: ryadav (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] master

Review in progress for https://review.opencontrail.org/37795
Submitter: ryadav (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R3.2

Review in progress for https://review.opencontrail.org/37797
Submitter: ryadav (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/37797
Committed: http://github.com/Juniper/contrail-controller/commit/847df127404903e14a2feedd8ab514788209ab74
Submitter: Zuul (<email address hidden>)
Branch: R3.2

commit 847df127404903e14a2feedd8ab514788209ab74
Author: ram-yadav <email address hidden>
Date: Tue Nov 21 11:33:03 2017 -0800

Ocata VM launch failure on Esxi host and guest VM not getting DHCP IP.

NoIdError exception suppressed and treated as non dpdk case to since
vcenter plugin cannot handle/expect the exception to be generated.
Resolved some serialization issues related to vif_details.

Closes-Bug: #1733273
Closes-Bug: #1733458

Change-Id: I89b1a76361a6be3d93c9312f3f6007cf3983621e
(cherry picked from commit 8a85d3ed6bf3d694031a7b716deb4941ca89ff55)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/37796
Committed: http://github.com/Juniper/contrail-controller/commit/9bdd66772e593b091afe021692650786e9844590
Submitter: Zuul (<email address hidden>)
Branch: R4.0

commit 9bdd66772e593b091afe021692650786e9844590
Author: ram-yadav <email address hidden>
Date: Tue Nov 21 11:33:03 2017 -0800

Ocata VM launch failure on Esxi host and guest VM not getting DHCP IP.

NoIdError exception suppressed and treated as non dpdk case to since
vcenter plugin cannot handle/expect the exception to be generated.
Resolved some serialization issues related to vif_details.

Closes-Bug: #1733273
Closes-Bug: #1733458

Change-Id: I89b1a76361a6be3d93c9312f3f6007cf3983621e
(cherry picked from commit 8a85d3ed6bf3d694031a7b716deb4941ca89ff55)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/37795
Committed: http://github.com/Juniper/contrail-controller/commit/dec9f0f61c47df8273f2e03bf1821e90e5c5a7d3
Submitter: Zuul (<email address hidden>)
Branch: master

commit dec9f0f61c47df8273f2e03bf1821e90e5c5a7d3
Author: ram-yadav <email address hidden>
Date: Tue Nov 21 11:33:03 2017 -0800

Ocata VM launch failure on Esxi host and guest VM not getting DHCP IP.

NoIdError exception suppressed and treated as non dpdk case to since
vcenter plugin cannot handle/expect the exception to be generated.
Resolved some serialization issues related to vif_details.

Closes-Bug: #1733273
Closes-Bug: #1733458

Change-Id: I89b1a76361a6be3d93c9312f3f6007cf3983621e
(cherry picked from commit 8a85d3ed6bf3d694031a7b716deb4941ca89ff55)

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.