Activity log for bug #1180044

Date Who What changed Old value New value Message
2013-05-14 18:27:08 Shawn Hartsock bug added bug
2013-05-15 02:00:59 Shawn Hartsock summary nova boot fails when a VMware vCenter managed data center is empty nova boot fails when any VMware vCenter managed datacenter object is empty
2013-05-15 02:02:06 Shawn Hartsock description When *any* VMware vCenter managed data center contains no hosts to provision an image, the nova boot command will fail silently. Empty data centers without ESX hosts are usually left behind by administrator error. Removing the empty data center resolves the issue. Stack trace from logs: ERROR nova.compute.manager [req-9395fe41-cf04-4434-bd77-663e93de1d4a foo bar] [instance: 484a42a2-642e-4594-93fe-4f72ddad361f] Error: ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 942, in _build_instance\n set_access_ip=set_access_ip)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1204, in _spawn\n LOG.exception(_(\'Instance failed to spawn\'), instance=instance)\n', ' File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__\n self.gen.next()\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1200, in _spawn\n block_device_info)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 176, in spawn\n block_device_info)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 208, in spawn\n _execute_create_vm()\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 204, in _execute_create_vm\n self._session._wait_for_task(instance[\'uuid\'], vm_create_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 559, in _wait_for_task\n ret_val = done.wait()\n', ' File "/usr/local/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait\n return hubs.get_hub().switch()\n', ' File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch\n return self.greenlet.switch()\n', 'NovaException: A specified parameter was not correct. \nspec.location.folder\n'] Work around: remove the empty data center. When *any* VMware vCenter managed data center contains no hosts to provision an image, the nova boot command will fail silently. Removing the empty data center resolves the issue completely. It doesn't matter what order the datacenters are listed in. Stack trace from logs: ERROR nova.compute.manager [req-9395fe41-cf04-4434-bd77-663e93de1d4a foo bar] [instance: 484a42a2-642e-4594-93fe-4f72ddad361f] Error: ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 942, in _build_instance\n set_access_ip=set_access_ip)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1204, in _spawn\n LOG.exception(_(\'Instance failed to spawn\'), instance=instance)\n', ' File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__\n self.gen.next()\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1200, in _spawn\n block_device_info)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 176, in spawn\n block_device_info)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 208, in spawn\n _execute_create_vm()\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 204, in _execute_create_vm\n self._session._wait_for_task(instance[\'uuid\'], vm_create_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 559, in _wait_for_task\n ret_val = done.wait()\n', ' File "/usr/local/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait\n return hubs.get_hub().switch()\n', ' File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch\n return self.greenlet.switch()\n', 'NovaException: A specified parameter was not correct. \nspec.location.folder\n'] Work around: remove the empty data center.
2013-05-17 17:24:44 Shawn Hartsock nova: assignee Shawn Hartsock (hartsock)
2013-05-17 17:25:14 Shawn Hartsock summary nova boot fails when any VMware vCenter managed datacenter object is empty nova boot fails when any VMware vCenter managed datacenter or container object is empty
2013-05-22 04:45:13 Michael Still nova: status New Confirmed
2013-05-22 04:45:16 Michael Still nova: importance Undecided High
2013-06-03 12:06:35 Shawn Hartsock description When *any* VMware vCenter managed data center contains no hosts to provision an image, the nova boot command will fail silently. Removing the empty data center resolves the issue completely. It doesn't matter what order the datacenters are listed in. Stack trace from logs: ERROR nova.compute.manager [req-9395fe41-cf04-4434-bd77-663e93de1d4a foo bar] [instance: 484a42a2-642e-4594-93fe-4f72ddad361f] Error: ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 942, in _build_instance\n set_access_ip=set_access_ip)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1204, in _spawn\n LOG.exception(_(\'Instance failed to spawn\'), instance=instance)\n', ' File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__\n self.gen.next()\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1200, in _spawn\n block_device_info)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 176, in spawn\n block_device_info)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 208, in spawn\n _execute_create_vm()\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 204, in _execute_create_vm\n self._session._wait_for_task(instance[\'uuid\'], vm_create_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 559, in _wait_for_task\n ret_val = done.wait()\n', ' File "/usr/local/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait\n return hubs.get_hub().switch()\n', ' File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch\n return self.greenlet.switch()\n', 'NovaException: A specified parameter was not correct. \nspec.location.folder\n'] Work around: remove the empty data center. When *any* VMware vCenter managed data center contains no hosts to provision an image, the nova boot command will fail silently. Removing the empty data center resolves the issue completely. It doesn't matter what order the datacenters are listed in. Stack trace from logs: ERROR nova.compute.manager [req-9395fe41-cf04-4434-bd77-663e93de1d4a foo bar] [instance: 484a42a2-642e-4594-93fe-4f72ddad361f] Error: ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 942, in _build_instance\n set_access_ip=set_access_ip)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1204, in _spawn\n LOG.exception(_(\'Instance failed to spawn\'), instance=instance)\n', ' File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__\n self.gen.next()\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1200, in _spawn\n block_device_info)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 176, in spawn\n block_device_info)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 208, in spawn\n _execute_create_vm()\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 204, in _execute_create_vm\n self._session._wait_for_task(instance[\'uuid\'], vm_create_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 559, in _wait_for_task\n ret_val = done.wait()\n', ' File "/usr/local/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait\n return hubs.get_hub().switch()\n', ' File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch\n return self.greenlet.switch()\n', 'NovaException: A specified parameter was not correct. \nspec.location.folder\n'] Work around: remove the empty data center. vCenter error is: "A specified parameter was not correct. spec.location.folder"
2013-06-03 12:07:10 Shawn Hartsock nova: milestone havana-2
2013-06-03 16:52:10 Shawn Hartsock nova: status Confirmed In Progress
2013-06-03 20:46:40 Shawn Hartsock summary nova boot fails when any VMware vCenter managed datacenter or container object is empty nova boot fails with multiple vCenter managed datacenters
2013-06-03 21:32:09 Sabari Murugesan nova: assignee Shawn Hartsock (hartsock) Sabari Kumar Murugesan (smurugesan)
2013-06-03 22:59:50 Shawn Hartsock nova: importance High Critical
2013-06-03 23:04:22 Shawn Hartsock tags vmware grizzly-backport-potential vmware
2013-06-04 15:53:17 Shawn Hartsock description When *any* VMware vCenter managed data center contains no hosts to provision an image, the nova boot command will fail silently. Removing the empty data center resolves the issue completely. It doesn't matter what order the datacenters are listed in. Stack trace from logs: ERROR nova.compute.manager [req-9395fe41-cf04-4434-bd77-663e93de1d4a foo bar] [instance: 484a42a2-642e-4594-93fe-4f72ddad361f] Error: ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 942, in _build_instance\n set_access_ip=set_access_ip)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1204, in _spawn\n LOG.exception(_(\'Instance failed to spawn\'), instance=instance)\n', ' File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__\n self.gen.next()\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1200, in _spawn\n block_device_info)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 176, in spawn\n block_device_info)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 208, in spawn\n _execute_create_vm()\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 204, in _execute_create_vm\n self._session._wait_for_task(instance[\'uuid\'], vm_create_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 559, in _wait_for_task\n ret_val = done.wait()\n', ' File "/usr/local/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait\n return hubs.get_hub().switch()\n', ' File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch\n return self.greenlet.switch()\n', 'NovaException: A specified parameter was not correct. \nspec.location.folder\n'] Work around: remove the empty data center. vCenter error is: "A specified parameter was not correct. spec.location.folder" When there is more than one data CENTER the driver can accidentally place the VMDK in a data STORE that is not accessible to the VM. For example... we place the VM at: dc0->OpenStackCluster->hostA->dsA then we place the VMDK at: dc1->OpenStackCluster->hostB->dsB Stack trace from logs: ERROR nova.compute.manager [req-9395fe41-cf04-4434-bd77-663e93de1d4a foo bar] [instance: 484a42a2-642e-4594-93fe-4f72ddad361f] Error: ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 942, in _build_instance\n set_access_ip=set_access_ip)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1204, in _spawn\n LOG.exception(_(\'Instance failed to spawn\'), instance=instance)\n', ' File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__\n self.gen.next()\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1200, in _spawn\n block_device_info)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 176, in spawn\n block_device_info)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 208, in spawn\n _execute_create_vm()\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 204, in _execute_create_vm\n self._session._wait_for_task(instance[\'uuid\'], vm_create_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 559, in _wait_for_task\n ret_val = done.wait()\n', ' File "/usr/local/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait\n return hubs.get_hub().switch()\n', ' File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch\n return self.greenlet.switch()\n', 'NovaException: A specified parameter was not correct. \nspec.location.folder\n'] Work around: remove the empty data center. vCenter error is: "A specified parameter was not correct. spec.location.folder"
2013-06-04 18:51:18 Shawn Hartsock nova: assignee Sabari Kumar Murugesan (smurugesan)
2013-06-04 18:51:23 Shawn Hartsock nova: assignee Shawn Hartsock (hartsock)
2013-06-06 22:31:52 Shawn Hartsock description When there is more than one data CENTER the driver can accidentally place the VMDK in a data STORE that is not accessible to the VM. For example... we place the VM at: dc0->OpenStackCluster->hostA->dsA then we place the VMDK at: dc1->OpenStackCluster->hostB->dsB Stack trace from logs: ERROR nova.compute.manager [req-9395fe41-cf04-4434-bd77-663e93de1d4a foo bar] [instance: 484a42a2-642e-4594-93fe-4f72ddad361f] Error: ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 942, in _build_instance\n set_access_ip=set_access_ip)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1204, in _spawn\n LOG.exception(_(\'Instance failed to spawn\'), instance=instance)\n', ' File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__\n self.gen.next()\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1200, in _spawn\n block_device_info)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 176, in spawn\n block_device_info)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 208, in spawn\n _execute_create_vm()\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 204, in _execute_create_vm\n self._session._wait_for_task(instance[\'uuid\'], vm_create_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 559, in _wait_for_task\n ret_val = done.wait()\n', ' File "/usr/local/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait\n return hubs.get_hub().switch()\n', ' File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch\n return self.greenlet.switch()\n', 'NovaException: A specified parameter was not correct. \nspec.location.folder\n'] Work around: remove the empty data center. vCenter error is: "A specified parameter was not correct. spec.location.folder" When there is more than one data CENTER the driver can accidentally place the VMDK in a data STORE that is not accessible to the VM. For example... we place the VM at:   dc0->OpenStackCluster->hostA->dsA then we place the VMDK at:   dc1->OpenStackCluster->hostB->dsB Stack trace from logs: ERROR nova.compute.manager [req-9395fe41-cf04-4434-bd77-663e93de1d4a foo bar] [instance: 484a42a2-642e-4594-93fe-4f72ddad361f] Error: ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 942, in _build_instance\n set_access_ip=set_access_ip)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1204, in _spawn\n LOG.exception(_(\'Instance failed to spawn\'), instance=instance)\n', ' File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__\n self.gen.next()\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1200, in _spawn\n block_device_info)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 176, in spawn\n block_device_info)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 208, in spawn\n _execute_create_vm()\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 204, in _execute_create_vm\n self._session._wait_for_task(instance[\'uuid\'], vm_create_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 559, in _wait_for_task\n ret_val = done.wait()\n', ' File "/usr/local/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait\n return hubs.get_hub().switch()\n', ' File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch\n return self.greenlet.switch()\n', 'NovaException: A specified parameter was not correct. \nspec.location.folder\n'] Work around: use only one datacenter. vCenter error is: "A specified parameter was not correct. spec.location.folder"
2013-06-06 23:26:42 Shawn Hartsock description When there is more than one data CENTER the driver can accidentally place the VMDK in a data STORE that is not accessible to the VM. For example... we place the VM at:   dc0->OpenStackCluster->hostA->dsA then we place the VMDK at:   dc1->OpenStackCluster->hostB->dsB Stack trace from logs: ERROR nova.compute.manager [req-9395fe41-cf04-4434-bd77-663e93de1d4a foo bar] [instance: 484a42a2-642e-4594-93fe-4f72ddad361f] Error: ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 942, in _build_instance\n set_access_ip=set_access_ip)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1204, in _spawn\n LOG.exception(_(\'Instance failed to spawn\'), instance=instance)\n', ' File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__\n self.gen.next()\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1200, in _spawn\n block_device_info)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 176, in spawn\n block_device_info)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 208, in spawn\n _execute_create_vm()\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 204, in _execute_create_vm\n self._session._wait_for_task(instance[\'uuid\'], vm_create_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 559, in _wait_for_task\n ret_val = done.wait()\n', ' File "/usr/local/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait\n return hubs.get_hub().switch()\n', ' File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch\n return self.greenlet.switch()\n', 'NovaException: A specified parameter was not correct. \nspec.location.folder\n'] Work around: use only one datacenter. vCenter error is: "A specified parameter was not correct. spec.location.folder" Root cause: when there is more than one HOST to choose from and no sensible default can be calculated by vCenter for the resource pool. IE: DRS is not turned on. When there is more than one data CENTER the driver can accidentally place the VMDK in a data STORE that is not accessible to the VM. For example... we place the VM at:   dc0->OpenStackCluster->hostA->dsA then we place the VMDK at:   dc1->OpenStackCluster->hostB->dsB Stack trace from logs: ERROR nova.compute.manager [req-9395fe41-cf04-4434-bd77-663e93de1d4a foo bar] [instance: 484a42a2-642e-4594-93fe-4f72ddad361f] Error: ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 942, in _build_instance\n set_access_ip=set_access_ip)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1204, in _spawn\n LOG.exception(_(\'Instance failed to spawn\'), instance=instance)\n', ' File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__\n self.gen.next()\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1200, in _spawn\n block_device_info)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 176, in spawn\n block_device_info)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 208, in spawn\n _execute_create_vm()\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 204, in _execute_create_vm\n self._session._wait_for_task(instance[\'uuid\'], vm_create_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 559, in _wait_for_task\n ret_val = done.wait()\n', ' File "/usr/local/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait\n return hubs.get_hub().switch()\n', ' File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch\n return self.greenlet.switch()\n', 'NovaException: A specified parameter was not correct. \nspec.location.folder\n'] Work around: use only one datacenter. vCenter error is: "A specified parameter was not correct. spec.location.folder"
2013-06-06 23:27:49 Shawn Hartsock description Root cause: when there is more than one HOST to choose from and no sensible default can be calculated by vCenter for the resource pool. IE: DRS is not turned on. When there is more than one data CENTER the driver can accidentally place the VMDK in a data STORE that is not accessible to the VM. For example... we place the VM at:   dc0->OpenStackCluster->hostA->dsA then we place the VMDK at:   dc1->OpenStackCluster->hostB->dsB Stack trace from logs: ERROR nova.compute.manager [req-9395fe41-cf04-4434-bd77-663e93de1d4a foo bar] [instance: 484a42a2-642e-4594-93fe-4f72ddad361f] Error: ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 942, in _build_instance\n set_access_ip=set_access_ip)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1204, in _spawn\n LOG.exception(_(\'Instance failed to spawn\'), instance=instance)\n', ' File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__\n self.gen.next()\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1200, in _spawn\n block_device_info)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 176, in spawn\n block_device_info)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 208, in spawn\n _execute_create_vm()\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 204, in _execute_create_vm\n self._session._wait_for_task(instance[\'uuid\'], vm_create_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 559, in _wait_for_task\n ret_val = done.wait()\n', ' File "/usr/local/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait\n return hubs.get_hub().switch()\n', ' File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch\n return self.greenlet.switch()\n', 'NovaException: A specified parameter was not correct. \nspec.location.folder\n'] Work around: use only one datacenter. vCenter error is: "A specified parameter was not correct. spec.location.folder" Root cause: when there is more than one HOST to choose from and no sensible default can be calculated by vCenter for the resource pool. IE: DRS is not turned on. When there is more than one data CENTER the driver does not know which datacenter to choose and fails the CreateVM_Task. For example... we place the VM at:   dc0->OpenStackCluster->hostA->dsA then we place the VMDK at:   dc1->OpenStackCluster->hostB->dsB Stack trace from logs: ERROR nova.compute.manager [req-9395fe41-cf04-4434-bd77-663e93de1d4a foo bar] [instance: 484a42a2-642e-4594-93fe-4f72ddad361f] Error: ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 942, in _build_instance\n set_access_ip=set_access_ip)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1204, in _spawn\n LOG.exception(_(\'Instance failed to spawn\'), instance=instance)\n', ' File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__\n self.gen.next()\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1200, in _spawn\n block_device_info)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 176, in spawn\n block_device_info)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 208, in spawn\n _execute_create_vm()\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 204, in _execute_create_vm\n self._session._wait_for_task(instance[\'uuid\'], vm_create_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 559, in _wait_for_task\n ret_val = done.wait()\n', ' File "/usr/local/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait\n return hubs.get_hub().switch()\n', ' File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch\n return self.greenlet.switch()\n', 'NovaException: A specified parameter was not correct. \nspec.location.folder\n'] Work around: use only one datacenter. vCenter error is: "A specified parameter was not correct. spec.location.folder"
2013-06-06 23:28:27 Shawn Hartsock description Root cause: when there is more than one HOST to choose from and no sensible default can be calculated by vCenter for the resource pool. IE: DRS is not turned on. When there is more than one data CENTER the driver does not know which datacenter to choose and fails the CreateVM_Task. For example... we place the VM at:   dc0->OpenStackCluster->hostA->dsA then we place the VMDK at:   dc1->OpenStackCluster->hostB->dsB Stack trace from logs: ERROR nova.compute.manager [req-9395fe41-cf04-4434-bd77-663e93de1d4a foo bar] [instance: 484a42a2-642e-4594-93fe-4f72ddad361f] Error: ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 942, in _build_instance\n set_access_ip=set_access_ip)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1204, in _spawn\n LOG.exception(_(\'Instance failed to spawn\'), instance=instance)\n', ' File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__\n self.gen.next()\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1200, in _spawn\n block_device_info)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 176, in spawn\n block_device_info)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 208, in spawn\n _execute_create_vm()\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 204, in _execute_create_vm\n self._session._wait_for_task(instance[\'uuid\'], vm_create_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 559, in _wait_for_task\n ret_val = done.wait()\n', ' File "/usr/local/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait\n return hubs.get_hub().switch()\n', ' File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch\n return self.greenlet.switch()\n', 'NovaException: A specified parameter was not correct. \nspec.location.folder\n'] Work around: use only one datacenter. vCenter error is: "A specified parameter was not correct. spec.location.folder" Root cause: when there is more than one HOST to choose from and no sensible default can be calculated by vCenter for the resource pool. IE: DRS is not turned on. When there is more than one data CENTER the driver does not know which datacenter to choose and fails the CreateVM_Task. For example... do we place the VM at:   dc0->OpenStackClusterA->hostA or at:   dc1->OpenStackClusterB->hostB Stack trace from logs: ERROR nova.compute.manager [req-9395fe41-cf04-4434-bd77-663e93de1d4a foo bar] [instance: 484a42a2-642e-4594-93fe-4f72ddad361f] Error: ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 942, in _build_instance\n set_access_ip=set_access_ip)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1204, in _spawn\n LOG.exception(_(\'Instance failed to spawn\'), instance=instance)\n', ' File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__\n self.gen.next()\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1200, in _spawn\n block_device_info)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 176, in spawn\n block_device_info)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 208, in spawn\n _execute_create_vm()\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 204, in _execute_create_vm\n self._session._wait_for_task(instance[\'uuid\'], vm_create_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 559, in _wait_for_task\n ret_val = done.wait()\n', ' File "/usr/local/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait\n return hubs.get_hub().switch()\n', ' File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch\n return self.greenlet.switch()\n', 'NovaException: A specified parameter was not correct. \nspec.location.folder\n'] Work around: use only one datacenter. vCenter error is: "A specified parameter was not correct. spec.location.folder"
2013-06-06 23:29:12 Shawn Hartsock description Root cause: when there is more than one HOST to choose from and no sensible default can be calculated by vCenter for the resource pool. IE: DRS is not turned on. When there is more than one data CENTER the driver does not know which datacenter to choose and fails the CreateVM_Task. For example... do we place the VM at:   dc0->OpenStackClusterA->hostA or at:   dc1->OpenStackClusterB->hostB Stack trace from logs: ERROR nova.compute.manager [req-9395fe41-cf04-4434-bd77-663e93de1d4a foo bar] [instance: 484a42a2-642e-4594-93fe-4f72ddad361f] Error: ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 942, in _build_instance\n set_access_ip=set_access_ip)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1204, in _spawn\n LOG.exception(_(\'Instance failed to spawn\'), instance=instance)\n', ' File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__\n self.gen.next()\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1200, in _spawn\n block_device_info)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 176, in spawn\n block_device_info)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 208, in spawn\n _execute_create_vm()\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 204, in _execute_create_vm\n self._session._wait_for_task(instance[\'uuid\'], vm_create_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 559, in _wait_for_task\n ret_val = done.wait()\n', ' File "/usr/local/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait\n return hubs.get_hub().switch()\n', ' File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch\n return self.greenlet.switch()\n', 'NovaException: A specified parameter was not correct. \nspec.location.folder\n'] Work around: use only one datacenter. vCenter error is: "A specified parameter was not correct. spec.location.folder" Root cause: when there is more than one HOST to choose from and no sensible default can be calculated by vCenter for the resource pool. IE: DRS is not turned on. When there is more than one data CENTER the driver does not know which datacenter to choose and fails the CreateVM_Task. For example... do we place the VM at:   dc0->OpenStackClusterA->hostA or at:   dc1->OpenStackClusterB->hostB Stack trace from logs: ERROR nova.compute.manager [req-9395fe41-cf04-4434-bd77-663e93de1d4a foo bar] [instance: 484a42a2-642e-4594-93fe-4f72ddad361f] Error: ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 942, in _build_instance\n set_access_ip=set_access_ip)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1204, in _spawn\n LOG.exception(_(\'Instance failed to spawn\'), instance=instance)\n', ' File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__\n self.gen.next()\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1200, in _spawn\n block_device_info)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 176, in spawn\n block_device_info)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 208, in spawn\n _execute_create_vm()\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 204, in _execute_create_vm\n self._session._wait_for_task(instance[\'uuid\'], vm_create_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 559, in _wait_for_task\n ret_val = done.wait()\n', ' File "/usr/local/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait\n return hubs.get_hub().switch()\n', ' File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch\n return self.greenlet.switch()\n', 'NovaException: A specified parameter was not correct. \nspec.location.folder\n'] Work around: use only one datacenter, use only one cluster, turn on DRS vCenter error is: "A specified parameter was not correct. spec.location.folder"
2013-06-07 14:54:19 Shawn Hartsock description Root cause: when there is more than one HOST to choose from and no sensible default can be calculated by vCenter for the resource pool. IE: DRS is not turned on. When there is more than one data CENTER the driver does not know which datacenter to choose and fails the CreateVM_Task. For example... do we place the VM at:   dc0->OpenStackClusterA->hostA or at:   dc1->OpenStackClusterB->hostB Stack trace from logs: ERROR nova.compute.manager [req-9395fe41-cf04-4434-bd77-663e93de1d4a foo bar] [instance: 484a42a2-642e-4594-93fe-4f72ddad361f] Error: ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 942, in _build_instance\n set_access_ip=set_access_ip)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1204, in _spawn\n LOG.exception(_(\'Instance failed to spawn\'), instance=instance)\n', ' File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__\n self.gen.next()\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1200, in _spawn\n block_device_info)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 176, in spawn\n block_device_info)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 208, in spawn\n _execute_create_vm()\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 204, in _execute_create_vm\n self._session._wait_for_task(instance[\'uuid\'], vm_create_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 559, in _wait_for_task\n ret_val = done.wait()\n', ' File "/usr/local/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait\n return hubs.get_hub().switch()\n', ' File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch\n return self.greenlet.switch()\n', 'NovaException: A specified parameter was not correct. \nspec.location.folder\n'] Work around: use only one datacenter, use only one cluster, turn on DRS vCenter error is: "A specified parameter was not correct. spec.location.folder" Root cause: when there is more than one HOST to choose from and no sensible default can be calculated by vCenter for the resource pool. IE: DRS is not turned on. When there is more than one data CENTER the driver does not know which datacenter to choose and fails the CreateVM_Task. For example... do we place the VM at:   dc0->OpenStackClusterA->hostA or at:   dc1->OpenStackClusterB->hostB Stack trace from logs: ERROR nova.compute.manager [req-9395fe41-cf04-4434-bd77-663e93de1d4a foo bar] [instance: 484a42a2-642e-4594-93fe-4f72ddad361f] Error: ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 942, in _build_instance\n set_access_ip=set_access_ip)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1204, in _spawn\n LOG.exception(_(\'Instance failed to spawn\'), instance=instance)\n', ' File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__\n self.gen.next()\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1200, in _spawn\n block_device_info)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 176, in spawn\n block_device_info)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 208, in spawn\n _execute_create_vm()\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 204, in _execute_create_vm\n self._session._wait_for_task(instance[\'uuid\'], vm_create_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 559, in _wait_for_task\n ret_val = done.wait()\n', ' File "/usr/local/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait\n return hubs.get_hub().switch()\n', ' File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch\n return self.greenlet.switch()\n', 'NovaException: A specified parameter was not correct. \nspec.location.folder\n'] vCenter error is: "A specified parameter was not correct. spec.location.folder" Work around: use only one datacenter, use only one cluster, turn on DRS
2013-06-07 14:55:47 Shawn Hartsock summary nova boot fails with multiple vCenter managed datacenters nova boot fails when vCenter has multiple managed stand-alone hosts and no clear default host
2013-06-10 17:15:06 Shawn Hartsock summary nova boot fails when vCenter has multiple managed stand-alone hosts and no clear default host nova boot fails when vCenter has multiple managed hosts and no clear default host
2013-06-10 17:15:43 Shawn Hartsock nova: importance Critical High
2013-06-14 15:00:36 Shawn Hartsock nova: importance High Critical
2013-06-18 13:20:54 Thierry Carrez nova: importance Critical High
2013-06-18 13:53:01 Shawn Hartsock nova: importance High Critical
2013-06-18 13:53:21 Shawn Hartsock nova: importance Critical High
2013-06-18 14:02:27 Shawn Hartsock nova: importance High Critical
2013-06-18 14:03:03 Shawn Hartsock nova: importance Critical High
2013-07-17 12:24:10 Russell Bryant nova: milestone havana-2 havana-3
2013-07-30 16:38:47 Shawn Hartsock nova: importance High Low
2013-07-30 16:39:00 Shawn Hartsock nova: status In Progress Triaged
2013-08-06 16:17:56 Shawn Hartsock summary nova boot fails when vCenter has multiple managed hosts and no clear default host nova boot fails when vCenter has multiple datacenters, managed hosts, or clusters and no clear default host
2013-08-06 16:18:23 Shawn Hartsock summary nova boot fails when vCenter has multiple datacenters, managed hosts, or clusters and no clear default host nova failures when vCenter multiple datacenters, managed hosts, or clusters and no clear default host
2013-08-06 16:18:29 Shawn Hartsock nova: importance Low High
2013-08-06 16:18:34 Shawn Hartsock nova: status Triaged In Progress
2013-08-06 17:35:43 Shawn Hartsock summary nova failures when vCenter multiple datacenters, managed hosts, or clusters and no clear default host nova failures when vCenter has multiple datacenters, managed hosts, or clusters and no clear default host
2013-08-06 17:36:53 Shawn Hartsock description Root cause: when there is more than one HOST to choose from and no sensible default can be calculated by vCenter for the resource pool. IE: DRS is not turned on. When there is more than one data CENTER the driver does not know which datacenter to choose and fails the CreateVM_Task. For example... do we place the VM at:   dc0->OpenStackClusterA->hostA or at:   dc1->OpenStackClusterB->hostB Stack trace from logs: ERROR nova.compute.manager [req-9395fe41-cf04-4434-bd77-663e93de1d4a foo bar] [instance: 484a42a2-642e-4594-93fe-4f72ddad361f] Error: ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 942, in _build_instance\n set_access_ip=set_access_ip)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1204, in _spawn\n LOG.exception(_(\'Instance failed to spawn\'), instance=instance)\n', ' File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__\n self.gen.next()\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1200, in _spawn\n block_device_info)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 176, in spawn\n block_device_info)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 208, in spawn\n _execute_create_vm()\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 204, in _execute_create_vm\n self._session._wait_for_task(instance[\'uuid\'], vm_create_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 559, in _wait_for_task\n ret_val = done.wait()\n', ' File "/usr/local/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait\n return hubs.get_hub().switch()\n', ' File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch\n return self.greenlet.switch()\n', 'NovaException: A specified parameter was not correct. \nspec.location.folder\n'] vCenter error is: "A specified parameter was not correct. spec.location.folder" Work around: use only one datacenter, use only one cluster, turn on DRS Root cause: when there is more than one HOST to choose from and no sensible default can be calculated by vCenter for the resource pool. IE: DRS is not turned on. When there is more than one data CENTER the driver does not know which datacenter to choose and fails the CreateVM_Task. For example... do we place the VM at:   dc0->OpenStackClusterA->hostA or at:   dc1->OpenStackClusterB->hostB Stack trace from logs: ERROR nova.compute.manager [req-9395fe41-cf04-4434-bd77-663e93de1d4a foo bar] [instance: 484a42a2-642e-4594-93fe-4f72ddad361f] Error: ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 942, in _build_instance\n set_access_ip=set_access_ip)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1204, in _spawn\n LOG.exception(_(\'Instance failed to spawn\'), instance=instance)\n', ' File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__\n self.gen.next()\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1200, in _spawn\n block_device_info)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 176, in spawn\n block_device_info)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 208, in spawn\n _execute_create_vm()\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 204, in _execute_create_vm\n self._session._wait_for_task(instance[\'uuid\'], vm_create_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 559, in _wait_for_task\n ret_val = done.wait()\n', ' File "/usr/local/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait\n return hubs.get_hub().switch()\n', ' File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch\n return self.greenlet.switch()\n', 'NovaException: A specified parameter was not correct. \nspec.location.folder\n'] vCenter error is: "A specified parameter was not correct. spec.location.folder" Work around: use only one datacenter, use only one cluster, turn on DRS Additional failures: 2013-07-18 10:59:12.788 DEBUG nova.virt.vmwareapi.vmware_images [req-e8306ffe-c6c7-4d0f-a466-fb532375cbd3 7799f10ca7da47f3b2660feb363b370b 0e1771f8db984a3599596fae62609d9a] [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] Got image size of 687865856 for the image cde14862-60b8-4360-a145-06585b06577c get_vmdk_size_and_properties /usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/vmware_images.py:156 2013-07-18 10:59:12.963 WARNING nova.virt.vmwareapi.network_util [req-e8306ffe-c6c7-4d0f-a466-fb532375cbd3 7799f10ca7da47f3b2660feb363b370b 0e1771f8db984a3599596fae62609d9a] [(ManagedObjectReference){ value = "network-1501" _type = "Network" }, (ManagedObjectReference){ value = "network-1458" _type = "Network" }, (ManagedObjectReference){ value = "network-2085" _type = "Network" }, (ManagedObjectReference){ value = "network-1143" _type = "Network" }] 2013-07-18 10:59:13.326 DEBUG nova.virt.vmwareapi.vmops [req-e8306ffe-c6c7-4d0f-a466-fb532375cbd3 7799f10ca7da47f3b2660feb363b370b 0e1771f8db984a3599596fae62609d9a] [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] Creating VM on the ESX host _execute_create_vm /usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/vmops.py:207 2013-07-18 10:59:14.258 3145 DEBUG nova.openstack.common.rpc.amqp [-] Making synchronous call on conductor ... multicall /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py:583 2013-07-18 10:59:14.259 3145 DEBUG nova.openstack.common.rpc.amqp [-] MSG_ID is 8ef36d061a9341a09d3a5451df798673 multicall /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py:586 2013-07-18 10:59:14.259 3145 DEBUG nova.openstack.common.rpc.amqp [-] UNIQUE_ID is 680b790574c64a9783fd2138c43f5f6d. _add_unique_id /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py:337 2013-07-18 10:59:18.757 3145 WARNING nova.virt.vmwareapi.driver [-] Task [CreateVM_Task] (returnval){ value = "task-33558" _type = "Task" } status: error The input arguments had entities that did not belong to the same datacenter. 2013-07-18 10:59:18.758 ERROR nova.compute.manager [req-e8306ffe-c6c7-4d0f-a466-fb532375cbd3 7799f10ca7da47f3b2660feb363b370b 0e1771f8db984a3599596fae62609d9a] [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] Instance failed to spawn 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] Traceback (most recent call last): 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1103, in _spawn 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] block_device_info) 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] File "/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/driver.py", line 177, in spawn 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] block_device_info) 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] File "/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/vmops.py", line 217, in spawn 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] _execute_create_vm() 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] File "/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/vmops.py", line 213, in _execute_create_vm 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] self._session._wait_for_task(instance['uuid'], vm_create_task) 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] File "/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/driver.py", line 554, in _wait_for_task 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] ret_val = done.wait() 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] File "/usr/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] return hubs.get_hub().switch() 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] File "/usr/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] return self.greenlet.switch() 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] NovaException: The input arguments had entities that did not belong to the same datacenter. 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] 2013-07-18 10:59:20.029 ERROR nova.compute.manager [req-e8306ffe-c6c7-4d0f-a466-fb532375cbd3 7799f10ca7da47f3b2660feb363b370b 0e1771f8db984a3599596fae62609d9a] [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] Error: ['Traceback (most recent call last):\n', ' File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 848, in _run_instance\n set_access_ip=set_access_ip)\n', ' File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1107, in _spawn\n LOG.exception(_(\'Instance failed to spawn\'), instance=instance)\n', ' File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__\n self.gen.next()\n', ' File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1103, in _spawn\n block_device_info)\n', ' File "/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/driver.py", line 177, in spawn\n block_device_info)\n', ' File "/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/vmops.py", line 217, in spawn\n _execute_create_vm()\n', ' File "/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/vmops.py", line 213, in _execute_create_vm\n self._session._wait_for_task(instance[\'uuid\'], vm_create_task)\n', ' File "/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/driver.py", line 554, in _wait_for_task\n ret_val = done.wait()\n', ' File "/usr/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait\n return hubs.get_hub().switch()\n', ' File "/usr/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch\n return self.greenlet.switch()\n', 'NovaException: The input arguments had entities that did not belong to the same datacenter.\n'] 2013-07-18 10:59:23.831 3145 WARNING nova.virt.vmwareapi.driver [-] Task [CreateVM_Task] (returnval){ value = "task-33558" _type = "Task" } status: error The input arguments had entities that did not belong to the same datacenter. 2013-07-18 10:59:23.832 3145 WARNING nova.virt.vmwareapi.driver [-] In vmwareapi:_poll_task, Got this error Trying to re-send() an already-triggered event. 2013-07-18 10:59:23.833 3145 ERROR nova.utils [-] in fixed duration looping call 2013-07-18 10:59:23.833 3145 TRACE nova.utils Traceback (most recent call last): 2013-07-18 10:59:23.833 3145 TRACE nova.utils File "/usr/lib/python2.7/dist-packages/nova/utils.py", line 594, in _inner 2013-07-18 10:59:23.833 3145 TRACE nova.utils self.f(*self.args, **self.kw) 2013-07-18 10:59:23.833 3145 TRACE nova.utils File "/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/driver.py", line 580, in _poll_task 2013-07-18 10:59:23.833 3145 TRACE nova.utils done.send_exception(excep) 2013-07-18 10:59:23.833 3145 TRACE nova.utils File "/usr/lib/python2.7/dist-packages/eventlet/event.py", line 208, in send_exception 2013-07-18 10:59:23.833 3145 TRACE nova.utils return self.send(None, args) 2013-07-18 10:59:23.833 3145 TRACE nova.utils File "/usr/lib/python2.7/dist-packages/eventlet/event.py", line 150, in send 2013-07-18 10:59:23.833 3145 TRACE nova.utils assert self._result is NOT_USED, 'Trying to re-send() an already-triggered event.' 2013-07-18 10:59:23.833 3145 TRACE nova.utils AssertionError: Trying to re-send() an already-triggered event. 2013-07-18 10:59:23.833 3145 TRACE nova.utils
2013-08-26 20:56:29 Shawn Hartsock summary nova failures when vCenter has multiple datacenters, managed hosts, or clusters and no clear default host nova failures when vCenter has multiple datacenters
2013-08-26 20:57:44 Shawn Hartsock description Root cause: when there is more than one HOST to choose from and no sensible default can be calculated by vCenter for the resource pool. IE: DRS is not turned on. When there is more than one data CENTER the driver does not know which datacenter to choose and fails the CreateVM_Task. For example... do we place the VM at:   dc0->OpenStackClusterA->hostA or at:   dc1->OpenStackClusterB->hostB Stack trace from logs: ERROR nova.compute.manager [req-9395fe41-cf04-4434-bd77-663e93de1d4a foo bar] [instance: 484a42a2-642e-4594-93fe-4f72ddad361f] Error: ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 942, in _build_instance\n set_access_ip=set_access_ip)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1204, in _spawn\n LOG.exception(_(\'Instance failed to spawn\'), instance=instance)\n', ' File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__\n self.gen.next()\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1200, in _spawn\n block_device_info)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 176, in spawn\n block_device_info)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 208, in spawn\n _execute_create_vm()\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 204, in _execute_create_vm\n self._session._wait_for_task(instance[\'uuid\'], vm_create_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 559, in _wait_for_task\n ret_val = done.wait()\n', ' File "/usr/local/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait\n return hubs.get_hub().switch()\n', ' File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch\n return self.greenlet.switch()\n', 'NovaException: A specified parameter was not correct. \nspec.location.folder\n'] vCenter error is: "A specified parameter was not correct. spec.location.folder" Work around: use only one datacenter, use only one cluster, turn on DRS Additional failures: 2013-07-18 10:59:12.788 DEBUG nova.virt.vmwareapi.vmware_images [req-e8306ffe-c6c7-4d0f-a466-fb532375cbd3 7799f10ca7da47f3b2660feb363b370b 0e1771f8db984a3599596fae62609d9a] [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] Got image size of 687865856 for the image cde14862-60b8-4360-a145-06585b06577c get_vmdk_size_and_properties /usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/vmware_images.py:156 2013-07-18 10:59:12.963 WARNING nova.virt.vmwareapi.network_util [req-e8306ffe-c6c7-4d0f-a466-fb532375cbd3 7799f10ca7da47f3b2660feb363b370b 0e1771f8db984a3599596fae62609d9a] [(ManagedObjectReference){ value = "network-1501" _type = "Network" }, (ManagedObjectReference){ value = "network-1458" _type = "Network" }, (ManagedObjectReference){ value = "network-2085" _type = "Network" }, (ManagedObjectReference){ value = "network-1143" _type = "Network" }] 2013-07-18 10:59:13.326 DEBUG nova.virt.vmwareapi.vmops [req-e8306ffe-c6c7-4d0f-a466-fb532375cbd3 7799f10ca7da47f3b2660feb363b370b 0e1771f8db984a3599596fae62609d9a] [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] Creating VM on the ESX host _execute_create_vm /usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/vmops.py:207 2013-07-18 10:59:14.258 3145 DEBUG nova.openstack.common.rpc.amqp [-] Making synchronous call on conductor ... multicall /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py:583 2013-07-18 10:59:14.259 3145 DEBUG nova.openstack.common.rpc.amqp [-] MSG_ID is 8ef36d061a9341a09d3a5451df798673 multicall /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py:586 2013-07-18 10:59:14.259 3145 DEBUG nova.openstack.common.rpc.amqp [-] UNIQUE_ID is 680b790574c64a9783fd2138c43f5f6d. _add_unique_id /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py:337 2013-07-18 10:59:18.757 3145 WARNING nova.virt.vmwareapi.driver [-] Task [CreateVM_Task] (returnval){ value = "task-33558" _type = "Task" } status: error The input arguments had entities that did not belong to the same datacenter. 2013-07-18 10:59:18.758 ERROR nova.compute.manager [req-e8306ffe-c6c7-4d0f-a466-fb532375cbd3 7799f10ca7da47f3b2660feb363b370b 0e1771f8db984a3599596fae62609d9a] [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] Instance failed to spawn 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] Traceback (most recent call last): 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1103, in _spawn 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] block_device_info) 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] File "/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/driver.py", line 177, in spawn 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] block_device_info) 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] File "/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/vmops.py", line 217, in spawn 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] _execute_create_vm() 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] File "/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/vmops.py", line 213, in _execute_create_vm 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] self._session._wait_for_task(instance['uuid'], vm_create_task) 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] File "/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/driver.py", line 554, in _wait_for_task 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] ret_val = done.wait() 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] File "/usr/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] return hubs.get_hub().switch() 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] File "/usr/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] return self.greenlet.switch() 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] NovaException: The input arguments had entities that did not belong to the same datacenter. 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] 2013-07-18 10:59:20.029 ERROR nova.compute.manager [req-e8306ffe-c6c7-4d0f-a466-fb532375cbd3 7799f10ca7da47f3b2660feb363b370b 0e1771f8db984a3599596fae62609d9a] [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] Error: ['Traceback (most recent call last):\n', ' File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 848, in _run_instance\n set_access_ip=set_access_ip)\n', ' File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1107, in _spawn\n LOG.exception(_(\'Instance failed to spawn\'), instance=instance)\n', ' File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__\n self.gen.next()\n', ' File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1103, in _spawn\n block_device_info)\n', ' File "/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/driver.py", line 177, in spawn\n block_device_info)\n', ' File "/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/vmops.py", line 217, in spawn\n _execute_create_vm()\n', ' File "/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/vmops.py", line 213, in _execute_create_vm\n self._session._wait_for_task(instance[\'uuid\'], vm_create_task)\n', ' File "/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/driver.py", line 554, in _wait_for_task\n ret_val = done.wait()\n', ' File "/usr/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait\n return hubs.get_hub().switch()\n', ' File "/usr/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch\n return self.greenlet.switch()\n', 'NovaException: The input arguments had entities that did not belong to the same datacenter.\n'] 2013-07-18 10:59:23.831 3145 WARNING nova.virt.vmwareapi.driver [-] Task [CreateVM_Task] (returnval){ value = "task-33558" _type = "Task" } status: error The input arguments had entities that did not belong to the same datacenter. 2013-07-18 10:59:23.832 3145 WARNING nova.virt.vmwareapi.driver [-] In vmwareapi:_poll_task, Got this error Trying to re-send() an already-triggered event. 2013-07-18 10:59:23.833 3145 ERROR nova.utils [-] in fixed duration looping call 2013-07-18 10:59:23.833 3145 TRACE nova.utils Traceback (most recent call last): 2013-07-18 10:59:23.833 3145 TRACE nova.utils File "/usr/lib/python2.7/dist-packages/nova/utils.py", line 594, in _inner 2013-07-18 10:59:23.833 3145 TRACE nova.utils self.f(*self.args, **self.kw) 2013-07-18 10:59:23.833 3145 TRACE nova.utils File "/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/driver.py", line 580, in _poll_task 2013-07-18 10:59:23.833 3145 TRACE nova.utils done.send_exception(excep) 2013-07-18 10:59:23.833 3145 TRACE nova.utils File "/usr/lib/python2.7/dist-packages/eventlet/event.py", line 208, in send_exception 2013-07-18 10:59:23.833 3145 TRACE nova.utils return self.send(None, args) 2013-07-18 10:59:23.833 3145 TRACE nova.utils File "/usr/lib/python2.7/dist-packages/eventlet/event.py", line 150, in send 2013-07-18 10:59:23.833 3145 TRACE nova.utils assert self._result is NOT_USED, 'Trying to re-send() an already-triggered event.' 2013-07-18 10:59:23.833 3145 TRACE nova.utils AssertionError: Trying to re-send() an already-triggered event. 2013-07-18 10:59:23.833 3145 TRACE nova.utils The method at vmops.py _get_datacenter_ref_and_name does not calculate datacenter properly. def _get_datacenter_ref_and_name(self): """Get the datacenter name and the reference.""" dc_obj = self._session._call_method(vim_util, "get_objects", "Datacenter", ["name"]) vm_util._cancel_retrieve_if_necessary(self._session, dc_obj) return dc_obj.objects[0].obj, dc_obj.objects[0].propSet[0].val This will not be correct on systems with more than one datacenter. Stack trace from logs: ERROR nova.compute.manager [req-9395fe41-cf04-4434-bd77-663e93de1d4a foo bar] [instance: 484a42a2-642e-4594-93fe-4f72ddad361f] Error: ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 942, in _build_instance\n set_access_ip=set_access_ip)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1204, in _spawn\n LOG.exception(_(\'Instance failed to spawn\'), instance=instance)\n', ' File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__\n self.gen.next()\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1200, in _spawn\n block_device_info)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 176, in spawn\n block_device_info)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 208, in spawn\n _execute_create_vm()\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 204, in _execute_create_vm\n self._session._wait_for_task(instance[\'uuid\'], vm_create_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 559, in _wait_for_task\n ret_val = done.wait()\n', ' File "/usr/local/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait\n return hubs.get_hub().switch()\n', ' File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch\n return self.greenlet.switch()\n', 'NovaException: A specified parameter was not correct. \nspec.location.folder\n'] vCenter error is: "A specified parameter was not correct. spec.location.folder" Work around: use only one datacenter, use only one cluster, turn on DRS Additional failures: 2013-07-18 10:59:12.788 DEBUG nova.virt.vmwareapi.vmware_images [req-e8306ffe-c6c7-4d0f-a466-fb532375cbd3 7799f10ca7da47f3b2660feb363b370b 0e1771f8db984a3599596fae62609d9a] [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] Got image size of 687865856 for the image cde14862-60b8-4360-a145-06585b06577c get_vmdk_size_and_properties /usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/vmware_images.py:156 2013-07-18 10:59:12.963 WARNING nova.virt.vmwareapi.network_util [req-e8306ffe-c6c7-4d0f-a466-fb532375cbd3 7799f10ca7da47f3b2660feb363b370b 0e1771f8db984a3599596fae62609d9a] [(ManagedObjectReference){    value = "network-1501"    _type = "Network"  }, (ManagedObjectReference){    value = "network-1458"    _type = "Network"  }, (ManagedObjectReference){    value = "network-2085"    _type = "Network"  }, (ManagedObjectReference){    value = "network-1143"    _type = "Network"  }] 2013-07-18 10:59:13.326 DEBUG nova.virt.vmwareapi.vmops [req-e8306ffe-c6c7-4d0f-a466-fb532375cbd3 7799f10ca7da47f3b2660feb363b370b 0e1771f8db984a3599596fae62609d9a] [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] Creating VM on the ESX host _execute_create_vm /usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/vmops.py:207 2013-07-18 10:59:14.258 3145 DEBUG nova.openstack.common.rpc.amqp [-] Making synchronous call on conductor ... multicall /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py:583 2013-07-18 10:59:14.259 3145 DEBUG nova.openstack.common.rpc.amqp [-] MSG_ID is 8ef36d061a9341a09d3a5451df798673 multicall /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py:586 2013-07-18 10:59:14.259 3145 DEBUG nova.openstack.common.rpc.amqp [-] UNIQUE_ID is 680b790574c64a9783fd2138c43f5f6d. _add_unique_id /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py:337 2013-07-18 10:59:18.757 3145 WARNING nova.virt.vmwareapi.driver [-] Task [CreateVM_Task] (returnval){    value = "task-33558"    _type = "Task"  } status: error The input arguments had entities that did not belong to the same datacenter. 2013-07-18 10:59:18.758 ERROR nova.compute.manager [req-e8306ffe-c6c7-4d0f-a466-fb532375cbd3 7799f10ca7da47f3b2660feb363b370b 0e1771f8db984a3599596fae62609d9a] [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] Instance failed to spawn 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] Traceback (most recent call last): 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1103, in _spawn 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] block_device_info) 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] File "/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/driver.py", line 177, in spawn 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] block_device_info) 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] File "/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/vmops.py", line 217, in spawn 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] _execute_create_vm() 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] File "/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/vmops.py", line 213, in _execute_create_vm 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] self._session._wait_for_task(instance['uuid'], vm_create_task) 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] File "/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/driver.py", line 554, in _wait_for_task 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] ret_val = done.wait() 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] File "/usr/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] return hubs.get_hub().switch() 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] File "/usr/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] return self.greenlet.switch() 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] NovaException: The input arguments had entities that did not belong to the same datacenter. 2013-07-18 10:59:18.758 3145 TRACE nova.compute.manager [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] 2013-07-18 10:59:20.029 ERROR nova.compute.manager [req-e8306ffe-c6c7-4d0f-a466-fb532375cbd3 7799f10ca7da47f3b2660feb363b370b 0e1771f8db984a3599596fae62609d9a] [instance: 5b3961b6-38d9-409c-881e-fe50f67b1539] Error: ['Traceback (most recent call last):\n', ' File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 848, in _run_instance\n set_access_ip=set_access_ip)\n', ' File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1107, in _spawn\n LOG.exception(_(\'Instance failed to spawn\'), instance=instance)\n', ' File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__\n self.gen.next()\n', ' File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1103, in _spawn\n block_device_info)\n', ' File "/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/driver.py", line 177, in spawn\n block_device_info)\n', ' File "/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/vmops.py", line 217, in spawn\n _execute_create_vm()\n', ' File "/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/vmops.py", line 213, in _execute_create_vm\n self._session._wait_for_task(instance[\'uuid\'], vm_create_task)\n', ' File "/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/driver.py", line 554, in _wait_for_task\n ret_val = done.wait()\n', ' File "/usr/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait\n return hubs.get_hub().switch()\n', ' File "/usr/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch\n return self.greenlet.switch()\n', 'NovaException: The input arguments had entities that did not belong to the same datacenter.\n'] 2013-07-18 10:59:23.831 3145 WARNING nova.virt.vmwareapi.driver [-] Task [CreateVM_Task] (returnval){    value = "task-33558"    _type = "Task"  } status: error The input arguments had entities that did not belong to the same datacenter. 2013-07-18 10:59:23.832 3145 WARNING nova.virt.vmwareapi.driver [-] In vmwareapi:_poll_task, Got this error Trying to re-send() an already-triggered event. 2013-07-18 10:59:23.833 3145 ERROR nova.utils [-] in fixed duration looping call 2013-07-18 10:59:23.833 3145 TRACE nova.utils Traceback (most recent call last): 2013-07-18 10:59:23.833 3145 TRACE nova.utils File "/usr/lib/python2.7/dist-packages/nova/utils.py", line 594, in _inner 2013-07-18 10:59:23.833 3145 TRACE nova.utils self.f(*self.args, **self.kw) 2013-07-18 10:59:23.833 3145 TRACE nova.utils File "/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/driver.py", line 580, in _poll_task 2013-07-18 10:59:23.833 3145 TRACE nova.utils done.send_exception(excep) 2013-07-18 10:59:23.833 3145 TRACE nova.utils File "/usr/lib/python2.7/dist-packages/eventlet/event.py", line 208, in send_exception 2013-07-18 10:59:23.833 3145 TRACE nova.utils return self.send(None, args) 2013-07-18 10:59:23.833 3145 TRACE nova.utils File "/usr/lib/python2.7/dist-packages/eventlet/event.py", line 150, in send 2013-07-18 10:59:23.833 3145 TRACE nova.utils assert self._result is NOT_USED, 'Trying to re-send() an already-triggered event.' 2013-07-18 10:59:23.833 3145 TRACE nova.utils AssertionError: Trying to re-send() an already-triggered event. 2013-07-18 10:59:23.833 3145 TRACE nova.utils
2013-08-27 20:34:49 Shawn Hartsock nova: assignee Shawn Hartsock (hartsock) Gary Kotton (garyk)
2013-08-30 00:11:36 Shawn Hartsock nova: assignee Gary Kotton (garyk)
2013-08-30 00:11:51 Shawn Hartsock nova: assignee Vui Lam (vui)
2013-09-05 13:37:32 Thierry Carrez nova: milestone havana-3 havana-rc1
2013-09-10 21:40:14 Russell Bryant nova: importance High Medium
2013-09-23 19:09:52 Russell Bryant tags grizzly-backport-potential vmware grizzly-backport-potential havana-rc-potential vmware
2013-09-23 19:10:01 Russell Bryant nova: milestone havana-rc1
2013-09-26 15:03:17 Shawn Hartsock nova: assignee Vui Lam (vui) Shawn Hartsock (hartsock)
2013-10-14 14:27:28 Thierry Carrez tags grizzly-backport-potential havana-rc-potential vmware grizzly-backport-potential havana-backport-potential vmware
2013-10-17 13:22:45 Gary Kotton nova: importance Medium High
2013-10-18 13:23:42 Gary Kotton nova: assignee Shawn Hartsock (hartsock) Gary Kotton (garyk)
2013-10-18 13:23:47 Gary Kotton nova: milestone icehouse-1
2013-12-03 23:08:34 Russell Bryant nova: milestone icehouse-1 icehouse-2
2013-12-05 14:58:59 Gary Kotton bug task added openstack-vmwareapi-team
2013-12-05 14:59:08 Gary Kotton openstack-vmwareapi-team: importance Undecided Critical
2013-12-08 14:24:39 Gary Kotton openstack-vmwareapi-team: assignee Gary Kotton (garyk)
2013-12-16 20:15:09 OpenStack Infra nova: status In Progress Fix Committed
2014-01-10 21:06:23 Tracy Jones openstack-vmwareapi-team: status New Fix Committed
2014-01-22 16:10:24 Thierry Carrez nova: status Fix Committed Fix Released
2014-01-23 17:56:11 Alan Pevec nominated for series nova/havana
2014-01-23 17:56:11 Alan Pevec bug task added nova/havana
2014-01-23 17:56:40 Alan Pevec nova/havana: status New In Progress
2014-01-23 17:56:44 Alan Pevec nova/havana: importance Undecided High
2014-01-23 17:57:05 Alan Pevec nova/havana: assignee Gary Kotton (garyk)
2014-01-23 17:57:43 Alan Pevec tags grizzly-backport-potential havana-backport-potential vmware grizzly-backport-potential vmware
2014-02-05 09:44:39 Ante Karamatić bug task added nova (Ubuntu)
2014-02-11 11:01:47 Launchpad Janitor nova (Ubuntu): status New Confirmed
2014-02-12 10:50:09 Thierry Carrez nova/havana: status In Progress Fix Committed
2014-02-12 10:50:09 Thierry Carrez nova/havana: milestone 2013.2.2
2014-02-13 19:18:08 Alan Pevec nova/havana: status Fix Committed Fix Released
2014-03-30 23:07:06 Alan Pevec tags grizzly-backport-potential vmware vmware
2014-04-17 08:57:28 Thierry Carrez nova: milestone icehouse-2 2014.1
2014-08-01 12:58:05 James Page nova (Ubuntu): status Confirmed Fix Released