vmware: nova-compute will not start if some instance relying datastore not available

[zhu zhu]

First use vcenter driver to spawn some instances to one of the datastores that esxi is binding to. Later this datastore became unavailable due to certain reason(power off or network problem). Then when restart nova-compute, found that compute service will exit with errors. This will openstack compute not usable.

2014-07-03 01:38:13.961 3634 DEBUG nova.compute.manager [req-11bc0618-8696-464d-8820-7565db8f44c3 None None] [instance: 9428cf95-5
37f-48f6-b79e-faa981f6066d] NV-AC7AA80 Checking state _get_power_state /usr/lib/python2.6/site-packages/nova/compute/manager.py:10
54
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/eventlet/hubs/poll.py", line 97, in wait
    readers.get(fileno, noop).cb(fileno)
  File "/usr/lib/python2.6/site-packages/eventlet/greenthread.py", line 194, in main
    result = function(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/nova/openstack/common/service.py", line 480, in run_service
    service.start()
  File "/usr/lib/python2.6/site-packages/nova/service.py", line 180, in start
    self.manager.init_host()
  File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 1037, in init_host
    self._init_instance(context, instance)
  File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 865, in _init_instance
    try_reboot, reboot_type = self._retry_reboot(context, instance)
  File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 963, in _retry_reboot
    current_power_state = self._get_power_state(context, instance)
  File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 1056, in _get_power_state
    return self.driver.get_info(instance)["state"]
  File "/usr/lib/python2.6/site-packages/nova/virt/vmwareapi/driver.py", line 862, in get_info
    return _vmops.get_info(instance)
  File "/usr/lib/python2.6/site-packages/nova/virt/vmwareapi/vmops.py", line 1376, in get_info
    max_mem = int(query['summary.config.memorySizeMB']) * 1024
KeyError: 'summary.config.memorySizeMB'

[zhu zhu (zhuzhubj)]

https://review.openstack.org/#/c/104495/

[Qin Zhao (zhaoqin)]

It seems to be a must fix one. Thanks to Zhu Zhu for reporting it!

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/104495
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=488f88c4d021bf429f62ca46dd8299a70d31505e
Submitter: Jenkins
Branch: master

commit 488f88c4d021bf429f62ca46dd8299a70d31505e
Author: ZHU ZHU <email address hidden>
Date: Thu Jul 3 03:19:19 2014 -0500

    VMWare: Fix nova-compute crash when instance datastore not available

    During nova-compute service startup, init_host will try to get_info for
    instances. If datastore of certain instances become not reachable, some
    VM properties like 'summary.config.memorySizeMB' will not be available.
    This will cause nova-compute process exit with error.

    Change-Id: I6753ce5658b629c216ebfbaab610c8fd0755466a
    Closes-Bug: #1337236