OpenStack Compute (nova)

KeyError: 'summary.guest.toolsStatus' in vmware driver

Series newton
Bug #1563325

Bug #1563325 reported by Luo Gangyi on 2016-03-29

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	OpenStack Compute (nova)	Fix Released	Low	Sivasathurappan Radhakrishnan
	Newton	Fix Committed	Low	Lee Yarwood

Bug Description

When I do a hard reboot of in vmware instances which do not have vmware tools , I get following error

2016-03-29 21:11:02.463 8893 ERROR oslo_messaging.rpc.dispatcher [req-181921d8-7253-4bfc-b613-6fbae41824b4 4412e38ec9814b96a03e63097ec51f1a 8f75187cd29f4715881f450646fc6e08 - - -] Exception during message handling: 'summary.guest.toolsStatus'
2016-03-29 21:11:02.463 8893 TRACE oslo_messaging.rpc.dispatcher Traceback (most recent call last):
2016-03-29 21:11:02.463 8893 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply
2016-03-29 21:11:02.463 8893 TRACE oslo_messaging.rpc.dispatcher executor_callback))
2016-03-29 21:11:02.463 8893 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch
2016-03-29 21:11:02.463 8893 TRACE oslo_messaging.rpc.dispatcher executor_callback)
2016-03-29 21:11:02.463 8893 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 130, in _do_dispatch
2016-03-29 21:11:02.463 8893 TRACE oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args)
2016-03-29 21:11:02.463 8893 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 6751, in reboot_instance
2016-03-29 21:11:02.463 8893 TRACE oslo_messaging.rpc.dispatcher reboot_type)
2016-03-29 21:11:02.463 8893 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/exception.py", line 88, in wrapped
2016-03-29 21:11:02.463 8893 TRACE oslo_messaging.rpc.dispatcher payload)
2016-03-29 21:11:02.463 8893 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 85, in __exit__
2016-03-29 21:11:02.463 8893 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2016-03-29 21:11:02.463 8893 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/exception.py", line 71, in wrapped
2016-03-29 21:11:02.463 8893 TRACE oslo_messaging.rpc.dispatcher return f(self, context, *args, **kw)
2016-03-29 21:11:02.463 8893 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 327, in decorated_function
2016-03-29 21:11:02.463 8893 TRACE oslo_messaging.rpc.dispatcher LOG.warning(msg, e, instance_uuid=instance_uuid)
2016-03-29 21:11:02.463 8893 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 85, in __exit__
2016-03-29 21:11:02.463 8893 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2016-03-29 21:11:02.463 8893 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 298, in decorated_function
2016-03-29 21:11:02.463 8893 TRACE oslo_messaging.rpc.dispatcher return function(self, context, *args, **kwargs)
2016-03-29 21:11:02.463 8893 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 377, in decorated_function
2016-03-29 21:11:02.463 8893 TRACE oslo_messaging.rpc.dispatcher return function(self, context, *args, **kwargs)
2016-03-29 21:11:02.463 8893 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 355, in decorated_function
2016-03-29 21:11:02.463 8893 TRACE oslo_messaging.rpc.dispatcher kwargs['instance'], e, sys.exc_info())
2016-03-29 21:11:02.463 8893 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 85, in __exit__
2016-03-29 21:11:02.463 8893 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2016-03-29 21:11:02.463 8893 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 343, in decorated_function
2016-03-29 21:11:02.463 8893 TRACE oslo_messaging.rpc.dispatcher return function(self, context, *args, **kwargs)
2016-03-29 21:11:02.463 8893 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 3211, in reboot_instance
2016-03-29 21:11:02.463 8893 TRACE oslo_messaging.rpc.dispatcher self._set_instance_obj_error_state(context, instance)
2016-03-29 21:11:02.463 8893 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 85, in __exit__
2016-03-29 21:11:02.463 8893 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2016-03-29 21:11:02.463 8893 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 3192, in reboot_instance
2016-03-29 21:11:02.463 8893 TRACE oslo_messaging.rpc.dispatcher bad_volumes_callback=bad_volumes_callback)
2016-03-29 21:11:02.463 8893 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/virt/vmwareapi/driver.py", line 512, in reboot
2016-03-29 21:11:02.463 8893 TRACE oslo_messaging.rpc.dispatcher self._vmops.reboot(instance, network_info, reboot_type)
2016-03-29 21:11:02.463 8893 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/virt/vmwareapi/vmops.py", line 894, in reboot
2016-03-29 21:11:02.463 8893 TRACE oslo_messaging.rpc.dispatcher tools_status = query['summary.guest.toolsStatus']
2016-03-29 21:11:02.463 8893 TRACE oslo_messaging.rpc.dispatcher KeyError: 'summary.guest.toolsStatus'
2016-03-29 21:11:02.463 8893 TRACE oslo_messaging.rpc.dispatcher
2016-03-29 21:11:02.472 8893 DEBUG oslo_vmware.api [-] Task: (returnval){

Tags:

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-03-29: Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/298793

Changed in nova:
assignee:	nobody → Luo Gangyi (luogangyi)
status:	New → In Progress

Sylvain Bauza (sylvain-bauza) on 2016-03-30

tags:	added: vmware
tags:	added: low-hanging-fruit
Changed in nova:
importance:	Undecided → Low

Sarafraj Singh (sarafraj-singh) on 2016-07-06

Changed in nova:
assignee:	Luo Gangyi (luogangyi) → Sarafraj Singh (sarafraj-singh)

Sarafraj Singh (sarafraj-singh) on 2016-08-04

Changed in nova:
assignee:	Sarafraj Singh (sarafraj-singh) → nobody

Robin Naundorf (senk) on 2016-08-21

Changed in nova:
status:	In Progress → Confirmed

Sivasathurappan Radhakrishnan (siva-radhakrishnan) on 2016-08-23

Changed in nova:
assignee:	nobody → Sivasathurappan Radhakrishnan (siva-radhakrishnan)

Sivasathurappan Radhakrishnan (siva-radhakrishnan) on 2016-08-23

Changed in nova:
status:	Confirmed → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-08-23:

Fix proposed to branch: master
Review: https://review.openstack.org/359424

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-10-25: Change abandoned on nova (master)

Change abandoned by Michael Still (<email address hidden>) on branch: master
Review: https://review.openstack.org/298793
Reason: This patch has been sitting unchanged for more than 12 weeks. I am therefore going to abandon it to keep the nova review queue sane. Please feel free to restore the change if you're still working on it.

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-11-26: Fix merged to nova (master)

Reviewed: https://review.openstack.org/359424
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=d3c7396d87681b82327c59c27fe14cf5e20c3ba5
Submitter: Jenkins
Branch: master

commit d3c7396d87681b82327c59c27fe14cf5e20c3ba5
Author: Sivasathurappan Radhakrishnan <email address hidden>
Date: Tue Aug 23 20:47:06 2016 +0000

Using get() method to prevent KeyError

    Using get() to access keys like "summary.guest.toolsStatus" and
    "summary.guest.toolrunningStatus" to avoid exception during hard reboot.
    This change would use a default values even if they don't exist in dict
    as we can have VMs where VM tools wouldn't be installed/running which leads
    to hard reboot

Change-Id: I332867fe783d6d47b81d8a53153af140d4158217
Closes-bug: #1563325

Changed in nova:
status:	In Progress → Fix Released

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-11-30: Fix proposed to nova (stable/newton)

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/404658

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-12-09: Fix merged to nova (stable/newton)

Reviewed: https://review.openstack.org/404658
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=e0f021f187758268e2bd732f63cf88fbf703320c
Submitter: Jenkins
Branch: stable/newton

commit e0f021f187758268e2bd732f63cf88fbf703320c
Author: Sivasathurappan Radhakrishnan <email address hidden>
Date: Tue Aug 23 20:47:06 2016 +0000

Using get() method to prevent KeyError

    Change-Id: I332867fe783d6d47b81d8a53153af140d4158217
    Closes-bug: #1563325
    (cherry picked from commit d3c7396d87681b82327c59c27fe14cf5e20c3ba5)