KeyError: 'instance_id' in manila/share/drivers/service_instance.py", line 354, in ensure_service_instance

[Matt Riedemann]

http://logs.openstack.org/95/273495/3/check/gate-manila-tempest-dsvm-neutron/5143008/logs/screen-m-shr.txt.gz?level=TRACE#_2016-02-03_20_33_50_321

2016-02-03 20:33:50.321 30256 ERROR oslo_messaging.rpc.dispatcher [req-c9747a98-4bdd-4d0d-abcd-66625378fd22 - - - - -] Exception during message handling: 'instance_id'
2016-02-03 20:33:50.321 30256 ERROR oslo_messaging.rpc.dispatcher Traceback (most recent call last):
2016-02-03 20:33:50.321 30256 ERROR oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 143, in _dispatch_and_reply
2016-02-03 20:33:50.321 30256 ERROR oslo_messaging.rpc.dispatcher executor_callback))
2016-02-03 20:33:50.321 30256 ERROR oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 189, in _dispatch
2016-02-03 20:33:50.321 30256 ERROR oslo_messaging.rpc.dispatcher executor_callback)
2016-02-03 20:33:50.321 30256 ERROR oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 130, in _do_dispatch
2016-02-03 20:33:50.321 30256 ERROR oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args)
2016-02-03 20:33:50.321 30256 ERROR oslo_messaging.rpc.dispatcher File "/opt/stack/new/manila/manila/share/manager.py", line 113, in wrapped
2016-02-03 20:33:50.321 30256 ERROR oslo_messaging.rpc.dispatcher return f(self, *args, **kwargs)
2016-02-03 20:33:50.321 30256 ERROR oslo_messaging.rpc.dispatcher File "/opt/stack/new/manila/manila/utils.py", line 614, in wrapper
2016-02-03 20:33:50.321 30256 ERROR oslo_messaging.rpc.dispatcher return func(self, *args, **kwargs)
2016-02-03 20:33:50.321 30256 ERROR oslo_messaging.rpc.dispatcher File "/opt/stack/new/manila/manila/share/manager.py", line 980, in delete_share_instance
2016-02-03 20:33:50.321 30256 ERROR oslo_messaging.rpc.dispatcher {'status': constants.STATUS_ERROR_DELETING})
2016-02-03 20:33:50.321 30256 ERROR oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 204, in __exit__
2016-02-03 20:33:50.321 30256 ERROR oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2016-02-03 20:33:50.321 30256 ERROR oslo_messaging.rpc.dispatcher File "/opt/stack/new/manila/manila/share/manager.py", line 974, in delete_share_instance
2016-02-03 20:33:50.321 30256 ERROR oslo_messaging.rpc.dispatcher share_server=share_server)
2016-02-03 20:33:50.321 30256 ERROR oslo_messaging.rpc.dispatcher File "/opt/stack/new/manila/manila/share/drivers/generic.py", line 718, in delete_share
2016-02-03 20:33:50.321 30256 ERROR oslo_messaging.rpc.dispatcher if self._is_share_server_active(context, share_server):
2016-02-03 20:33:50.321 30256 ERROR oslo_messaging.rpc.dispatcher File "/opt/stack/new/manila/manila/share/drivers/generic.py", line 710, in _is_share_server_active
2016-02-03 20:33:50.321 30256 ERROR oslo_messaging.rpc.dispatcher context, share_server['backend_details']))
2016-02-03 20:33:50.321 30256 ERROR oslo_messaging.rpc.dispatcher File "/opt/stack/new/manila/manila/share/drivers/service_instance.py", line 354, in ensure_service_instance
2016-02-03 20:33:50.321 30256 ERROR oslo_messaging.rpc.dispatcher server['instance_id'])
2016-02-03 20:33:50.321 30256 ERROR oslo_messaging.rpc.dispatcher KeyError: 'instance_id'
2016-02-03 20:33:50.321 30256 ERROR oslo_messaging.rpc.dispatcher

Looks like it started on 2/1:

http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22server%5B'instance_id'%5D)%5C%22%20AND%20tags%3A%5C%22screen-m-shr.txt%5C%22&from=7d

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/276812

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/276812
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=21baff116afd6eec05119946351026eb5354f241
Submitter: Jenkins
Branch: master

commit 21baff116afd6eec05119946351026eb5354f241
Author: Matt Riedemann <email address hidden>
Date: Fri Feb 5 08:22:12 2016 -0800

    Avoid KeyError on instance_id in ensure_service_instance

    There are cases where the server details don't have an instance_id
    set but we need to check for that so we can avoid a KeyError. We
    also log this case as a warning like the InstanceNotFound in the same
    method. Ideally we'd log the share server ID for context but we don't
    have that available in this method which is given ShareServerBackendDetails
    key/values.

    Change-Id: I8cabf7f3a5a3480a1f55a19fbdcebfc272906332
    Closes-Bug: #1542365

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/manila 2.0.0.0b3 development milestone.