VMware: cinder-volume operation fails with Exception in VMwareVcVmdkDriver

[Johnson koil raj]

Cinder-volume VMwareVcVmdkDriver throws the below mentioned exception when the driver is idle for long time, So the session retry failed with an exceptions

2014-09-17 12:06:44.279 12997 INFO cinder.volume.flows.manager.create_volume [req-cc74ae3b-78d0-4d5f-a6b7-f304c5163d05 f9445b196ed240368a017cb7fc9b6fab 5872a1cc7faa43bfa43c6158736734c2 - - -] Volume f3bde528-5e1c-4d68-b0a4-7360c0689d8b: being created as raw with specification: {'status': u'creating', 'volume_size': 2, 'volume_name': u'volume-f3bde528-5e1c-4d68-b0a4-7360c0689d8b'}
2014-09-17 12:06:44.361 12997 INFO requests.packages.urllib3.connectionpool [-] Resetting dropped connection: 192.168.92.226
2014-09-17 12:06:44.382 12997 DEBUG requests.packages.urllib3.connectionpool [-] "POST /sdk HTTP/1.1" 200 406 _make_request /opt/stack/venvs/cinder/local/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:362
2014-09-17 12:06:44.386 12997 DEBUG cinder.volume.drivers.vmware.api [-] Checking if the current session: 528e1196-3ea8-c2b7-f66b-62bd591dffb6 is active. _is_current_session_active /opt/stack/venvs/cinder/local/lib/python2.7/site-packages/cinder/volume/drivers/vmware/api.py:262
2014-09-17 12:06:44.394 12997 DEBUG requests.packages.urllib3.connectionpool [-] "POST /sdk HTTP/1.1" 500 656 _make_request /opt/stack/venvs/cinder/local/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:362
2014-09-17 12:06:44.398 12997 ERROR cinder.openstack.common.loopingcall [-] in dynamic looping call
2014-09-17 12:06:44.398 12997 TRACE cinder.openstack.common.loopingcall Traceback (most recent call last):
2014-09-17 12:06:44.398 12997 TRACE cinder.openstack.common.loopingcall File "/opt/stack/venvs/cinder/local/lib/python2.7/site-packages/cinder/openstack/common/loopingcall.py", line 123, in _inner
2014-09-17 12:06:44.398 12997 TRACE cinder.openstack.common.loopingcall idle = self.f(*self.args, **self.kw)
2014-09-17 12:06:44.398 12997 TRACE cinder.openstack.common.loopingcall File "/opt/stack/venvs/cinder/local/lib/python2.7/site-packages/cinder/volume/drivers/vmware/api.py", line 83, in _func
2014-09-17 12:06:44.398 12997 TRACE cinder.openstack.common.loopingcall raise excep
2014-09-17 12:06:44.398 12997 TRACE cinder.openstack.common.loopingcall AttributeError: 'NoneType' object has no attribute 'getChildren'
2014-09-17 12:06:44.398 12997 TRACE cinder.openstack.common.loopingcall
2014-09-17 12:06:44.403 12997 WARNING cinder.volume.manager [req-cc74ae3b-78d0-4d5f-a6b7-f304c5163d05 f9445b196ed240368a017cb7fc9b6fab 5872a1cc7faa43bfa43c6158736734c2 - - -] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d378ab70-9791-46ec-8965-04b81fc571f8) transitioned into state 'FAILURE'
2014-09-17 12:06:44.403 12997 TRACE cinder.volume.manager Traceback (most recent call last):
2014-09-17 12:06:44.403 12997 TRACE cinder.volume.manager File "/opt/stack/venvs/cinder/local/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 34, in _execute_task
2014-09-17 12:06:44.403 12997 TRACE cinder.volume.manager result = task.execute(**arguments)
2014-09-17 12:06:44.403 12997 TRACE cinder.volume.manager File "/opt/stack/venvs/cinder/local/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 583, in execute
2014-09-17 12:06:44.403 12997 TRACE cinder.volume.manager **volume_spec)
2014-09-17 12:06:44.403 12997 TRACE cinder.volume.manager File "/opt/stack/venvs/cinder/local/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 557, in _create_raw_volume
2014-09-17 12:06:44.403 12997 TRACE cinder.volume.manager return self.driver.create_volume(volume_ref)
2014-09-17 12:06:44.403 12997 TRACE cinder.volume.manager File "/opt/stack/venvs/cinder/local/lib/python2.7/site-packages/cinder/volume/drivers/vmware/vmdk.py", line 303, in create_volume
2014-09-17 12:06:44.403 12997 TRACE cinder.volume.manager self._verify_volume_creation(volume)
2014-09-17 12:06:44.403 12997 TRACE cinder.volume.manager File "/opt/stack/venvs/cinder/local/lib/python2.7/site-packages/cinder/volume/drivers/vmware/vmdk.py", line 287, in _verify_volume_creation
2014-09-17 12:06:44.403 12997 TRACE cinder.volume.manager self._select_ds_for_volume(volume)
2014-09-17 12:06:44.403 12997 TRACE cinder.volume.manager File "/opt/stack/venvs/cinder/local/lib/python2.7/site-packages/cinder/volume/drivers/vmware/vmdk.py", line 540, in _select_ds_for_volume
2014-09-17 12:06:44.403 12997 TRACE cinder.volume.manager retrv_result = self.volumeops.get_hosts()
2014-09-17 12:06:44.403 12997 TRACE cinder.volume.manager File "/opt/stack/venvs/cinder/local/lib/python2.7/site-packages/cinder/volume/drivers/vmware/volumeops.py", line 325, in get_hosts
2014-09-17 12:06:44.403 12997 TRACE cinder.volume.manager 'HostSystem', self._max_objects)
2014-09-17 12:06:44.403 12997 TRACE cinder.volume.manager File "/opt/stack/venvs/cinder/local/lib/python2.7/site-packages/cinder/volume/drivers/vmware/api.py", line 254, in invoke_api
2014-09-17 12:06:44.403 12997 TRACE cinder.volume.manager return _invoke_api(module, method, *args, **kwargs)
2014-09-17 12:06:44.403 12997 TRACE cinder.volume.manager File "/opt/stack/venvs/cinder/local/lib/python2.7/site-packages/cinder/volume/drivers/vmware/api.py", line 90, in func
2014-09-17 12:06:44.403 12997 TRACE cinder.volume.manager return timer.wait()
2014-09-17 12:06:44.403 12997 TRACE cinder.volume.manager File "/opt/stack/venvs/cinder/local/lib/python2.7/site-packages/eventlet/event.py", line 121, in wait
2014-09-17 12:06:44.403 12997 TRACE cinder.volume.manager return hubs.get_hub().switch()
2014-09-17 12:06:44.403 12997 TRACE cinder.volume.manager File "/opt/stack/venvs/cinder/local/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 293, in switch
2014-09-17 12:06:44.403 12997 TRACE cinder.volume.manager return self.greenlet.switch()
2014-09-17 12:06:44.403 12997 TRACE cinder.volume.manager File "/opt/stack/venvs/cinder/local/lib/python2.7/site-packages/cinder/openstack/common/loopingcall.py", line 123, in _inner
2014-09-17 12:06:44.403 12997 TRACE cinder.volume.manager idle = self.f(*self.args, **self.kw)
2014-09-17 12:06:44.403 12997 TRACE cinder.volume.manager File "/opt/stack/venvs/cinder/local/lib/python2.7/site-packages/cinder/volume/drivers/vmware/api.py", line 83, in _func
2014-09-17 12:06:44.403 12997 TRACE cinder.volume.manager raise excep
2014-09-17 12:06:44.403 12997 TRACE cinder.volume.manager AttributeError: 'NoneType' object has no attribute 'getChildren'
2014-09-17 12:06:44.403 12997 TRACE cinder.volume.manager
2014-09-17 12:06:44.406 12997 WARNING cinder.volume.manager [req-cc74ae3b-78d0-4d5f-a6b7-f304c5163d05 f9445b196ed240368a017cb7fc9b6fab 5872a1cc7faa43bfa43c6158736734c2 - - -] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d378ab70-9791-46ec-8965-04b81fc571f8) transitioned into state 'REVERTING'

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/124491

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/124491
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=a28c9709f3e3c820f125aeb18cff8e5d34da013b
Submitter: Jenkins
Branch: master

commit a28c9709f3e3c820f125aeb18cff8e5d34da013b
Author: Johnson Koil Raj <email address hidden>
Date: Sat Sep 27 00:07:47 2014 +0530

    VMware: cinder-volume create_session fail at retry

    cinder-volume exits with an exception while accessing a None
    object during retry of create session failure.

    Closes-Bug: #1374585

    Change-Id: I1d8dd034e9c6eb7568cf6f243a203df7de1777da