With the latest code, cluster as the compute driver, Taking a snapshot is failing with following error :
ERROR oslo_messaging.rpc.dispatcher [req-9c49bfdd-641c-4022-90cc-aae697e387f3 admin admin] Exception during message handling: Error occurred while opening URL: https://10.1.214.7/nfc/52135ab3-7409-6a9c-30ba-65f00d264262/disk-1.vmdk for reading.
Cause: HTTPSConnectionPool(host='10.1.214.7', port=443): Max retries exceeded with url: /nfc/52135ab3-7409-6a9c-30ba-65f00d264262/disk-1.vmdk (Caused by ProxyError('Cannot connect to proxy.', error('Tunnel connection failed: 503 Service Unavailable',)))
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher Traceback (most recent call last):
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher executor_callback))
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher executor_callback)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 130, in _do_dispatch
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/exception.py", line 88, in wrapped
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher payload)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/exception.py", line 71, in wrapped
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher return f(self, context, *args, **kw)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/compute/manager.py", line 324, in decorated_function
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher LOG.warning(msg, e, instance_uuid=instance_uuid)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/compute/manager.py", line 295, in decorated_function
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher return function(self, context, *args, **kwargs)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/compute/manager.py", line 352, in decorated_function
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher kwargs['instance'], e, sys.exc_info())
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/compute/manager.py", line 340, in decorated_function
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher return function(self, context, *args, **kwargs)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/compute/manager.py", line 400, in decorated_function
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher instance=instance)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/compute/manager.py", line 390, in decorated_function
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher *args, **kwargs)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/compute/manager.py", line 3201, in snapshot_instance
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher task_states.IMAGE_SNAPSHOT)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/compute/manager.py", line 3231, in _snapshot_instance
2015-04-07 2
1:16:47.166 TRACE oslo_messaging.rpc.dispatcher update_task_state)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 506, in snapshot
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher self._vmops.snapshot(context, instance, image_id, update_task_state)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 833, in snapshot
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher vmdk_size=vmdk.capacity_in_bytes)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/virt/vmwareapi/images.py", line 457, in upload_image_stream_optimized
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher vmdk_size)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_vmware/rw_handles.py", line 500, in __init__
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher cacerts=cacerts)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_vmware/rw_handles.py", line 79, in _create_read_connection
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher raise exceptions.VimException(excep_msg, excep)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher VimException: Error occurred while opening URL: https://10.1.214.7/nfc/52135ab3-7409-6a9c-30ba-65f00d264262/disk-1.vmdk for reading.
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher Cause: HTTPSConnectionPool(host='10.1.214.7', port=443): Max retries exceeded with url: /nfc/52135ab3-7409-6a9c-30ba-65f00d264262/disk-1.vmdk (Caused by ProxyError('Cannot connect to proxy.', error('Tunnel connection failed: 503 Service Unavailable',)))
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher
git log -1 :
commit bf70df295b0529da9f4381f52fc94d328fa2fdb3
Merge: ad329b0 7560620
Author: Jenkins <email address hidden>
Date: Mon Apr 6 23:57:10 2015 +0000
Merge "neutronv2: only create client once when adding/removing fixed IPs"
Environment : VC 5.5 U1, ESXi 5.5.0, 1834933
Is this an environment issue ? Export OVS template task is stuck for a long time and it fails, attached the snapshot.
On further debugging, checking the ESXi hostd logs, contains this error
2015-04- 17T06:28: 49.081Z [FFD60920 verbose 'Vmsvc. vm:/vmfs/ volumes/ 535f3d61- 7c0442d6- 56c9-62552a5000 42/OSTACK_ SNAP_3dce2adb- fbe0-49ff- a606-aed25f69c8 75/OSTACK_ SNAP_3dce2adb- fbe0-49ff- a606-aed25f69c8 75.vmx' opID=2a248111-4f-bb user=vpxuser] Is disk present message: Expected device (ide0:1) does not exist.
OSTACK_ SNAP_3dce2adb- fbe0-49ff- a606-aed25f69c8 75 is the name of the OVF template which is being created as part of CreateSnapshot_Task
The OVF template does not get created and hence snapshot operation fails
Changed the adapter type to "lsiLogic" in glance image, still the same error
2015-04- 17T07:02: 25.686Z [2A440B70 verbose 'Vmsvc. vm:/vmfs/ volumes/ 535f3d61- 7c0442d6- 56c9-62552a5000 42/OSTACK_ SNAP_ce9e74ed- c847-4bc6- 8d47-10b8e924d3 a1/OSTACK_ SNAP_ce9e74ed- c847-4bc6- 8d47-10b8e924d3 a1.vmx' opID=74009579-27-56 user=vpxuser] Is disk present message: Expected device (scsi0:0) does not exist.
The issue is occurring on devstack - nova master branch
Glance image property ChinmayDevstack :~/devstack$ glance image-show 9e4f5f08- 364c-4f9e- ac4c-67642e2286 65 ------- ------- ------- ----+-- ------- ------- ------- ------- ------- -+ ------- ------- ------- ----+-- ------- ------- ------- ------- ------- -+ adaptertype' | lsiLogic | b4926b610769e5b a4 | 15T11:19: 00.000000 | 364c-4f9e- ac4c-67642e2286 65 | 0.3.2-i386- disk | fbfe1ad11fb24e3 c2 | 17T07:01: 58.000000 | ------- ------- ------- ----+-- ------- ------- ------- ------- ------- -+
chinmay@
+------
| Property | Value |
+------
| Property 'hw_vif_model' | e1000 |
| Property 'vmware_
| Property 'vmware_disktype' | sparse |
| checksum | 3fd19141ff969dc
| container_format | bare |
| created_at | 2015-04-
| deleted | False |
| disk_format | vmdk |
| id | 9e4f5f08-
| is_public | True |
| min_disk | 0 |
| min_ram | 0 |
| name | cirros-
| owner | e4cad1de0bb44f5
| protected | False |
| size | 17104896 |
| status | active |
| updated_at | 2015-04-
+------