Retested this afternoon from scratch. Two nodes with absolutely no shared storage between them will trigger the exact same exception. I also tried migrating instances in either direction, just to make sure it wasn't just an issue with the controller node.

In every case Nova claims that the source node is on shared storage and throws the exception.

Here's another log snippet from the most recent attempt:


2015-09-09 15:31:23.632 DEBUG keystoneclient.session [req-855df685-1203-4481-9a65-558408337709 admin admin] RESP: [200] content-length: 262 x-compute-request-id: req-9c71a22d-6d21-46e5-bf29-fe000910645d connection: keep-alive date: Wed, 09 Sep 2015 22:27:27 GMT content-type: application/json x-openstack-request-id: req-9c71a22d-6d21-46e5-bf29-fe000910645d
RESP BODY: {"connection_info": {"driver_volume_type": "iscsi", "data": {"target_discovered": true, "encrypted": false, "qos_specs": null, "target_iqn": "iqn.2000-05.com.3pardata:20210002ac002db6", "target_portal": "172.16.1.24:3260", "target_lun": 0, "access_mode": "rw"}}}
 from (pid=14936) _http_log_response /usr/local/lib/python2.7/dist-packages/keystoneclient/session.py:215
2015-09-09 15:31:23.675 DEBUG nova.virt.libvirt.driver [req-855df685-1203-4481-9a65-558408337709 admin admin] skipping disk /dev/disk/by-path/ip-172.16.1.24:3260-iscsi-iqn.2000-05.com.3pardata:20210002ac002db6-lun-0 (vda) as it is a volume from (pid=14936) _get_instance_disk_info /opt/stack/nova/nova/virt/libvirt/driver.py:6476
2015-09-09 15:31:23.684 DEBUG nova.virt.libvirt.driver [req-855df685-1203-4481-9a65-558408337709 admin admin] skipping disk /dev/disk/by-path/ip-172.16.1.24:3260-iscsi-iqn.2000-05.com.3pardata:20210002ac002db6-lun-0 (vda) as it is a volume from (pid=14936) _get_instance_disk_info /opt/stack/nova/nova/virt/libvirt/driver.py:6476
2015-09-09 15:31:23.756 ERROR oslo_messaging.rpc.dispatcher [req-855df685-1203-4481-9a65-558408337709 admin admin] Exception during message handling: cld5b15 is not on local storage: Block migration can not be used with shared storage.
2015-09-09 15:31:23.756 TRACE oslo_messaging.rpc.dispatcher Traceback (most recent call last):
2015-09-09 15:31:23.756 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply
2015-09-09 15:31:23.756 TRACE oslo_messaging.rpc.dispatcher     executor_callback))
2015-09-09 15:31:23.756 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch
2015-09-09 15:31:23.756 TRACE oslo_messaging.rpc.dispatcher     executor_callback)
2015-09-09 15:31:23.756 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 129, in _do_dispatch
2015-09-09 15:31:23.756 TRACE oslo_messaging.rpc.dispatcher     result = func(ctxt, **new_args)
2015-09-09 15:31:23.756 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/exception.py", line 89, in wrapped
2015-09-09 15:31:23.756 TRACE oslo_messaging.rpc.dispatcher     payload)
2015-09-09 15:31:23.756 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 195, in __exit__
2015-09-09 15:31:23.756 TRACE oslo_messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
2015-09-09 15:31:23.756 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/exception.py", line 72, in wrapped
2015-09-09 15:31:23.756 TRACE oslo_messaging.rpc.dispatcher     return f(self, context, *args, **kw)
2015-09-09 15:31:23.756 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/compute/manager.py", line 399, in decorated_function
2015-09-09 15:31:23.756 TRACE oslo_messaging.rpc.dispatcher     return function(self, context, *args, **kwargs)
2015-09-09 15:31:23.756 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/compute/manager.py", line 377, in decorated_function
2015-09-09 15:31:23.756 TRACE oslo_messaging.rpc.dispatcher     kwargs['instance'], e, sys.exc_info())
2015-09-09 15:31:23.756 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 195, in __exit__
2015-09-09 15:31:23.756 TRACE oslo_messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
2015-09-09 15:31:23.756 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/compute/manager.py", line 365, in decorated_function
2015-09-09 15:31:23.756 TRACE oslo_messaging.rpc.dispatcher     return function(self, context, *args, **kwargs)
2015-09-09 15:31:23.756 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/compute/manager.py", line 4928, in check_can_live_migrate_source
2015-09-09 15:31:23.756 TRACE oslo_messaging.rpc.dispatcher     block_device_info)
2015-09-09 15:31:23.756 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 5168, in check_can_live_migrate_source
2015-09-09 15:31:23.756 TRACE oslo_messaging.rpc.dispatcher     raise exception.InvalidLocalStorage(reason=reason, path=source)
2015-09-09 15:31:23.756 TRACE oslo_messaging.rpc.dispatcher InvalidLocalStorage: cld5b15 is not on local storage: Block migration can not be used with shared storage.
2015-09-09 15:31:23.756 TRACE oslo_messaging.rpc.dispatcher
2015-09-09 15:31:23.758 ERROR oslo_messaging._drivers.common [req-855df685-1203-4481-9a65-558408337709 admin admin] Returning exception cld5b15 is not on local storage: Block migration can not be used with shared storage. to caller
2015-09-09 15:31:23.758 ERROR oslo_messaging._drivers.common [req-855df685-1203-4481-9a65-558408337709 admin admin] ['Traceback (most recent call last):\n', '  File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply\n    executor_callback))\n', '  File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch\n    executor_callback)\n', '  File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 129, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/nova/nova/exception.py", line 89, in wrapped\n    payload)\n', '  File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 195, in __exit__\n    six.reraise(self.type_, self.value, self.tb)\n', '  File "/opt/stack/nova/nova/exception.py", line 72, in wrapped\n    return f(self, context, *args, **kw)\n', '  File "/opt/stack/nova/nova/compute/manager.py", line 399, in decorated_function\n    return function(self, context, *args, **kwargs)\n', '  File "/opt/stack/nova/nova/compute/manager.py", line 377, in decorated_function\n    kwargs[\'instance\'], e, sys.exc_info())\n', '  File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 195, in __exit__\n    six.reraise(self.type_, self.value, self.tb)\n', '  File "/opt/stack/nova/nova/compute/manager.py", line 365, in decorated_function\n    return function(self, context, *args, **kwargs)\n', '  File "/opt/stack/nova/nova/compute/manager.py", line 4928, in check_can_live_migrate_source\n    block_device_info)\n', '  File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 5168, in check_can_live_migrate_source\n    raise exception.InvalidLocalStorage(reason=reason, path=source)\n', 'InvalidLocalStorage: cld5b15 is not on local storage: Block migration can not be used with shared storage.\n']