Actually there appears to be a regression here. In the current code, zfssarest.get_lun() throws a exception.VolumeNotFound if the LUN doesn't exist (https://github.com/openstack/cinder/blob/master/cinder/volume/drivers/zfssa/zfssarest.py#L746) but zfssaiscsi.delete_volume() only catches a exception.VolumeBackendAPIException (https://github.com/openstack/cinder/blob/master/cinder/volume/drivers/zfssa/zfssaiscsi.py#L337)

So we end up with:

2016-10-13 21:37:21.454 26 ERROR cinder.volume.drivers.zfssa.zfssarest [req-abe82fc1-2e99-486b-b146-573db1f7f5ef b2ae6b7bdac142ddb708a3550f61d998 66234fea2ccc42398a1ae5300c594d49 - default default] Error Getting Volume: 3385ece4-0d9f-45d2-8b5b-f2a2096a65e4 on Pool: pool-609 Project: imot04 Return code: 404 Message: Not Found.
2016-10-13 21:37:21.629 26 ERROR oslo_messaging.rpc.server [req-abe82fc1-2e99-486b-b146-573db1f7f5ef b2ae6b7bdac142ddb708a3550f61d998 66234fea2ccc42398a1ae5300c594d49 - default default] Exception during message handling
2016-10-13 21:37:21.629 26 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2016-10-13 21:37:21.629 26 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 133, in _process_incoming
2016-10-13 21:37:21.629 26 ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
2016-10-13 21:37:21.629 26 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 150, in dispatch
2016-10-13 21:37:21.629 26 ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
2016-10-13 21:37:21.629 26 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 121, in _do_dispatch
2016-10-13 21:37:21.629 26 ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
2016-10-13 21:37:21.629 26 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python2.7/site-packages/cinder/coordination.py", line 285, in wrapped
2016-10-13 21:37:21.629 26 ERROR oslo_messaging.rpc.server     return f(*a, **k)
2016-10-13 21:37:21.629 26 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python2.7/site-packages/cinder/volume/manager.py", line 759, in delete_volume
2016-10-13 21:37:21.629 26 ERROR oslo_messaging.rpc.server     'error_deleting')
2016-10-13 21:37:21.629 26 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-10-13 21:37:21.629 26 ERROR oslo_messaging.rpc.server     self.force_reraise()
2016-10-13 21:37:21.629 26 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-10-13 21:37:21.629 26 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
2016-10-13 21:37:21.629 26 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python2.7/site-packages/cinder/volume/manager.py", line 745, in delete_volume
2016-10-13 21:37:21.629 26 ERROR oslo_messaging.rpc.server     self.driver.delete_volume(volume)
2016-10-13 21:37:21.629 26 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python2.7/site-packages/cinder/volume/drivers/zfssa/zfssaiscsi.py", line 336, in delete_volume
2016-10-13 21:37:21.629 26 ERROR oslo_messaging.rpc.server     volume['name'])
2016-10-13 21:37:21.629 26 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python2.7/site-packages/cinder/volume/drivers/zfssa/zfssarest.py", line 746, in get_lun
2016-10-13 21:37:21.629 26 ERROR oslo_messaging.rpc.server     raise exception.VolumeNotFound(volume_id=lun)
2016-10-13 21:37:21.629 26 ERROR oslo_messaging.rpc.server VolumeNotFound: Volume 3385ece4-0d9f-45d2-8b5b-f2a2096a65e4 could not be found.
2016-10-13 21:37:21.629 26 ERROR oslo_messaging.rpc.server