2016-03-23 02:50:25.249 24631 WARNING oslo_reports.guru_meditation_report [-] Guru mediation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports. 2016-03-23 02:50:26.904 24631 WARNING oslo_config.cfg [-] Option "notification_driver" from group "DEFAULT" is deprecated. Use option "driver" from group "oslo_messaging_notifications". 2016-03-23 02:50:30.585 24631 WARNING py.warnings [req-203a338c-84b1-4065-a6bc-cf17db85f079 - - - - -] /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py:241: NotSupportedWarning: Configuration option(s) ['use_tpool'] not supported exception.NotSupportedWarning 2016-03-23 02:50:31.210 24631 INFO cinder.rpc [req-203a338c-84b1-4065-a6bc-cf17db85f079 - - - - -] Automatically selected cinder-scheduler objects version None as minimum service version. 2016-03-23 02:50:31.240 24631 INFO cinder.rpc [req-203a338c-84b1-4065-a6bc-cf17db85f079 - - - - -] Automatically selected cinder-scheduler RPC version None as minimum service version. 2016-03-23 02:50:31.518 24631 INFO cinder.volume.manager [req-203a338c-84b1-4065-a6bc-cf17db85f079 - - - - -] Determined volume DB was empty at startup. 2016-03-23 02:50:31.575 24631 INFO cinder.volume.manager [req-203a338c-84b1-4065-a6bc-cf17db85f079 - - - - -] Service not found for updating active_backend_id, assuming default for driver init. 2016-03-23 02:50:32.087 24631 INFO cinder.volume.manager [req-203a338c-84b1-4065-a6bc-cf17db85f079 - - - - -] Image-volume cache disabled for host hostgroup@tripleo_iscsi. 2016-03-23 02:50:32.091 24631 INFO oslo_service.service [req-203a338c-84b1-4065-a6bc-cf17db85f079 - - - - -] Starting 1 workers 2016-03-23 02:50:32.151 24784 INFO cinder.service [-] Starting cinder-volume node (version 8.0.0) 2016-03-23 02:50:32.153 24784 INFO cinder.volume.manager [req-255f7ba3-32a9-43a4-b075-60620ba09b1f - - - - -] Starting volume driver LVMVolumeDriver (3.0.0) 2016-03-23 02:50:35.809 24784 INFO cinder.volume.manager [req-255f7ba3-32a9-43a4-b075-60620ba09b1f - - - - -] Service not found for updating replication_status. 2016-03-23 02:50:35.810 24784 INFO cinder.volume.manager [req-255f7ba3-32a9-43a4-b075-60620ba09b1f - - - - -] Driver initialization completed successfully. 2016-03-23 03:02:00.861 24784 INFO cinder.volume.flows.manager.create_volume [req-38929f0f-6be5-4413-8efb-83e77b8ef5d0 48bd2edb8cc7401e837bbae5f3b18a99 1f6231e896d346c5a15c377c8ba9c559 - - -] Volume a17948cf-14ef-4fd0-b93f-4fd8c3f4fcc4: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/adb3b78d-2141-4c80-b9ab-d1a5df3448a3', None), 'volume_size': 10, 'volume_name': 'volume-a17948cf-14ef-4fd0-b93f-4fd8c3f4fcc4', 'image_id': u'adb3b78d-2141-4c80-b9ab-d1a5df3448a3', 'image_service': , 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'pingtest_image', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2016, 3, 23, 3, 1, 26, tzinfo=), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2016, 3, 23, 3, 1, 34, tzinfo=), u'visibility': u'public', 'properties': {}, u'owner': u'1f6231e896d346c5a15c377c8ba9c559', u'protected': False, u'id': u'adb3b78d-2141-4c80-b9ab-d1a5df3448a3', u'file': u'/v2/images/adb3b78d-2141-4c80-b9ab-d1a5df3448a3/file', u'checksum': u'd009530079fd6567a3f0579a09c03af0', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/adb3b78d-2141-4c80-b9ab-d1a5df3448a3', u'min_ram': 0, u'size': 158443520}} 2016-03-23 03:02:29.149 24784 INFO cinder.image.image_utils [req-38929f0f-6be5-4413-8efb-83e77b8ef5d0 48bd2edb8cc7401e837bbae5f3b18a99 1f6231e896d346c5a15c377c8ba9c559 - - -] Image download 151.00 MB at 5.34 MB/s 2016-03-23 03:03:21.836 24784 INFO cinder.image.image_utils [req-38929f0f-6be5-4413-8efb-83e77b8ef5d0 48bd2edb8cc7401e837bbae5f3b18a99 1f6231e896d346c5a15c377c8ba9c559 - - -] Converted 3072.00 MB image at 76.87 MB/s 2016-03-23 03:03:24.739 24784 INFO cinder.volume.flows.manager.create_volume [req-38929f0f-6be5-4413-8efb-83e77b8ef5d0 48bd2edb8cc7401e837bbae5f3b18a99 1f6231e896d346c5a15c377c8ba9c559 - - -] Volume volume-a17948cf-14ef-4fd0-b93f-4fd8c3f4fcc4 (a17948cf-14ef-4fd0-b93f-4fd8c3f4fcc4): created successfully 2016-03-23 03:03:24.745 24784 INFO cinder.volume.manager [req-38929f0f-6be5-4413-8efb-83e77b8ef5d0 48bd2edb8cc7401e837bbae5f3b18a99 1f6231e896d346c5a15c377c8ba9c559 - - -] Created volume successfully. 2016-03-23 03:03:46.755 24784 INFO cinder.volume.targets.lio [req-ce060250-5eb3-4349-b7a1-45543b195069 48bd2edb8cc7401e837bbae5f3b18a99 1f6231e896d346c5a15c377c8ba9c559 - - -] Creating iscsi_target for volume: volume-a17948cf-14ef-4fd0-b93f-4fd8c3f4fcc4 2016-03-23 03:03:50.043 24784 ERROR cinder.volume.targets.lio [req-ce060250-5eb3-4349-b7a1-45543b195069 48bd2edb8cc7401e837bbae5f3b18a99 1f6231e896d346c5a15c377c8ba9c559 - - -] Failed to create iscsi target for volume id:volume-a17948cf-14ef-4fd0-b93f-4fd8c3f4fcc4. 2016-03-23 03:03:50.043 24784 ERROR cinder.volume.targets.lio Traceback (most recent call last): 2016-03-23 03:03:50.043 24784 ERROR cinder.volume.targets.lio File "/usr/lib/python2.7/site-packages/cinder/volume/targets/lio.py", line 127, in create_iscsi_target 2016-03-23 03:03:50.043 24784 ERROR cinder.volume.targets.lio self._execute(*command_args, run_as_root=True) 2016-03-23 03:03:50.043 24784 ERROR cinder.volume.targets.lio File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 271, in inner 2016-03-23 03:03:50.043 24784 ERROR cinder.volume.targets.lio return f(*args, **kwargs) 2016-03-23 03:03:50.043 24784 ERROR cinder.volume.targets.lio File "/usr/lib/python2.7/site-packages/cinder/volume/targets/lio.py", line 52, in _execute 2016-03-23 03:03:50.043 24784 ERROR cinder.volume.targets.lio return utils.execute(*args, **kwargs) 2016-03-23 03:03:50.043 24784 ERROR cinder.volume.targets.lio File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 147, in execute 2016-03-23 03:03:50.043 24784 ERROR cinder.volume.targets.lio return processutils.execute(*cmd, **kwargs) 2016-03-23 03:03:50.043 24784 ERROR cinder.volume.targets.lio File "/usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py", line 371, in execute 2016-03-23 03:03:50.043 24784 ERROR cinder.volume.targets.lio cmd=sanitized_cmd) 2016-03-23 03:03:50.043 24784 ERROR cinder.volume.targets.lio ProcessExecutionError: Unexpected error while running command. 2016-03-23 03:03:50.043 24784 ERROR cinder.volume.targets.lio Command: sudo cinder-rootwrap /etc/cinder/rootwrap.conf cinder-rtstool create /dev/cinder-volumes/volume-a17948cf-14ef-4fd0-b93f-4fd8c3f4fcc4 iqn.2010-10.org.openstack:volume-a17948cf-14ef-4fd0-b93f-4fd8c3f4fcc4 VCHGDqLucq9e223DxTop o2Gu7gysN3DNJnn9 False -p3260 -afd00:fd00:fd00:3000::11 2016-03-23 03:03:50.043 24784 ERROR cinder.volume.targets.lio Exit code: 1 2016-03-23 03:03:50.043 24784 ERROR cinder.volume.targets.lio Stdout: u'Error: creating NetworkPortal: ensure port 3260 on ip fd00:fd00:fd00:3000::11 is not in use by another service.\n' 2016-03-23 03:03:50.043 24784 ERROR cinder.volume.targets.lio Stderr: u'Traceback (most recent call last):\n File "/bin/cinder-rtstool", line 10, in \n sys.exit(main())\n File "/usr/lib/python2.7/site-packages/cinder/cmd/rtstool.py", line 286, in main\n **optional_args)\n File "/usr/lib/python2.7/site-packages/cinder/cmd/rtstool.py", line 91, in create\n mode=\'any\')\n File "/usr/lib/python2.7/site-packages/rtslib_fb/target.py", line 702, in __init__\n self._create_in_cfs_ine(mode)\n File "/usr/lib/python2.7/site-packages/rtslib_fb/node.py", line 67, in _create_in_cfs_ine\n % self.__class__.__name__)\nrtslib_fb.utils.RTSLibError: Could not create NetworkPortal in configFS\n' 2016-03-23 03:03:50.043 24784 ERROR cinder.volume.targets.lio 2016-03-23 03:03:50.070 24784 ERROR cinder.volume.manager [req-ce060250-5eb3-4349-b7a1-45543b195069 48bd2edb8cc7401e837bbae5f3b18a99 1f6231e896d346c5a15c377c8ba9c559 - - -] Create export for volume failed. 2016-03-23 03:03:50.070 24784 ERROR cinder.volume.manager Traceback (most recent call last): 2016-03-23 03:03:50.070 24784 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1440, in initialize_connection 2016-03-23 03:03:50.070 24784 ERROR cinder.volume.manager volume, connector) 2016-03-23 03:03:50.070 24784 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/lvm.py", line 760, in create_export 2016-03-23 03:03:50.070 24784 ERROR cinder.volume.manager volume_path) 2016-03-23 03:03:50.070 24784 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/targets/iscsi.py", line 210, in create_export 2016-03-23 03:03:50.070 24784 ERROR cinder.volume.manager **portals_config) 2016-03-23 03:03:50.070 24784 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/targets/lio.py", line 132, in create_iscsi_target 2016-03-23 03:03:50.070 24784 ERROR cinder.volume.manager raise exception.ISCSITargetCreateFailed(volume_id=vol_id) 2016-03-23 03:03:50.070 24784 ERROR cinder.volume.manager ISCSITargetCreateFailed: Failed to create iscsi target for volume volume-a17948cf-14ef-4fd0-b93f-4fd8c3f4fcc4. 2016-03-23 03:03:50.070 24784 ERROR cinder.volume.manager 2016-03-23 03:03:50.114 24784 ERROR oslo_messaging.rpc.dispatcher [req-ce060250-5eb3-4349-b7a1-45543b195069 48bd2edb8cc7401e837bbae5f3b18a99 1f6231e896d346c5a15c377c8ba9c559 - - -] Exception during message handling: Bad or unexpected response from the storage volume backend API: Create export for volume failed. 2016-03-23 03:03:50.114 24784 ERROR oslo_messaging.rpc.dispatcher Traceback (most recent call last): 2016-03-23 03:03:50.114 24784 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 138, in _dispatch_and_reply 2016-03-23 03:03:50.114 24784 ERROR oslo_messaging.rpc.dispatcher incoming.message)) 2016-03-23 03:03:50.114 24784 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _dispatch 2016-03-23 03:03:50.114 24784 ERROR oslo_messaging.rpc.dispatcher return self._do_dispatch(endpoint, method, ctxt, args) 2016-03-23 03:03:50.114 24784 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 127, in _do_dispatch 2016-03-23 03:03:50.114 24784 ERROR oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args) 2016-03-23 03:03:50.114 24784 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1444, in initialize_connection 2016-03-23 03:03:50.114 24784 ERROR oslo_messaging.rpc.dispatcher raise exception.VolumeBackendAPIException(data=err_msg) 2016-03-23 03:03:50.114 24784 ERROR oslo_messaging.rpc.dispatcher VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Create export for volume failed. 2016-03-23 03:03:50.114 24784 ERROR oslo_messaging.rpc.dispatcher 2016-03-23 03:03:50.551 24784 ERROR oslo_messaging._drivers.common [req-ce060250-5eb3-4349-b7a1-45543b195069 48bd2edb8cc7401e837bbae5f3b18a99 1f6231e896d346c5a15c377c8ba9c559 - - -] Returning exception Bad or unexpected response from the storage volume backend API: Create export for volume failed. to caller 2016-03-23 03:03:50.551 24784 ERROR oslo_messaging._drivers.common [req-ce060250-5eb3-4349-b7a1-45543b195069 48bd2edb8cc7401e837bbae5f3b18a99 1f6231e896d346c5a15c377c8ba9c559 - - -] ['Traceback (most recent call last):\n', ' File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 138, in _dispatch_and_reply\n incoming.message))\n', ' File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _dispatch\n return self._do_dispatch(endpoint, method, ctxt, args)\n', ' File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 127, in _do_dispatch\n result = func(ctxt, **new_args)\n', ' File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1444, in initialize_connection\n raise exception.VolumeBackendAPIException(data=err_msg)\n', 'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Create export for volume failed.\n'] 2016-03-23 03:03:52.267 24784 ERROR cinder.volume.manager [req-106e9881-b098-41ff-8fee-4768de58a14c 48bd2edb8cc7401e837bbae5f3b18a99 1f6231e896d346c5a15c377c8ba9c559 - - -] Terminate volume connection failed: 'NoneType' object has no attribute 'split' 2016-03-23 03:03:52.269 24784 ERROR oslo_messaging.rpc.dispatcher [req-106e9881-b098-41ff-8fee-4768de58a14c 48bd2edb8cc7401e837bbae5f3b18a99 1f6231e896d346c5a15c377c8ba9c559 - - -] Exception during message handling: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: 'NoneType' object has no attribute 'split' 2016-03-23 03:03:52.269 24784 ERROR oslo_messaging.rpc.dispatcher Traceback (most recent call last): 2016-03-23 03:03:52.269 24784 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 138, in _dispatch_and_reply 2016-03-23 03:03:52.269 24784 ERROR oslo_messaging.rpc.dispatcher incoming.message)) 2016-03-23 03:03:52.269 24784 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _dispatch 2016-03-23 03:03:52.269 24784 ERROR oslo_messaging.rpc.dispatcher return self._do_dispatch(endpoint, method, ctxt, args) 2016-03-23 03:03:52.269 24784 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 127, in _do_dispatch 2016-03-23 03:03:52.269 24784 ERROR oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args) 2016-03-23 03:03:52.269 24784 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1539, in terminate_connection 2016-03-23 03:03:52.269 24784 ERROR oslo_messaging.rpc.dispatcher raise exception.VolumeBackendAPIException(data=err_msg) 2016-03-23 03:03:52.269 24784 ERROR oslo_messaging.rpc.dispatcher VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: 'NoneType' object has no attribute 'split' 2016-03-23 03:03:52.269 24784 ERROR oslo_messaging.rpc.dispatcher 2016-03-23 03:03:52.272 24784 ERROR oslo_messaging._drivers.common [req-106e9881-b098-41ff-8fee-4768de58a14c 48bd2edb8cc7401e837bbae5f3b18a99 1f6231e896d346c5a15c377c8ba9c559 - - -] Returning exception Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: 'NoneType' object has no attribute 'split' to caller 2016-03-23 03:03:52.273 24784 ERROR oslo_messaging._drivers.common [req-106e9881-b098-41ff-8fee-4768de58a14c 48bd2edb8cc7401e837bbae5f3b18a99 1f6231e896d346c5a15c377c8ba9c559 - - -] ['Traceback (most recent call last):\n', ' File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 138, in _dispatch_and_reply\n incoming.message))\n', ' File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _dispatch\n return self._do_dispatch(endpoint, method, ctxt, args)\n', ' File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 127, in _do_dispatch\n result = func(ctxt, **new_args)\n', ' File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1539, in terminate_connection\n raise exception.VolumeBackendAPIException(data=err_msg)\n', "VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: 'NoneType' object has no attribute 'split'\n"] 2016-03-23 03:04:08.511 24784 INFO cinder.volume.targets.lio [req-0d064152-2067-4d4c-ae21-7c807dabc17d 48bd2edb8cc7401e837bbae5f3b18a99 1f6231e896d346c5a15c377c8ba9c559 - - -] Creating iscsi_target for volume: volume-a17948cf-14ef-4fd0-b93f-4fd8c3f4fcc4 2016-03-23 03:04:11.469 24784 INFO cinder.volume.manager [req-0d064152-2067-4d4c-ae21-7c807dabc17d 48bd2edb8cc7401e837bbae5f3b18a99 1f6231e896d346c5a15c377c8ba9c559 - - -] Initialize volume connection completed successfully. 2016-03-23 03:04:13.536 24784 INFO cinder.volume.manager [req-a2c010d9-f5fd-463c-badb-5677730b34d4 48bd2edb8cc7401e837bbae5f3b18a99 1f6231e896d346c5a15c377c8ba9c559 - - -] Attach volume completed successfully. 2016-03-23 03:06:19.623 24784 INFO cinder.volume.manager [req-b91473be-3516-4ea9-88ee-da76620c01db 48bd2edb8cc7401e837bbae5f3b18a99 1f6231e896d346c5a15c377c8ba9c559 - - -] Terminate volume connection completed successfully. 2016-03-23 03:06:21.604 24784 INFO cinder.volume.targets.lio [req-c578e9ed-564c-490f-947e-95cae5a4c57d 48bd2edb8cc7401e837bbae5f3b18a99 1f6231e896d346c5a15c377c8ba9c559 - - -] Removing iscsi_target: a17948cf-14ef-4fd0-b93f-4fd8c3f4fcc4 2016-03-23 03:06:22.831 24784 INFO cinder.volume.manager [req-c578e9ed-564c-490f-947e-95cae5a4c57d 48bd2edb8cc7401e837bbae5f3b18a99 1f6231e896d346c5a15c377c8ba9c559 - - -] Detach volume completed successfully. 2016-03-23 03:06:39.606 24784 INFO cinder.volume.targets.lio [req-0647c74c-4842-4adc-8813-5558786cd81d 48bd2edb8cc7401e837bbae5f3b18a99 1f6231e896d346c5a15c377c8ba9c559 - - -] Creating iscsi_target for volume: volume-a17948cf-14ef-4fd0-b93f-4fd8c3f4fcc4 2016-03-23 03:06:40.212 24784 ERROR cinder.volume.targets.lio [req-0647c74c-4842-4adc-8813-5558786cd81d 48bd2edb8cc7401e837bbae5f3b18a99 1f6231e896d346c5a15c377c8ba9c559 - - -] Failed to create iscsi target for volume id:volume-a17948cf-14ef-4fd0-b93f-4fd8c3f4fcc4. 2016-03-23 03:06:40.212 24784 ERROR cinder.volume.targets.lio Traceback (most recent call last): 2016-03-23 03:06:40.212 24784 ERROR cinder.volume.targets.lio File "/usr/lib/python2.7/site-packages/cinder/volume/targets/lio.py", line 127, in create_iscsi_target 2016-03-23 03:06:40.212 24784 ERROR cinder.volume.targets.lio self._execute(*command_args, run_as_root=True) 2016-03-23 03:06:40.212 24784 ERROR cinder.volume.targets.lio File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 271, in inner 2016-03-23 03:06:40.212 24784 ERROR cinder.volume.targets.lio return f(*args, **kwargs) 2016-03-23 03:06:40.212 24784 ERROR cinder.volume.targets.lio File "/usr/lib/python2.7/site-packages/cinder/volume/targets/lio.py", line 52, in _execute 2016-03-23 03:06:40.212 24784 ERROR cinder.volume.targets.lio return utils.execute(*args, **kwargs) 2016-03-23 03:06:40.212 24784 ERROR cinder.volume.targets.lio File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 147, in execute 2016-03-23 03:06:40.212 24784 ERROR cinder.volume.targets.lio return processutils.execute(*cmd, **kwargs) 2016-03-23 03:06:40.212 24784 ERROR cinder.volume.targets.lio File "/usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py", line 371, in execute 2016-03-23 03:06:40.212 24784 ERROR cinder.volume.targets.lio cmd=sanitized_cmd) 2016-03-23 03:06:40.212 24784 ERROR cinder.volume.targets.lio ProcessExecutionError: Unexpected error while running command. 2016-03-23 03:06:40.212 24784 ERROR cinder.volume.targets.lio Command: sudo cinder-rootwrap /etc/cinder/rootwrap.conf cinder-rtstool create /dev/cinder-volumes/volume-a17948cf-14ef-4fd0-b93f-4fd8c3f4fcc4 iqn.2010-10.org.openstack:volume-a17948cf-14ef-4fd0-b93f-4fd8c3f4fcc4 T6MMpVuvfUxafb6eKjyx 9uDFwL3nGG5Nedbf False -p3260 -afd00:fd00:fd00:3000::11 2016-03-23 03:06:40.212 24784 ERROR cinder.volume.targets.lio Exit code: 1 2016-03-23 03:06:40.212 24784 ERROR cinder.volume.targets.lio Stdout: u'Error: creating NetworkPortal: ensure port 3260 on ip fd00:fd00:fd00:3000::11 is not in use by another service.\n' 2016-03-23 03:06:40.212 24784 ERROR cinder.volume.targets.lio Stderr: u'Traceback (most recent call last):\n File "/bin/cinder-rtstool", line 10, in \n sys.exit(main())\n File "/usr/lib/python2.7/site-packages/cinder/cmd/rtstool.py", line 286, in main\n **optional_args)\n File "/usr/lib/python2.7/site-packages/cinder/cmd/rtstool.py", line 91, in create\n mode=\'any\')\n File "/usr/lib/python2.7/site-packages/rtslib_fb/target.py", line 702, in __init__\n self._create_in_cfs_ine(mode)\n File "/usr/lib/python2.7/site-packages/rtslib_fb/node.py", line 67, in _create_in_cfs_ine\n % self.__class__.__name__)\nrtslib_fb.utils.RTSLibError: Could not create NetworkPortal in configFS\n' 2016-03-23 03:06:40.212 24784 ERROR cinder.volume.targets.lio 2016-03-23 03:06:40.216 24784 ERROR cinder.volume.manager [req-0647c74c-4842-4adc-8813-5558786cd81d 48bd2edb8cc7401e837bbae5f3b18a99 1f6231e896d346c5a15c377c8ba9c559 - - -] Create export for volume failed. 2016-03-23 03:06:40.216 24784 ERROR cinder.volume.manager Traceback (most recent call last): 2016-03-23 03:06:40.216 24784 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1440, in initialize_connection 2016-03-23 03:06:40.216 24784 ERROR cinder.volume.manager volume, connector) 2016-03-23 03:06:40.216 24784 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/lvm.py", line 760, in create_export 2016-03-23 03:06:40.216 24784 ERROR cinder.volume.manager volume_path) 2016-03-23 03:06:40.216 24784 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/targets/iscsi.py", line 210, in create_export 2016-03-23 03:06:40.216 24784 ERROR cinder.volume.manager **portals_config) 2016-03-23 03:06:40.216 24784 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/targets/lio.py", line 132, in create_iscsi_target 2016-03-23 03:06:40.216 24784 ERROR cinder.volume.manager raise exception.ISCSITargetCreateFailed(volume_id=vol_id) 2016-03-23 03:06:40.216 24784 ERROR cinder.volume.manager ISCSITargetCreateFailed: Failed to create iscsi target for volume volume-a17948cf-14ef-4fd0-b93f-4fd8c3f4fcc4. 2016-03-23 03:06:40.216 24784 ERROR cinder.volume.manager 2016-03-23 03:06:40.218 24784 ERROR oslo_messaging.rpc.dispatcher [req-0647c74c-4842-4adc-8813-5558786cd81d 48bd2edb8cc7401e837bbae5f3b18a99 1f6231e896d346c5a15c377c8ba9c559 - - -] Exception during message handling: Bad or unexpected response from the storage volume backend API: Create export for volume failed. 2016-03-23 03:06:40.218 24784 ERROR oslo_messaging.rpc.dispatcher Traceback (most recent call last): 2016-03-23 03:06:40.218 24784 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 138, in _dispatch_and_reply 2016-03-23 03:06:40.218 24784 ERROR oslo_messaging.rpc.dispatcher incoming.message)) 2016-03-23 03:06:40.218 24784 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _dispatch 2016-03-23 03:06:40.218 24784 ERROR oslo_messaging.rpc.dispatcher return self._do_dispatch(endpoint, method, ctxt, args) 2016-03-23 03:06:40.218 24784 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 127, in _do_dispatch 2016-03-23 03:06:40.218 24784 ERROR oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args) 2016-03-23 03:06:40.218 24784 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1444, in initialize_connection 2016-03-23 03:06:40.218 24784 ERROR oslo_messaging.rpc.dispatcher raise exception.VolumeBackendAPIException(data=err_msg) 2016-03-23 03:06:40.218 24784 ERROR oslo_messaging.rpc.dispatcher VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Create export for volume failed. 2016-03-23 03:06:40.218 24784 ERROR oslo_messaging.rpc.dispatcher 2016-03-23 03:06:40.251 24784 ERROR oslo_messaging._drivers.common [req-0647c74c-4842-4adc-8813-5558786cd81d 48bd2edb8cc7401e837bbae5f3b18a99 1f6231e896d346c5a15c377c8ba9c559 - - -] Returning exception Bad or unexpected response from the storage volume backend API: Create export for volume failed. to caller 2016-03-23 03:06:40.252 24784 ERROR oslo_messaging._drivers.common [req-0647c74c-4842-4adc-8813-5558786cd81d 48bd2edb8cc7401e837bbae5f3b18a99 1f6231e896d346c5a15c377c8ba9c559 - - -] ['Traceback (most recent call last):\n', ' File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 138, in _dispatch_and_reply\n incoming.message))\n', ' File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _dispatch\n return self._do_dispatch(endpoint, method, ctxt, args)\n', ' File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 127, in _do_dispatch\n result = func(ctxt, **new_args)\n', ' File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1444, in initialize_connection\n raise exception.VolumeBackendAPIException(data=err_msg)\n', 'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Create export for volume failed.\n'] 2016-03-23 03:06:41.662 24784 INFO cinder.volume.manager [req-121d8b74-7246-4ec6-b64f-abd56c0116f4 48bd2edb8cc7401e837bbae5f3b18a99 1f6231e896d346c5a15c377c8ba9c559 - - -] Terminate volume connection completed successfully. 2016-03-23 03:07:02.839 24784 INFO cinder.volume.targets.lio [req-933a6302-27b7-45c8-ba52-d365622deffe 48bd2edb8cc7401e837bbae5f3b18a99 1f6231e896d346c5a15c377c8ba9c559 - - -] Creating iscsi_target for volume: volume-a17948cf-14ef-4fd0-b93f-4fd8c3f4fcc4 2016-03-23 03:07:06.974 24784 INFO cinder.volume.manager [req-933a6302-27b7-45c8-ba52-d365622deffe 48bd2edb8cc7401e837bbae5f3b18a99 1f6231e896d346c5a15c377c8ba9c559 - - -] Initialize volume connection completed successfully. 2016-03-23 03:07:09.312 24784 INFO cinder.volume.manager [req-44c0222c-b30d-4991-bce5-e4cc34b44785 48bd2edb8cc7401e837bbae5f3b18a99 1f6231e896d346c5a15c377c8ba9c559 - - -] Attach volume completed successfully. 2016-03-23 03:09:14.098 24784 INFO cinder.volume.manager [req-1fe34d40-93ac-40ec-938a-2a8a8e118ea8 48bd2edb8cc7401e837bbae5f3b18a99 1f6231e896d346c5a15c377c8ba9c559 - - -] Terminate volume connection completed successfully. 2016-03-23 03:09:15.978 24784 INFO cinder.volume.targets.lio [req-d6877765-b841-4314-ab0e-7cff442b6acd 48bd2edb8cc7401e837bbae5f3b18a99 1f6231e896d346c5a15c377c8ba9c559 - - -] Removing iscsi_target: a17948cf-14ef-4fd0-b93f-4fd8c3f4fcc4 2016-03-23 03:09:17.219 24784 INFO cinder.volume.manager [req-d6877765-b841-4314-ab0e-7cff442b6acd 48bd2edb8cc7401e837bbae5f3b18a99 1f6231e896d346c5a15c377c8ba9c559 - - -] Detach volume completed successfully. 2016-03-23 03:09:22.355 24784 ERROR cinder.volume.targets.lio [req-82ebe1a5-d2f1-46b0-bc87-d61e3e930bd1 48bd2edb8cc7401e837bbae5f3b18a99 1f6231e896d346c5a15c377c8ba9c559 - - -] Failed to delete initiator iqn iqn.1994-05.com.redhat:cf99835fe1b8 from target. 2016-03-23 03:09:22.355 24784 ERROR cinder.volume.targets.lio Traceback (most recent call last): 2016-03-23 03:09:22.355 24784 ERROR cinder.volume.targets.lio File "/usr/lib/python2.7/site-packages/cinder/volume/targets/lio.py", line 197, in terminate_connection 2016-03-23 03:09:22.355 24784 ERROR cinder.volume.targets.lio run_as_root=True) 2016-03-23 03:09:22.355 24784 ERROR cinder.volume.targets.lio File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 271, in inner 2016-03-23 03:09:22.355 24784 ERROR cinder.volume.targets.lio return f(*args, **kwargs) 2016-03-23 03:09:22.355 24784 ERROR cinder.volume.targets.lio File "/usr/lib/python2.7/site-packages/cinder/volume/targets/lio.py", line 52, in _execute 2016-03-23 03:09:22.355 24784 ERROR cinder.volume.targets.lio return utils.execute(*args, **kwargs) 2016-03-23 03:09:22.355 24784 ERROR cinder.volume.targets.lio File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 147, in execute 2016-03-23 03:09:22.355 24784 ERROR cinder.volume.targets.lio return processutils.execute(*cmd, **kwargs) 2016-03-23 03:09:22.355 24784 ERROR cinder.volume.targets.lio File "/usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py", line 371, in execute 2016-03-23 03:09:22.355 24784 ERROR cinder.volume.targets.lio cmd=sanitized_cmd) 2016-03-23 03:09:22.355 24784 ERROR cinder.volume.targets.lio ProcessExecutionError: Unexpected error while running command. 2016-03-23 03:09:22.355 24784 ERROR cinder.volume.targets.lio Command: sudo cinder-rootwrap /etc/cinder/rootwrap.conf cinder-rtstool delete-initiator iqn.2010-10.org.openstack:volume-a17948cf-14ef-4fd0-b93f-4fd8c3f4fcc4 iqn.1994-05.com.redhat:cf99835fe1b8 2016-03-23 03:09:22.355 24784 ERROR cinder.volume.targets.lio Exit code: 1 2016-03-23 03:09:22.355 24784 ERROR cinder.volume.targets.lio Stdout: u'' 2016-03-23 03:09:22.355 24784 ERROR cinder.volume.targets.lio Stderr: u'Traceback (most recent call last):\n File "/bin/cinder-rtstool", line 10, in \n sys.exit(main())\n File "/usr/lib/python2.7/site-packages/cinder/cmd/rtstool.py", line 306, in main\n delete_initiator(target_iqn, initiator_iqn)\n File "/usr/lib/python2.7/site-packages/cinder/cmd/rtstool.py", line 142, in delete_initiator\n target = _lookup_target(target_iqn, initiator_iqn)\n File "/usr/lib/python2.7/site-packages/cinder/cmd/rtstool.py", line 122, in _lookup_target\n raise RtstoolError(_(\'Could not find target %s\') % target_iqn)\ncinder.cmd.rtstool.RtstoolError: Could not find target iqn.2010-10.org.openstack:volume-a17948cf-14ef-4fd0-b93f-4fd8c3f4fcc4\n' 2016-03-23 03:09:22.355 24784 ERROR cinder.volume.targets.lio 2016-03-23 03:09:22.360 24784 ERROR cinder.volume.manager [req-82ebe1a5-d2f1-46b0-bc87-d61e3e930bd1 48bd2edb8cc7401e837bbae5f3b18a99 1f6231e896d346c5a15c377c8ba9c559 - - -] Terminate volume connection failed: Failed to detach iSCSI target for volume a17948cf-14ef-4fd0-b93f-4fd8c3f4fcc4. 2016-03-23 03:09:22.361 24784 ERROR oslo_messaging.rpc.dispatcher [req-82ebe1a5-d2f1-46b0-bc87-d61e3e930bd1 48bd2edb8cc7401e837bbae5f3b18a99 1f6231e896d346c5a15c377c8ba9c559 - - -] Exception during message handling: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: Failed to detach iSCSI target for volume a17948cf-14ef-4fd0-b93f-4fd8c3f4fcc4. 2016-03-23 03:09:22.361 24784 ERROR oslo_messaging.rpc.dispatcher Traceback (most recent call last): 2016-03-23 03:09:22.361 24784 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 138, in _dispatch_and_reply 2016-03-23 03:09:22.361 24784 ERROR oslo_messaging.rpc.dispatcher incoming.message)) 2016-03-23 03:09:22.361 24784 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _dispatch 2016-03-23 03:09:22.361 24784 ERROR oslo_messaging.rpc.dispatcher return self._do_dispatch(endpoint, method, ctxt, args) 2016-03-23 03:09:22.361 24784 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 127, in _do_dispatch 2016-03-23 03:09:22.361 24784 ERROR oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args) 2016-03-23 03:09:22.361 24784 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1539, in terminate_connection 2016-03-23 03:09:22.361 24784 ERROR oslo_messaging.rpc.dispatcher raise exception.VolumeBackendAPIException(data=err_msg) 2016-03-23 03:09:22.361 24784 ERROR oslo_messaging.rpc.dispatcher VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: Failed to detach iSCSI target for volume a17948cf-14ef-4fd0-b93f-4fd8c3f4fcc4. 2016-03-23 03:09:22.361 24784 ERROR oslo_messaging.rpc.dispatcher 2016-03-23 03:09:22.377 24784 ERROR oslo_messaging._drivers.common [req-82ebe1a5-d2f1-46b0-bc87-d61e3e930bd1 48bd2edb8cc7401e837bbae5f3b18a99 1f6231e896d346c5a15c377c8ba9c559 - - -] Returning exception Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: Failed to detach iSCSI target for volume a17948cf-14ef-4fd0-b93f-4fd8c3f4fcc4. to caller 2016-03-23 03:09:22.377 24784 ERROR oslo_messaging._drivers.common [req-82ebe1a5-d2f1-46b0-bc87-d61e3e930bd1 48bd2edb8cc7401e837bbae5f3b18a99 1f6231e896d346c5a15c377c8ba9c559 - - -] ['Traceback (most recent call last):\n', ' File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 138, in _dispatch_and_reply\n incoming.message))\n', ' File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _dispatch\n return self._do_dispatch(endpoint, method, ctxt, args)\n', ' File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 127, in _do_dispatch\n result = func(ctxt, **new_args)\n', ' File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1539, in terminate_connection\n raise exception.VolumeBackendAPIException(data=err_msg)\n', 'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: Failed to detach iSCSI target for volume a17948cf-14ef-4fd0-b93f-4fd8c3f4fcc4.\n']