Activity log for bug #1362985

Date Who What changed Old value New value Message
2014-08-29 08:45:31 Valeriy Ponomaryov bug added bug
2014-08-29 08:52:06 Valeriy Ponomaryov description One backend - generic driver. Bug case: 1) Service m-shr is down, service m-api is up. 2) m-api receives request to create share with share-network that "has not" server. 3) m-shr starts and outputs: 2014-08-29 04:37:47.405 ERROR oslo.messaging.rpc.dispatcher [req-2b7e770d-727b-40ba-a20d-088742788198 7fd80a12ed7e4f62ae59fc68f93d7963 8fec1ddd5dd54f339f50b9695335002d] Exception during message handling: 'GenericShareDriver' objec t has no attribute 'service_instance_manager' 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last): 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 134, in _dispatch_and_reply 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher incoming.message)) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 177, in _dispatch 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher return self._do_dispatch(endpoint, method, ctxt, args) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 123, in _do_dispatch 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher result = getattr(endpoint, method)(ctxt, **new_args) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 203, in create_share 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher {'status': 'error'}) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/openstack/common/excutils.py", line 82, in __exit__ 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 197, in create_share 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher context, share_network_id, share_id) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 154, in _provide_share_server_for_share 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher return _provide_share_server_for_share() 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/openstack/common/lockutils.py", line 325, in inner 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher return f(*args, **kwargs) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 147, in _provide_share_server_for_share 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher share_server = self._setup_server(context, share_server) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 423, in _setup_server 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher self.network_api.deallocate_network(context, share_network) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/openstack/common/excutils.py", line 82, in __exit__ 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 411, in _setup_server 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher metadata=metadata) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/drivers/generic.py", line 565, in setup_server 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher server = self.service_instance_manager.set_up_service_instance( 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher AttributeError: 'GenericShareDriver' object has no attribute 'service_instance_manager' 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher 2014-08-29 04:37:47.413 INFO oslo.messaging._drivers.impl_rabbit [req-2b7e770d-727b-40ba-a20d-088742788198 7fd80a12ed7e4f62ae59fc68f93d7963 8fec1ddd5dd54f339f50b9695335002d] Connecting to AMQP server on 172.18.198.52:5672 2014-08-29 04:37:47.422 INFO oslo.messaging._drivers.impl_rabbit [req-2b7e770d-727b-40ba-a20d-088742788198 7fd80a12ed7e4f62ae59fc68f93d7963 8fec1ddd5dd54f339f50b9695335002d] Connected to AMQP server on 172.18.198.52:5672 2014-08-29 04:37:47.423 ERROR oslo.messaging._drivers.common [req-2b7e770d-727b-40ba-a20d-088742788198 7fd80a12ed7e4f62ae59fc68f93d7963 8fec1ddd5dd54f339f50b9695335002d] Returning exception 'GenericShareDriver' object has no attribute 'service_instance_manager' to caller Another case: 1) Service m-shr is down, service m-api is up. 2) m-api receives request to create share with share-network that "has" server. 3) m-shr starts and outputs: 2014-08-29 04:42:13.379 ERROR manila.service [req-46e99303-ab4f-4e0d-82e4-8a85d66f326a None None] Unhandled exception 2014-08-29 04:42:13.379 TRACE manila.service Traceback (most recent call last): 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/service.py", line 225, in _start_child 2014-08-29 04:42:13.379 TRACE manila.service self._child_process(wrap.server) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/service.py", line 202, in _child_process 2014-08-29 04:42:13.379 TRACE manila.service launcher.run_server(server) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/service.py", line 93, in run_server 2014-08-29 04:42:13.379 TRACE manila.service server.start() 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/service.py", line 374, in start 2014-08-29 04:42:13.379 TRACE manila.service self.manager.init_host() 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/share/manager.py", line 85, in init_host 2014-08-29 04:42:13.379 TRACE manila.service share_server=share_server) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/share/drivers/generic.py", line 93, in wrap 2014-08-29 04:42:13.379 TRACE manila.service context, server['backend_details']): 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/share/drivers/service_instance.py", line 256, in ensure_service_instance 2014-08-29 04:42:13.379 TRACE manila.service server['instance_id']) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/compute/nova.py", line 144, in wrapper 2014-08-29 04:42:13.379 TRACE manila.service res = method(self, ctx, instance_id, *args, **kwargs) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/compute/nova.py", line 181, in server_get 2014-08-29 04:42:13.379 TRACE manila.service novaclient(context).servers.get(instance_id) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/python-novaclient/novaclient/v1_1/servers.py", line 554, in get 2014-08-29 04:42:13.379 TRACE manila.service return self._get("/servers/%s" % base.getid(server), "server") 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/python-novaclient/novaclient/base.py", line 93, in _get 2014-08-29 04:42:13.379 TRACE manila.service _resp, body = self.api.client.get(url) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/python-novaclient/novaclient/client.py", line 487, in get 2014-08-29 04:42:13.379 TRACE manila.service return self._cs_request(url, 'GET', **kwargs) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/python-novaclient/novaclient/client.py", line 479, in _cs_request 2014-08-29 04:42:13.379 TRACE manila.service raise e 2014-08-29 04:42:13.379 TRACE manila.service Unauthorized: Unauthorized (HTTP 401) (Request-ID: req-70a2d5b0-9078-4108-97ff-df811a37dc81) 2014-08-29 04:42:13.379 TRACE manila.service So, if share was scheduled with any share-network it won't be created in any case. One backend - generic driver. Bug case: 1) Service m-shr is down, service m-api is up. 2) m-api receives request to create share with share-network that "has not" server. 3) m-shr starts and outputs: 2014-08-29 04:37:47.405 ERROR oslo.messaging.rpc.dispatcher [req-2b7e770d-727b-40ba-a20d-088742788198 7fd80a12ed7e4f62ae59fc68f93d7963 8fec1ddd5dd54f339f50b9695335002d] Exception during message handling: 'GenericShareDriver' objec t has no attribute 'service_instance_manager' 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last): 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 134, in _dispatch_and_reply 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher incoming.message)) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 177, in _dispatch 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher return self._do_dispatch(endpoint, method, ctxt, args) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 123, in _do_dispatch 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher result = getattr(endpoint, method)(ctxt, **new_args) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 203, in create_share 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher {'status': 'error'}) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/openstack/common/excutils.py", line 82, in __exit__ 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 197, in create_share 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher context, share_network_id, share_id) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 154, in _provide_share_server_for_share 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher return _provide_share_server_for_share() 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/openstack/common/lockutils.py", line 325, in inner 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher return f(*args, **kwargs) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 147, in _provide_share_server_for_share 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher share_server = self._setup_server(context, share_server) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 423, in _setup_server 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher self.network_api.deallocate_network(context, share_network) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/openstack/common/excutils.py", line 82, in __exit__ 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 411, in _setup_server 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher metadata=metadata) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/drivers/generic.py", line 565, in setup_server 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher server = self.service_instance_manager.set_up_service_instance( 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher AttributeError: 'GenericShareDriver' object has no attribute 'service_instance_manager' 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher 2014-08-29 04:37:47.413 INFO oslo.messaging._drivers.impl_rabbit [req-2b7e770d-727b-40ba-a20d-088742788198 7fd80a12ed7e4f62ae59fc68f93d7963 8fec1ddd5dd54f339f50b9695335002d] Connecting to AMQP server on 172.18.198.52:5672 2014-08-29 04:37:47.422 INFO oslo.messaging._drivers.impl_rabbit [req-2b7e770d-727b-40ba-a20d-088742788198 7fd80a12ed7e4f62ae59fc68f93d7963 8fec1ddd5dd54f339f50b9695335002d] Connected to AMQP server on 172.18.198.52:5672 2014-08-29 04:37:47.423 ERROR oslo.messaging._drivers.common [req-2b7e770d-727b-40ba-a20d-088742788198 7fd80a12ed7e4f62ae59fc68f93d7963 8fec1ddd5dd54f339f50b9695335002d] Returning exception 'GenericShareDriver' object has no attribute 'service_instance_manager' to caller Another case: 1) Service m-shr is down, service m-api is up. 2) m-api receives request to create share with share-network that "has" server. 3) m-shr starts and outputs: 2014-08-29 04:42:13.379 ERROR manila.service [req-46e99303-ab4f-4e0d-82e4-8a85d66f326a None None] Unhandled exception 2014-08-29 04:42:13.379 TRACE manila.service Traceback (most recent call last): 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/service.py", line 225, in _start_child 2014-08-29 04:42:13.379 TRACE manila.service self._child_process(wrap.server) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/service.py", line 202, in _child_process 2014-08-29 04:42:13.379 TRACE manila.service launcher.run_server(server) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/service.py", line 93, in run_server 2014-08-29 04:42:13.379 TRACE manila.service server.start() 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/service.py", line 374, in start 2014-08-29 04:42:13.379 TRACE manila.service self.manager.init_host() 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/share/manager.py", line 85, in init_host 2014-08-29 04:42:13.379 TRACE manila.service share_server=share_server) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/share/drivers/generic.py", line 93, in wrap 2014-08-29 04:42:13.379 TRACE manila.service context, server['backend_details']): 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/share/drivers/service_instance.py", line 256, in ensure_service_instance 2014-08-29 04:42:13.379 TRACE manila.service server['instance_id']) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/compute/nova.py", line 144, in wrapper 2014-08-29 04:42:13.379 TRACE manila.service res = method(self, ctx, instance_id, *args, **kwargs) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/compute/nova.py", line 181, in server_get 2014-08-29 04:42:13.379 TRACE manila.service novaclient(context).servers.get(instance_id) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/python-novaclient/novaclient/v1_1/servers.py", line 554, in get 2014-08-29 04:42:13.379 TRACE manila.service return self._get("/servers/%s" % base.getid(server), "server") 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/python-novaclient/novaclient/base.py", line 93, in _get 2014-08-29 04:42:13.379 TRACE manila.service _resp, body = self.api.client.get(url) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/python-novaclient/novaclient/client.py", line 487, in get 2014-08-29 04:42:13.379 TRACE manila.service return self._cs_request(url, 'GET', **kwargs) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/python-novaclient/novaclient/client.py", line 479, in _cs_request 2014-08-29 04:42:13.379 TRACE manila.service raise e 2014-08-29 04:42:13.379 TRACE manila.service Unauthorized: Unauthorized (HTTP 401) (Request-ID: req-70a2d5b0-9078-4108-97ff-df811a37dc81) 2014-08-29 04:42:13.379 TRACE manila.service So, if share was scheduled with any share-network it won't be created in any case.
2014-08-29 08:53:47 Valeriy Ponomaryov description One backend - generic driver. Bug case: 1) Service m-shr is down, service m-api is up. 2) m-api receives request to create share with share-network that "has not" server. 3) m-shr starts and outputs: 2014-08-29 04:37:47.405 ERROR oslo.messaging.rpc.dispatcher [req-2b7e770d-727b-40ba-a20d-088742788198 7fd80a12ed7e4f62ae59fc68f93d7963 8fec1ddd5dd54f339f50b9695335002d] Exception during message handling: 'GenericShareDriver' objec t has no attribute 'service_instance_manager' 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last): 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 134, in _dispatch_and_reply 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher incoming.message)) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 177, in _dispatch 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher return self._do_dispatch(endpoint, method, ctxt, args) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 123, in _do_dispatch 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher result = getattr(endpoint, method)(ctxt, **new_args) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 203, in create_share 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher {'status': 'error'}) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/openstack/common/excutils.py", line 82, in __exit__ 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 197, in create_share 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher context, share_network_id, share_id) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 154, in _provide_share_server_for_share 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher return _provide_share_server_for_share() 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/openstack/common/lockutils.py", line 325, in inner 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher return f(*args, **kwargs) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 147, in _provide_share_server_for_share 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher share_server = self._setup_server(context, share_server) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 423, in _setup_server 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher self.network_api.deallocate_network(context, share_network) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/openstack/common/excutils.py", line 82, in __exit__ 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 411, in _setup_server 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher metadata=metadata) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/drivers/generic.py", line 565, in setup_server 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher server = self.service_instance_manager.set_up_service_instance( 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher AttributeError: 'GenericShareDriver' object has no attribute 'service_instance_manager' 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher 2014-08-29 04:37:47.413 INFO oslo.messaging._drivers.impl_rabbit [req-2b7e770d-727b-40ba-a20d-088742788198 7fd80a12ed7e4f62ae59fc68f93d7963 8fec1ddd5dd54f339f50b9695335002d] Connecting to AMQP server on 172.18.198.52:5672 2014-08-29 04:37:47.422 INFO oslo.messaging._drivers.impl_rabbit [req-2b7e770d-727b-40ba-a20d-088742788198 7fd80a12ed7e4f62ae59fc68f93d7963 8fec1ddd5dd54f339f50b9695335002d] Connected to AMQP server on 172.18.198.52:5672 2014-08-29 04:37:47.423 ERROR oslo.messaging._drivers.common [req-2b7e770d-727b-40ba-a20d-088742788198 7fd80a12ed7e4f62ae59fc68f93d7963 8fec1ddd5dd54f339f50b9695335002d] Returning exception 'GenericShareDriver' object has no attribute 'service_instance_manager' to caller Another case: 1) Service m-shr is down, service m-api is up. 2) m-api receives request to create share with share-network that "has" server. 3) m-shr starts and outputs: 2014-08-29 04:42:13.379 ERROR manila.service [req-46e99303-ab4f-4e0d-82e4-8a85d66f326a None None] Unhandled exception 2014-08-29 04:42:13.379 TRACE manila.service Traceback (most recent call last): 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/service.py", line 225, in _start_child 2014-08-29 04:42:13.379 TRACE manila.service self._child_process(wrap.server) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/service.py", line 202, in _child_process 2014-08-29 04:42:13.379 TRACE manila.service launcher.run_server(server) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/service.py", line 93, in run_server 2014-08-29 04:42:13.379 TRACE manila.service server.start() 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/service.py", line 374, in start 2014-08-29 04:42:13.379 TRACE manila.service self.manager.init_host() 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/share/manager.py", line 85, in init_host 2014-08-29 04:42:13.379 TRACE manila.service share_server=share_server) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/share/drivers/generic.py", line 93, in wrap 2014-08-29 04:42:13.379 TRACE manila.service context, server['backend_details']): 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/share/drivers/service_instance.py", line 256, in ensure_service_instance 2014-08-29 04:42:13.379 TRACE manila.service server['instance_id']) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/compute/nova.py", line 144, in wrapper 2014-08-29 04:42:13.379 TRACE manila.service res = method(self, ctx, instance_id, *args, **kwargs) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/compute/nova.py", line 181, in server_get 2014-08-29 04:42:13.379 TRACE manila.service novaclient(context).servers.get(instance_id) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/python-novaclient/novaclient/v1_1/servers.py", line 554, in get 2014-08-29 04:42:13.379 TRACE manila.service return self._get("/servers/%s" % base.getid(server), "server") 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/python-novaclient/novaclient/base.py", line 93, in _get 2014-08-29 04:42:13.379 TRACE manila.service _resp, body = self.api.client.get(url) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/python-novaclient/novaclient/client.py", line 487, in get 2014-08-29 04:42:13.379 TRACE manila.service return self._cs_request(url, 'GET', **kwargs) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/python-novaclient/novaclient/client.py", line 479, in _cs_request 2014-08-29 04:42:13.379 TRACE manila.service raise e 2014-08-29 04:42:13.379 TRACE manila.service Unauthorized: Unauthorized (HTTP 401) (Request-ID: req-70a2d5b0-9078-4108-97ff-df811a37dc81) 2014-08-29 04:42:13.379 TRACE manila.service So, if share was scheduled with any share-network it won't be created in any case. One backend - generic driver. Bug case: 1) Service m-shr is down, service m-api is up. 2) m-api receives request to create share with share-network that "has not" server. 3) m-shr starts and outputs: 2014-08-29 04:37:47.405 ERROR oslo.messaging.rpc.dispatcher [req-2b7e770d-727b-40ba-a20d-088742788198 7fd80a12ed7e4f62ae59fc68f93d7963 8fec1ddd5dd54f339f50b9695335002d] Exception during message handling: 'GenericShareDriver' objec t has no attribute 'service_instance_manager' 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last): 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 134, in _dispatch_and_reply 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher incoming.message)) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 177, in _dispatch 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher return self._do_dispatch(endpoint, method, ctxt, args) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 123, in _do_dispatch 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher result = getattr(endpoint, method)(ctxt, **new_args) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 203, in create_share 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher {'status': 'error'}) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/openstack/common/excutils.py", line 82, in __exit__ 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 197, in create_share 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher context, share_network_id, share_id) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 154, in _provide_share_server_for_share 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher return _provide_share_server_for_share() 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/openstack/common/lockutils.py", line 325, in inner 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher return f(*args, **kwargs) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 147, in _provide_share_server_for_share 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher share_server = self._setup_server(context, share_server) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 423, in _setup_server 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher self.network_api.deallocate_network(context, share_network) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/openstack/common/excutils.py", line 82, in __exit__ 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 411, in _setup_server 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher metadata=metadata) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/drivers/generic.py", line 565, in setup_server 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher server = self.service_instance_manager.set_up_service_instance( 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher AttributeError: 'GenericShareDriver' object has no attribute 'service_instance_manager' 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher 2014-08-29 04:37:47.413 INFO oslo.messaging._drivers.impl_rabbit [req-2b7e770d-727b-40ba-a20d-088742788198 7fd80a12ed7e4f62ae59fc68f93d7963 8fec1ddd5dd54f339f50b9695335002d] Connecting to AMQP server on 172.18.198.52:5672 2014-08-29 04:37:47.422 INFO oslo.messaging._drivers.impl_rabbit [req-2b7e770d-727b-40ba-a20d-088742788198 7fd80a12ed7e4f62ae59fc68f93d7963 8fec1ddd5dd54f339f50b9695335002d] Connected to AMQP server on 172.18.198.52:5672 2014-08-29 04:37:47.423 ERROR oslo.messaging._drivers.common [req-2b7e770d-727b-40ba-a20d-088742788198 7fd80a12ed7e4f62ae59fc68f93d7963 8fec1ddd5dd54f339f50b9695335002d] Returning exception 'GenericShareDriver' object has no attribute 'service_instance_manager' to caller Another case: 1) Service m-shr is down, service m-api is up. 2) m-api receives request to create share with share-network that "has" server. 3) m-shr starts and outputs: 2014-08-29 04:42:13.379 ERROR manila.service [req-46e99303-ab4f-4e0d-82e4-8a85d66f326a None None] Unhandled exception 2014-08-29 04:42:13.379 TRACE manila.service Traceback (most recent call last): 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/service.py", line 225, in _start_child 2014-08-29 04:42:13.379 TRACE manila.service self._child_process(wrap.server) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/service.py", line 202, in _child_process 2014-08-29 04:42:13.379 TRACE manila.service launcher.run_server(server) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/service.py", line 93, in run_server 2014-08-29 04:42:13.379 TRACE manila.service server.start() 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/service.py", line 374, in start 2014-08-29 04:42:13.379 TRACE manila.service self.manager.init_host() 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/share/manager.py", line 85, in init_host 2014-08-29 04:42:13.379 TRACE manila.service share_server=share_server) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/share/drivers/generic.py", line 93, in wrap 2014-08-29 04:42:13.379 TRACE manila.service context, server['backend_details']): 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/share/drivers/service_instance.py", line 256, in ensure_service_instance 2014-08-29 04:42:13.379 TRACE manila.service server['instance_id']) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/compute/nova.py", line 144, in wrapper 2014-08-29 04:42:13.379 TRACE manila.service res = method(self, ctx, instance_id, *args, **kwargs) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/compute/nova.py", line 181, in server_get 2014-08-29 04:42:13.379 TRACE manila.service novaclient(context).servers.get(instance_id) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/python-novaclient/novaclient/v1_1/servers.py", line 554, in get 2014-08-29 04:42:13.379 TRACE manila.service return self._get("/servers/%s" % base.getid(server), "server") 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/python-novaclient/novaclient/base.py", line 93, in _get 2014-08-29 04:42:13.379 TRACE manila.service _resp, body = self.api.client.get(url) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/python-novaclient/novaclient/client.py", line 487, in get 2014-08-29 04:42:13.379 TRACE manila.service return self._cs_request(url, 'GET', **kwargs) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/python-novaclient/novaclient/client.py", line 479, in _cs_request 2014-08-29 04:42:13.379 TRACE manila.service raise e 2014-08-29 04:42:13.379 TRACE manila.service Unauthorized: Unauthorized (HTTP 401) (Request-ID: req-70a2d5b0-9078-4108-97ff-df811a37dc81) 2014-08-29 04:42:13.379 TRACE manila.service So, if share was scheduled with any share-network it won't be created in any case. Summary: if there are some problems with existing shares, its access rules or share servers, then "init_host" of Manila fails and does not work.
2014-08-29 09:05:06 Valeriy Ponomaryov description One backend - generic driver. Bug case: 1) Service m-shr is down, service m-api is up. 2) m-api receives request to create share with share-network that "has not" server. 3) m-shr starts and outputs: 2014-08-29 04:37:47.405 ERROR oslo.messaging.rpc.dispatcher [req-2b7e770d-727b-40ba-a20d-088742788198 7fd80a12ed7e4f62ae59fc68f93d7963 8fec1ddd5dd54f339f50b9695335002d] Exception during message handling: 'GenericShareDriver' objec t has no attribute 'service_instance_manager' 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last): 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 134, in _dispatch_and_reply 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher incoming.message)) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 177, in _dispatch 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher return self._do_dispatch(endpoint, method, ctxt, args) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 123, in _do_dispatch 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher result = getattr(endpoint, method)(ctxt, **new_args) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 203, in create_share 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher {'status': 'error'}) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/openstack/common/excutils.py", line 82, in __exit__ 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 197, in create_share 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher context, share_network_id, share_id) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 154, in _provide_share_server_for_share 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher return _provide_share_server_for_share() 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/openstack/common/lockutils.py", line 325, in inner 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher return f(*args, **kwargs) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 147, in _provide_share_server_for_share 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher share_server = self._setup_server(context, share_server) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 423, in _setup_server 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher self.network_api.deallocate_network(context, share_network) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/openstack/common/excutils.py", line 82, in __exit__ 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 411, in _setup_server 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher metadata=metadata) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/drivers/generic.py", line 565, in setup_server 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher server = self.service_instance_manager.set_up_service_instance( 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher AttributeError: 'GenericShareDriver' object has no attribute 'service_instance_manager' 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher 2014-08-29 04:37:47.413 INFO oslo.messaging._drivers.impl_rabbit [req-2b7e770d-727b-40ba-a20d-088742788198 7fd80a12ed7e4f62ae59fc68f93d7963 8fec1ddd5dd54f339f50b9695335002d] Connecting to AMQP server on 172.18.198.52:5672 2014-08-29 04:37:47.422 INFO oslo.messaging._drivers.impl_rabbit [req-2b7e770d-727b-40ba-a20d-088742788198 7fd80a12ed7e4f62ae59fc68f93d7963 8fec1ddd5dd54f339f50b9695335002d] Connected to AMQP server on 172.18.198.52:5672 2014-08-29 04:37:47.423 ERROR oslo.messaging._drivers.common [req-2b7e770d-727b-40ba-a20d-088742788198 7fd80a12ed7e4f62ae59fc68f93d7963 8fec1ddd5dd54f339f50b9695335002d] Returning exception 'GenericShareDriver' object has no attribute 'service_instance_manager' to caller Another case: 1) Service m-shr is down, service m-api is up. 2) m-api receives request to create share with share-network that "has" server. 3) m-shr starts and outputs: 2014-08-29 04:42:13.379 ERROR manila.service [req-46e99303-ab4f-4e0d-82e4-8a85d66f326a None None] Unhandled exception 2014-08-29 04:42:13.379 TRACE manila.service Traceback (most recent call last): 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/service.py", line 225, in _start_child 2014-08-29 04:42:13.379 TRACE manila.service self._child_process(wrap.server) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/service.py", line 202, in _child_process 2014-08-29 04:42:13.379 TRACE manila.service launcher.run_server(server) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/service.py", line 93, in run_server 2014-08-29 04:42:13.379 TRACE manila.service server.start() 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/service.py", line 374, in start 2014-08-29 04:42:13.379 TRACE manila.service self.manager.init_host() 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/share/manager.py", line 85, in init_host 2014-08-29 04:42:13.379 TRACE manila.service share_server=share_server) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/share/drivers/generic.py", line 93, in wrap 2014-08-29 04:42:13.379 TRACE manila.service context, server['backend_details']): 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/share/drivers/service_instance.py", line 256, in ensure_service_instance 2014-08-29 04:42:13.379 TRACE manila.service server['instance_id']) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/compute/nova.py", line 144, in wrapper 2014-08-29 04:42:13.379 TRACE manila.service res = method(self, ctx, instance_id, *args, **kwargs) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/compute/nova.py", line 181, in server_get 2014-08-29 04:42:13.379 TRACE manila.service novaclient(context).servers.get(instance_id) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/python-novaclient/novaclient/v1_1/servers.py", line 554, in get 2014-08-29 04:42:13.379 TRACE manila.service return self._get("/servers/%s" % base.getid(server), "server") 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/python-novaclient/novaclient/base.py", line 93, in _get 2014-08-29 04:42:13.379 TRACE manila.service _resp, body = self.api.client.get(url) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/python-novaclient/novaclient/client.py", line 487, in get 2014-08-29 04:42:13.379 TRACE manila.service return self._cs_request(url, 'GET', **kwargs) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/python-novaclient/novaclient/client.py", line 479, in _cs_request 2014-08-29 04:42:13.379 TRACE manila.service raise e 2014-08-29 04:42:13.379 TRACE manila.service Unauthorized: Unauthorized (HTTP 401) (Request-ID: req-70a2d5b0-9078-4108-97ff-df811a37dc81) 2014-08-29 04:42:13.379 TRACE manila.service So, if share was scheduled with any share-network it won't be created in any case. Summary: if there are some problems with existing shares, its access rules or share servers, then "init_host" of Manila fails and does not work. One backend - generic driver. Bug case: 1) Service m-shr is down, service m-api is up. 2) m-api receives request to create share with share-network that "has not" server. 3) m-shr starts and outputs: 2014-08-29 04:37:47.405 ERROR oslo.messaging.rpc.dispatcher [req-2b7e770d-727b-40ba-a20d-088742788198 7fd80a12ed7e4f62ae59fc68f93d7963 8fec1ddd5dd54f339f50b9695335002d] Exception during message handling: 'GenericShareDriver' objec t has no attribute 'service_instance_manager' 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last): 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 134, in _dispatch_and_reply 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher incoming.message)) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 177, in _dispatch 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher return self._do_dispatch(endpoint, method, ctxt, args) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 123, in _do_dispatch 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher result = getattr(endpoint, method)(ctxt, **new_args) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 203, in create_share 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher {'status': 'error'}) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/openstack/common/excutils.py", line 82, in __exit__ 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 197, in create_share 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher context, share_network_id, share_id) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 154, in _provide_share_server_for_share 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher return _provide_share_server_for_share() 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/openstack/common/lockutils.py", line 325, in inner 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher return f(*args, **kwargs) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 147, in _provide_share_server_for_share 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher share_server = self._setup_server(context, share_server) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 423, in _setup_server 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher self.network_api.deallocate_network(context, share_network) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/openstack/common/excutils.py", line 82, in __exit__ 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 411, in _setup_server 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher metadata=metadata) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/drivers/generic.py", line 565, in setup_server 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher server = self.service_instance_manager.set_up_service_instance( 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher AttributeError: 'GenericShareDriver' object has no attribute 'service_instance_manager' 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher 2014-08-29 04:37:47.413 INFO oslo.messaging._drivers.impl_rabbit [req-2b7e770d-727b-40ba-a20d-088742788198 7fd80a12ed7e4f62ae59fc68f93d7963 8fec1ddd5dd54f339f50b9695335002d] Connecting to AMQP server on 172.18.198.52:5672 2014-08-29 04:37:47.422 INFO oslo.messaging._drivers.impl_rabbit [req-2b7e770d-727b-40ba-a20d-088742788198 7fd80a12ed7e4f62ae59fc68f93d7963 8fec1ddd5dd54f339f50b9695335002d] Connected to AMQP server on 172.18.198.52:5672 2014-08-29 04:37:47.423 ERROR oslo.messaging._drivers.common [req-2b7e770d-727b-40ba-a20d-088742788198 7fd80a12ed7e4f62ae59fc68f93d7963 8fec1ddd5dd54f339f50b9695335002d] Returning exception 'GenericShareDriver' object has no attribute 'service_instance_manager' to caller Looks like "setup_server" is called before "do_setup" method, that sets up attr "service_instance_manager". Another case: 1) Service m-shr is down, service m-api is up. 2) m-api receives request to create share with share-network that "has" server. 3) m-shr starts and outputs: 2014-08-29 04:42:13.379 ERROR manila.service [req-46e99303-ab4f-4e0d-82e4-8a85d66f326a None None] Unhandled exception 2014-08-29 04:42:13.379 TRACE manila.service Traceback (most recent call last): 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/service.py", line 225, in _start_child 2014-08-29 04:42:13.379 TRACE manila.service self._child_process(wrap.server) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/service.py", line 202, in _child_process 2014-08-29 04:42:13.379 TRACE manila.service launcher.run_server(server) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/service.py", line 93, in run_server 2014-08-29 04:42:13.379 TRACE manila.service server.start() 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/service.py", line 374, in start 2014-08-29 04:42:13.379 TRACE manila.service self.manager.init_host() 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/share/manager.py", line 85, in init_host 2014-08-29 04:42:13.379 TRACE manila.service share_server=share_server) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/share/drivers/generic.py", line 93, in wrap 2014-08-29 04:42:13.379 TRACE manila.service context, server['backend_details']): 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/share/drivers/service_instance.py", line 256, in ensure_service_instance 2014-08-29 04:42:13.379 TRACE manila.service server['instance_id']) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/compute/nova.py", line 144, in wrapper 2014-08-29 04:42:13.379 TRACE manila.service res = method(self, ctx, instance_id, *args, **kwargs) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/compute/nova.py", line 181, in server_get 2014-08-29 04:42:13.379 TRACE manila.service novaclient(context).servers.get(instance_id) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/python-novaclient/novaclient/v1_1/servers.py", line 554, in get 2014-08-29 04:42:13.379 TRACE manila.service return self._get("/servers/%s" % base.getid(server), "server") 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/python-novaclient/novaclient/base.py", line 93, in _get 2014-08-29 04:42:13.379 TRACE manila.service _resp, body = self.api.client.get(url) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/python-novaclient/novaclient/client.py", line 487, in get 2014-08-29 04:42:13.379 TRACE manila.service return self._cs_request(url, 'GET', **kwargs) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/python-novaclient/novaclient/client.py", line 479, in _cs_request 2014-08-29 04:42:13.379 TRACE manila.service raise e 2014-08-29 04:42:13.379 TRACE manila.service Unauthorized: Unauthorized (HTTP 401) (Request-ID: req-70a2d5b0-9078-4108-97ff-df811a37dc81) 2014-08-29 04:42:13.379 TRACE manila.service So, if share was scheduled with any share-network it won't be created in any case. Summary: if there are some problems with existing shares, its access rules or share servers, then "init_host" of Manila fails and does not work.
2014-08-29 09:09:53 Valeriy Ponomaryov description One backend - generic driver. Bug case: 1) Service m-shr is down, service m-api is up. 2) m-api receives request to create share with share-network that "has not" server. 3) m-shr starts and outputs: 2014-08-29 04:37:47.405 ERROR oslo.messaging.rpc.dispatcher [req-2b7e770d-727b-40ba-a20d-088742788198 7fd80a12ed7e4f62ae59fc68f93d7963 8fec1ddd5dd54f339f50b9695335002d] Exception during message handling: 'GenericShareDriver' objec t has no attribute 'service_instance_manager' 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last): 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 134, in _dispatch_and_reply 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher incoming.message)) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 177, in _dispatch 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher return self._do_dispatch(endpoint, method, ctxt, args) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 123, in _do_dispatch 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher result = getattr(endpoint, method)(ctxt, **new_args) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 203, in create_share 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher {'status': 'error'}) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/openstack/common/excutils.py", line 82, in __exit__ 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 197, in create_share 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher context, share_network_id, share_id) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 154, in _provide_share_server_for_share 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher return _provide_share_server_for_share() 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/openstack/common/lockutils.py", line 325, in inner 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher return f(*args, **kwargs) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 147, in _provide_share_server_for_share 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher share_server = self._setup_server(context, share_server) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 423, in _setup_server 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher self.network_api.deallocate_network(context, share_network) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/openstack/common/excutils.py", line 82, in __exit__ 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 411, in _setup_server 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher metadata=metadata) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/drivers/generic.py", line 565, in setup_server 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher server = self.service_instance_manager.set_up_service_instance( 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher AttributeError: 'GenericShareDriver' object has no attribute 'service_instance_manager' 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher 2014-08-29 04:37:47.413 INFO oslo.messaging._drivers.impl_rabbit [req-2b7e770d-727b-40ba-a20d-088742788198 7fd80a12ed7e4f62ae59fc68f93d7963 8fec1ddd5dd54f339f50b9695335002d] Connecting to AMQP server on 172.18.198.52:5672 2014-08-29 04:37:47.422 INFO oslo.messaging._drivers.impl_rabbit [req-2b7e770d-727b-40ba-a20d-088742788198 7fd80a12ed7e4f62ae59fc68f93d7963 8fec1ddd5dd54f339f50b9695335002d] Connected to AMQP server on 172.18.198.52:5672 2014-08-29 04:37:47.423 ERROR oslo.messaging._drivers.common [req-2b7e770d-727b-40ba-a20d-088742788198 7fd80a12ed7e4f62ae59fc68f93d7963 8fec1ddd5dd54f339f50b9695335002d] Returning exception 'GenericShareDriver' object has no attribute 'service_instance_manager' to caller Looks like "setup_server" is called before "do_setup" method, that sets up attr "service_instance_manager". Another case: 1) Service m-shr is down, service m-api is up. 2) m-api receives request to create share with share-network that "has" server. 3) m-shr starts and outputs: 2014-08-29 04:42:13.379 ERROR manila.service [req-46e99303-ab4f-4e0d-82e4-8a85d66f326a None None] Unhandled exception 2014-08-29 04:42:13.379 TRACE manila.service Traceback (most recent call last): 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/service.py", line 225, in _start_child 2014-08-29 04:42:13.379 TRACE manila.service self._child_process(wrap.server) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/service.py", line 202, in _child_process 2014-08-29 04:42:13.379 TRACE manila.service launcher.run_server(server) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/service.py", line 93, in run_server 2014-08-29 04:42:13.379 TRACE manila.service server.start() 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/service.py", line 374, in start 2014-08-29 04:42:13.379 TRACE manila.service self.manager.init_host() 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/share/manager.py", line 85, in init_host 2014-08-29 04:42:13.379 TRACE manila.service share_server=share_server) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/share/drivers/generic.py", line 93, in wrap 2014-08-29 04:42:13.379 TRACE manila.service context, server['backend_details']): 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/share/drivers/service_instance.py", line 256, in ensure_service_instance 2014-08-29 04:42:13.379 TRACE manila.service server['instance_id']) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/compute/nova.py", line 144, in wrapper 2014-08-29 04:42:13.379 TRACE manila.service res = method(self, ctx, instance_id, *args, **kwargs) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/manila/manila/compute/nova.py", line 181, in server_get 2014-08-29 04:42:13.379 TRACE manila.service novaclient(context).servers.get(instance_id) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/python-novaclient/novaclient/v1_1/servers.py", line 554, in get 2014-08-29 04:42:13.379 TRACE manila.service return self._get("/servers/%s" % base.getid(server), "server") 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/python-novaclient/novaclient/base.py", line 93, in _get 2014-08-29 04:42:13.379 TRACE manila.service _resp, body = self.api.client.get(url) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/python-novaclient/novaclient/client.py", line 487, in get 2014-08-29 04:42:13.379 TRACE manila.service return self._cs_request(url, 'GET', **kwargs) 2014-08-29 04:42:13.379 TRACE manila.service File "/opt/stack/python-novaclient/novaclient/client.py", line 479, in _cs_request 2014-08-29 04:42:13.379 TRACE manila.service raise e 2014-08-29 04:42:13.379 TRACE manila.service Unauthorized: Unauthorized (HTTP 401) (Request-ID: req-70a2d5b0-9078-4108-97ff-df811a37dc81) 2014-08-29 04:42:13.379 TRACE manila.service So, if share was scheduled with any share-network it won't be created in any case. Summary: if there are some problems with existing shares, its access rules or share servers, then "init_host" of Manila fails and does not work. One backend - generic driver. Bug case: 1) Service m-shr is down, service m-api is up. 2) m-api receives request to create share with share-network that "has not" server. 3) m-shr starts and outputs: 2014-08-29 04:37:47.405 ERROR oslo.messaging.rpc.dispatcher [req-2b7e770d-727b-40ba-a20d-088742788198 7fd80a12ed7e4f62ae59fc68f93d7963 8fec1ddd5dd54f339f50b9695335002d] Exception during message handling: 'GenericShareDriver' objec t has no attribute 'service_instance_manager' 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last): 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 134, in _dispatch_and_reply 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher incoming.message)) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 177, in _dispatch 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher return self._do_dispatch(endpoint, method, ctxt, args) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 123, in _do_dispatch 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher result = getattr(endpoint, method)(ctxt, **new_args) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 203, in create_share 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher {'status': 'error'}) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/openstack/common/excutils.py", line 82, in __exit__ 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 197, in create_share 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher context, share_network_id, share_id) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 154, in _provide_share_server_for_share 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher return _provide_share_server_for_share() 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/openstack/common/lockutils.py", line 325, in inner 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher return f(*args, **kwargs) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 147, in _provide_share_server_for_share 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher share_server = self._setup_server(context, share_server) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 423, in _setup_server 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher self.network_api.deallocate_network(context, share_network) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/openstack/common/excutils.py", line 82, in __exit__ 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 411, in _setup_server 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher metadata=metadata) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/drivers/generic.py", line 565, in setup_server 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher server = self.service_instance_manager.set_up_service_instance( 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher AttributeError: 'GenericShareDriver' object has no attribute 'service_instance_manager' 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher 2014-08-29 04:37:47.413 INFO oslo.messaging._drivers.impl_rabbit [req-2b7e770d-727b-40ba-a20d-088742788198 7fd80a12ed7e4f62ae59fc68f93d7963 8fec1ddd5dd54f339f50b9695335002d] Connecting to AMQP server on 172.18.198.52:5672 2014-08-29 04:37:47.422 INFO oslo.messaging._drivers.impl_rabbit [req-2b7e770d-727b-40ba-a20d-088742788198 7fd80a12ed7e4f62ae59fc68f93d7963 8fec1ddd5dd54f339f50b9695335002d] Connected to AMQP server on 172.18.198.52:5672 2014-08-29 04:37:47.423 ERROR oslo.messaging._drivers.common [req-2b7e770d-727b-40ba-a20d-088742788198 7fd80a12ed7e4f62ae59fc68f93d7963 8fec1ddd5dd54f339f50b9695335002d] Returning exception 'GenericShareDriver' object has no attribute 'service_instance_manager' to caller Looks like "setup_server" is called before "do_setup" method, that sets up attr "service_instance_manager".
2014-08-29 09:10:09 Valeriy Ponomaryov description One backend - generic driver. Bug case: 1) Service m-shr is down, service m-api is up. 2) m-api receives request to create share with share-network that "has not" server. 3) m-shr starts and outputs: 2014-08-29 04:37:47.405 ERROR oslo.messaging.rpc.dispatcher [req-2b7e770d-727b-40ba-a20d-088742788198 7fd80a12ed7e4f62ae59fc68f93d7963 8fec1ddd5dd54f339f50b9695335002d] Exception during message handling: 'GenericShareDriver' objec t has no attribute 'service_instance_manager' 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last): 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 134, in _dispatch_and_reply 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher incoming.message)) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 177, in _dispatch 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher return self._do_dispatch(endpoint, method, ctxt, args) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 123, in _do_dispatch 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher result = getattr(endpoint, method)(ctxt, **new_args) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 203, in create_share 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher {'status': 'error'}) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/openstack/common/excutils.py", line 82, in __exit__ 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 197, in create_share 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher context, share_network_id, share_id) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 154, in _provide_share_server_for_share 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher return _provide_share_server_for_share() 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/openstack/common/lockutils.py", line 325, in inner 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher return f(*args, **kwargs) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 147, in _provide_share_server_for_share 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher share_server = self._setup_server(context, share_server) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 423, in _setup_server 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher self.network_api.deallocate_network(context, share_network) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/openstack/common/excutils.py", line 82, in __exit__ 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 411, in _setup_server 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher metadata=metadata) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/drivers/generic.py", line 565, in setup_server 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher server = self.service_instance_manager.set_up_service_instance( 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher AttributeError: 'GenericShareDriver' object has no attribute 'service_instance_manager' 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher 2014-08-29 04:37:47.413 INFO oslo.messaging._drivers.impl_rabbit [req-2b7e770d-727b-40ba-a20d-088742788198 7fd80a12ed7e4f62ae59fc68f93d7963 8fec1ddd5dd54f339f50b9695335002d] Connecting to AMQP server on 172.18.198.52:5672 2014-08-29 04:37:47.422 INFO oslo.messaging._drivers.impl_rabbit [req-2b7e770d-727b-40ba-a20d-088742788198 7fd80a12ed7e4f62ae59fc68f93d7963 8fec1ddd5dd54f339f50b9695335002d] Connected to AMQP server on 172.18.198.52:5672 2014-08-29 04:37:47.423 ERROR oslo.messaging._drivers.common [req-2b7e770d-727b-40ba-a20d-088742788198 7fd80a12ed7e4f62ae59fc68f93d7963 8fec1ddd5dd54f339f50b9695335002d] Returning exception 'GenericShareDriver' object has no attribute 'service_instance_manager' to caller Looks like "setup_server" is called before "do_setup" method, that sets up attr "service_instance_manager". One backend - generic driver. Bug case: 1) Service m-shr is down, service m-api is up. 2) m-api receives request to create share with share-network. 3) m-shr starts and outputs: 2014-08-29 04:37:47.405 ERROR oslo.messaging.rpc.dispatcher [req-2b7e770d-727b-40ba-a20d-088742788198 7fd80a12ed7e4f62ae59fc68f93d7963 8fec1ddd5dd54f339f50b9695335002d] Exception during message handling: 'GenericShareDriver' objec t has no attribute 'service_instance_manager' 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last): 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 134, in _dispatch_and_reply 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher incoming.message)) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 177, in _dispatch 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher return self._do_dispatch(endpoint, method, ctxt, args) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 123, in _do_dispatch 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher result = getattr(endpoint, method)(ctxt, **new_args) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 203, in create_share 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher {'status': 'error'}) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/openstack/common/excutils.py", line 82, in __exit__ 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 197, in create_share 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher context, share_network_id, share_id) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 154, in _provide_share_server_for_share 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher return _provide_share_server_for_share() 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/openstack/common/lockutils.py", line 325, in inner 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher return f(*args, **kwargs) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 147, in _provide_share_server_for_share 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher share_server = self._setup_server(context, share_server) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 423, in _setup_server 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher self.network_api.deallocate_network(context, share_network) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/openstack/common/excutils.py", line 82, in __exit__ 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 411, in _setup_server 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher metadata=metadata) 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/drivers/generic.py", line 565, in setup_server 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher server = self.service_instance_manager.set_up_service_instance( 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher AttributeError: 'GenericShareDriver' object has no attribute 'service_instance_manager' 2014-08-29 04:37:47.405 TRACE oslo.messaging.rpc.dispatcher 2014-08-29 04:37:47.413 INFO oslo.messaging._drivers.impl_rabbit [req-2b7e770d-727b-40ba-a20d-088742788198 7fd80a12ed7e4f62ae59fc68f93d7963 8fec1ddd5dd54f339f50b9695335002d] Connecting to AMQP server on 172.18.198.52:5672 2014-08-29 04:37:47.422 INFO oslo.messaging._drivers.impl_rabbit [req-2b7e770d-727b-40ba-a20d-088742788198 7fd80a12ed7e4f62ae59fc68f93d7963 8fec1ddd5dd54f339f50b9695335002d] Connected to AMQP server on 172.18.198.52:5672 2014-08-29 04:37:47.423 ERROR oslo.messaging._drivers.common [req-2b7e770d-727b-40ba-a20d-088742788198 7fd80a12ed7e4f62ae59fc68f93d7963 8fec1ddd5dd54f339f50b9695335002d] Returning exception 'GenericShareDriver' object has no attribute 'service_instance_manager' to caller Looks like "setup_server" is called before "do_setup" method, that sets up attr "service_instance_manager".
2014-08-29 09:10:23 Valeriy Ponomaryov tags driver generic
2014-08-29 09:10:32 Valeriy Ponomaryov manila: importance Undecided High
2014-08-29 09:30:07 Valeriy Ponomaryov manila: assignee Valeriy Ponomaryov (vponomaryov)
2014-08-29 09:30:14 Valeriy Ponomaryov manila: status New In Progress
2014-08-29 09:30:23 Valeriy Ponomaryov summary share can not be created if scheduled with disabled m-shr share can not be created if scheduled with disabled m-shr using generic driver
2014-09-03 09:06:23 OpenStack Infra manila: status In Progress Fix Committed
2014-09-03 12:54:19 Valeriy Ponomaryov manila: status Fix Committed Fix Released
2014-09-03 15:25:32 Ben Swartzlander manila: milestone juno-3