Generic driver creates ports in Neutron for each share server but does not delete it with deletion of share servers. It leads to orphanage of ports and fixed ip leak.
Here is error from manila-share service:
http://paste.openstack.org/show/217201/
2015-05-08 18:51:52.46 ERROR manila.network.neutron.api [req-9677492e-ea8d-49f5-ad2f-194178c44b81 ca779e1e8d3f46308b5d2341c7ae3e27 3e255d4c73c34d2bb1a2797cd3e8a4e1] Neutron error creating port on network 31baf6a4-aa9b-442f-9034-8948feacad40 2015-05-08 18:51:52.46 TRACE manila.network.neutron.api Traceback (most recent call last): 2015-05-08 18:51:52.46 TRACE manila.network.neutron.api File "/opt/stack/manila/manila/network/neutron/api.py", line 164, in create_port 2015-05-08 18:51:52.46 TRACE manila.network.neutron.api port = self.client.create_port(port_req_body).get('port', {}) 2015-05-08 18:51:52.46 TRACE manila.network.neutron.api File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 99, in with_params 2015-05-08 18:51:52.46 TRACE manila.network.neutron.api ret = self.function(instance, *args, **kwargs) 2015-05-08 18:51:52.46 TRACE manila.network.neutron.api File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 507, in create_port 2015-05-08 18:51:52.46 TRACE manila.network.neutron.api return self.post(self.ports_path, body=body) 2015-05-08 18:51:52.46 TRACE manila.network.neutron.api File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 295, in post 2015-05-08 18:51:52.46 TRACE manila.network.neutron.api headers=headers, params=params) 2015-05-08 18:51:52.46 TRACE manila.network.neutron.api File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 208, in do_request 2015-05-08 18:51:52.46 TRACE manila.network.neutron.api self._handle_fault_response(status_code, replybody) 2015-05-08 18:51:52.46 TRACE manila.network.neutron.api File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 182, in _handle_fault_response 2015-05-08 18:51:52.46 TRACE manila.network.neutron.api exception_handler_v20(status_code, des_error_body) 2015-05-08 18:51:52.46 TRACE manila.network.neutron.api File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 67, in exception_handler_v20 2015-05-08 18:51:52.46 TRACE manila.network.neutron.api status_code=status_code) 2015-05-08 18:51:52.46 TRACE manila.network.neutron.api IpAddressGenerationFailureClient: No more IP addresses available on network 31baf6a4-aa9b-442f-9034-8948feacad40. 2015-05-08 18:51:52.46 TRACE manila.network.neutron.api 2015-05-08 18:51:52.66 DEBUG oslo_concurrency.lockutils [req-9677492e-ea8d-49f5-ad2f-194178c44b81 ca779e1e8d3f46308b5d2341c7ae3e27 3e255d4c73c34d2bb1a2797cd3e8a4e1] Releasing file lock "/tmp/custom_manila_lock_path/manila-service_instance_setup_and_teardown_network_for_instance" after holding it for 0.273s from (pid=6863) release /usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:227 2015-05-08 18:51:52.66 DEBUG oslo_concurrency.lockutils [req-9677492e-ea8d-49f5-ad2f-194178c44b81 ca779e1e8d3f46308b5d2341c7ae3e27 3e255d4c73c34d2bb1a2797cd3e8a4e1] Lock "service_instance_setup_and_teardown_network_for_instance" released by "setup_network" :: held 0.273s from (pid=6863) inner /usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:456 2015-05-08 18:51:52.67 WARNING manila.share.manager [req-9677492e-ea8d-49f5-ad2f-194178c44b81 ca779e1e8d3f46308b5d2341c7ae3e27 3e255d4c73c34d2bb1a2797cd3e8a4e1] Server Information in exception can not be written to db : Invalid server_details 'None' 2015-05-08 18:51:52.127 DEBUG oslo_concurrency.lockutils [req-9677492e-ea8d-49f5-ad2f-194178c44b81 ca779e1e8d3f46308b5d2341c7ae3e27 3e255d4c73c34d2bb1a2797cd3e8a4e1] Lock "share_manager_93042963-2a58-4106-b391-d2b96eb01579" released by "_provide_share_server_for_share" :: held 1.281s from (pid=6863) inner /usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:456 2015-05-08 18:51:52.128 ERROR manila.share.manager [req-9677492e-ea8d-49f5-ad2f-194178c44b81 ca779e1e8d3f46308b5d2341c7ae3e27 3e255d4c73c34d2bb1a2797cd3e8a4e1] Failed to get share server for share creation. 2015-05-08 18:51:52.176 ERROR oslo_messaging.rpc.dispatcher [req-9677492e-ea8d-49f5-ad2f-194178c44b81 ca779e1e8d3f46308b5d2341c7ae3e27 3e255d4c73c34d2bb1a2797cd3e8a4e1] Exception during message handling: Maximum number of ports exceeded. 2015-05-08 18:51:52.176 TRACE oslo_messaging.rpc.dispatcher Traceback (most recent call last): 2015-05-08 18:51:52.176 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply 2015-05-08 18:51:52.176 TRACE oslo_messaging.rpc.dispatcher executor_callback)) 2015-05-08 18:51:52.176 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch 2015-05-08 18:51:52.176 TRACE oslo_messaging.rpc.dispatcher executor_callback) 2015-05-08 18:51:52.176 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 130, in _do_dispatch 2015-05-08 18:51:52.176 TRACE oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args) 2015-05-08 18:51:52.176 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 308, in create_share 2015-05-08 18:51:52.176 TRACE oslo_messaging.rpc.dispatcher {'status': 'error'}) 2015-05-08 18:51:52.176 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__ 2015-05-08 18:51:52.176 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2015-05-08 18:51:52.176 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 302, in create_share 2015-05-08 18:51:52.176 TRACE oslo_messaging.rpc.dispatcher context, share_network_id, share_id) 2015-05-08 18:51:52.176 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 253, in _provide_share_server_for_share 2015-05-08 18:51:52.176 TRACE oslo_messaging.rpc.dispatcher return _provide_share_server_for_share() 2015-05-08 18:51:52.176 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py", line 445, in inner 2015-05-08 18:51:52.176 TRACE oslo_messaging.rpc.dispatcher return f(*args, **kwargs) 2015-05-08 18:51:52.176 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 245, in _provide_share_server_for_share 2015-05-08 18:51:52.176 TRACE oslo_messaging.rpc.dispatcher share_server = self._setup_server(context, share_server) 2015-05-08 18:51:52.176 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 741, in _setup_server 2015-05-08 18:51:52.176 TRACE oslo_messaging.rpc.dispatcher self.driver.deallocate_network(context, share_server['id']) 2015-05-08 18:51:52.176 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__ 2015-05-08 18:51:52.176 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2015-05-08 18:51:52.176 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 692, in _setup_server 2015-05-08 18:51:52.176 TRACE oslo_messaging.rpc.dispatcher network_info, metadata=metadata) 2015-05-08 18:51:52.176 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/driver.py", line 332, in setup_server 2015-05-08 18:51:52.176 TRACE oslo_messaging.rpc.dispatcher return self._setup_server(*args, **kwargs) 2015-05-08 18:51:52.176 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/drivers/generic.py", line 631, in _setup_server 2015-05-08 18:51:52.176 TRACE oslo_messaging.rpc.dispatcher self.admin_context, network_info) 2015-05-08 18:51:52.176 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/drivers/service_instance.py", line 386, in set_up_service_instance 2015-05-08 18:51:52.176 TRACE oslo_messaging.rpc.dispatcher context, instance_name, network_info) 2015-05-08 18:51:52.176 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/drivers/service_instance.py", line 477, in _create_service_instance 2015-05-08 18:51:52.176 TRACE oslo_messaging.rpc.dispatcher network_data = self.network_helper.setup_network(network_info) 2015-05-08 18:51:52.176 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py", line 445, in inner 2015-05-08 18:51:52.176 TRACE oslo_messaging.rpc.dispatcher return f(*args, **kwargs) 2015-05-08 18:51:52.176 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/drivers/service_instance.py", line 728, in setup_network 2015-05-08 18:51:52.176 TRACE oslo_messaging.rpc.dispatcher device_owner='manila') 2015-05-08 18:51:52.176 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/manila/manila/network/neutron/api.py", line 170, in create_port 2015-05-08 18:51:52.176 TRACE oslo_messaging.rpc.dispatcher raise exception.PortLimitExceeded() 2015-05-08 18:51:52.176 TRACE oslo_messaging.rpc.dispatcher PortLimitExceeded: Maximum number of ports exceeded.
Generic driver creates ports in Neutron for each share server but does not delete it with deletion of share servers. It leads to orphanage of ports and fixed ip leak.
Here is error from manila-share service:
http:// paste.openstack .org/show/ 217201/
2015-05-08 18:51:52.46 ERROR manila. network. neutron. api [req-9677492e- ea8d-49f5- ad2f-194178c44b 81 ca779e1e8d3f463 08b5d2341c7ae3e 27 3e255d4c73c34d2 bb1a2797cd3e8a4 e1] Neutron error creating port on network 31baf6a4- aa9b-442f- 9034-8948feacad 40 network. neutron. api Traceback (most recent call last): network. neutron. api File "/opt/stack/ manila/ manila/ network/ neutron/ api.py" , line 164, in create_port network. neutron. api port = self.client. create_ port(port_ req_body) .get('port' , {}) network. neutron. api File "/usr/local/ lib/python2. 7/dist- packages/ neutronclient/ v2_0/client. py", line 99, in with_params network. neutron. api ret = self.function( instance, *args, **kwargs) network. neutron. api File "/usr/local/ lib/python2. 7/dist- packages/ neutronclient/ v2_0/client. py", line 507, in create_port network. neutron. api return self.post( self.ports_ path, body=body) network. neutron. api File "/usr/local/ lib/python2. 7/dist- packages/ neutronclient/ v2_0/client. py", line 295, in post network. neutron. api headers=headers, params=params) network. neutron. api File "/usr/local/ lib/python2. 7/dist- packages/ neutronclient/ v2_0/client. py", line 208, in do_request network. neutron. api self._handle_ fault_response( status_ code, replybody) network. neutron. api File "/usr/local/ lib/python2. 7/dist- packages/ neutronclient/ v2_0/client. py", line 182, in _handle_ fault_response network. neutron. api exception_ handler_ v20(status_ code, des_error_body) network. neutron. api File "/usr/local/ lib/python2. 7/dist- packages/ neutronclient/ v2_0/client. py", line 67, in exception_ handler_ v20 network. neutron. api status_ code=status_ code) network. neutron. api IpAddressGenera tionFailureClie nt: No more IP addresses available on network 31baf6a4- aa9b-442f- 9034-8948feacad 40. network. neutron. api y.lockutils [req-9677492e- ea8d-49f5- ad2f-194178c44b 81 ca779e1e8d3f463 08b5d2341c7ae3e 27 3e255d4c73c34d2 bb1a2797cd3e8a4 e1] Releasing file lock "/tmp/custom_ manila_ lock_path/ manila- service_ instance_ setup_and_ teardown_ network_ for_instance" after holding it for 0.273s from (pid=6863) release /usr/local/ lib/python2. 7/dist- packages/ oslo_concurrenc y/lockutils. py:227 y.lockutils [req-9677492e- ea8d-49f5- ad2f-194178c44b 81 ca779e1e8d3f463 08b5d2341c7ae3e 27 3e255d4c73c34d2 bb1a2797cd3e8a4 e1] Lock "service_ instance_ setup_and_ teardown_ network_ for_instance" released by "setup_network" :: held 0.273s from (pid=6863) inner /usr/local/ lib/python2. 7/dist- packages/ oslo_concurrenc y/lockutils. py:456 share.manager [req-9677492e- ea8d-49f5- ad2f-194178c44b 81 ca779e1e8d3f463 08b5d2341c7ae3e 27 3e255d4c73c34d2 bb1a2797cd3e8a4 e1] Server Information in exception can not be written to db : Invalid server_details 'None' y.lockutils [req-9677492e- ea8d-49f5- ad2f-194178c44b 81 ca779e1e8d3f463 08b5d2341c7ae3e 27 3e255d4c73c34d2 bb1a2797cd3e8a4 e1] Lock "share_ manager_ 93042963- 2a58-4106- b391-d2b96eb015 79" released by "_provide_ share_server_ for_share" :: held 1.281s from (pid=6863) inner /usr/local/ lib/python2. 7/dist- packages/ oslo_concurrenc y/lockutils. py:456 share.manager [req-9677492e- ea8d-49f5- ad2f-194178c44b 81 ca779e1e8d3f463 08b5d2341c7ae3e 27 3e255d4c73c34d2 bb1a2797cd3e8a4 e1] Failed to get share server for share creation. rpc.dispatcher [req-9677492e- ea8d-49f5- ad2f-194178c44b 81 ca779e1e8d3f463 08b5d2341c7ae3e 27 3e255d4c73c34d2 bb1a2797cd3e8a4 e1] Exception during message handling: Maximum number of ports exceeded. rpc.dispatcher Traceback (most recent call last): rpc.dispatcher File "/usr/local/ lib/python2. 7/dist- packages/ oslo_messaging/ rpc/dispatcher. py", line 142, in _dispatch_and_reply rpc.dispatcher executor_callback)) rpc.dispatcher File "/usr/local/ lib/python2. 7/dist- packages/ oslo_messaging/ rpc/dispatcher. py", line 186, in _dispatch rpc.dispatcher executor_callback) rpc.dispatcher File "/usr/local/ lib/python2. 7/dist- packages/ oslo_messaging/ rpc/dispatcher. py", line 130, in _do_dispatch rpc.dispatcher result = func(ctxt, **new_args) rpc.dispatcher File "/opt/stack/ manila/ manila/ share/manager. py", line 308, in create_share rpc.dispatcher {'status': 'error'}) rpc.dispatcher File "/usr/local/ lib/python2. 7/dist- packages/ oslo_utils/ excutils. py", line 85, in __exit__ rpc.dispatcher six.reraise( self.type_ , self.value, self.tb) rpc.dispatcher File "/opt/stack/ manila/ manila/ share/manager. py", line 302, in create_share rpc.dispatcher context, share_network_id, share_id) rpc.dispatcher File "/opt/stack/ manila/ manila/ share/manager. py", line 253, in _provide_ share_server_ for_share rpc.dispatcher return _provide_ share_server_ for_share( ) rpc.dispatcher File "/usr/local/ lib/python2. 7/dist- packages/ oslo_concurrenc y/lockutils. py", line 445, in inner rpc.dispatcher return f(*args, **kwargs) rpc.dispatcher File "/opt/stack/ manila/ manila/ share/manager. py", line 245, in _provide_ share_server_ for_share rpc.dispatcher share_server = self._setup_ server( context, share_server) rpc.dispatcher File "/opt/stack/ manila/ manila/ share/manager. py", line 741, in _setup_server rpc.dispatcher self.driver. deallocate_ network( context, share_server['id']) rpc.dispatcher File "/usr/local/ lib/python2. 7/dist- packages/ oslo_utils/ excutils. py", line 85, in __exit__ rpc.dispatcher six.reraise( self.type_ , self.value, self.tb) rpc.dispatcher File "/opt/stack/ manila/ manila/ share/manager. py", line 692, in _setup_server rpc.dispatcher network_info, metadata=metadata) rpc.dispatcher File "/opt/stack/ manila/ manila/ share/driver. py", line 332, in setup_server rpc.dispatcher return self._setup_ server( *args, **kwargs) rpc.dispatcher File "/opt/stack/ manila/ manila/ share/drivers/ generic. py", line 631, in _setup_server rpc.dispatcher self.admin_context, network_info) rpc.dispatcher File "/opt/stack/ manila/ manila/ share/drivers/ service_ instance. py", line 386, in set_up_ service_ instance rpc.dispatcher context, instance_name, network_info) rpc.dispatcher File "/opt/stack/ manila/ manila/ share/drivers/ service_ instance. py", line 477, in _create_ service_ instance rpc.dispatcher network_data = self.network_ helper. setup_network( network_ info) rpc.dispatcher File "/usr/local/ lib/python2. 7/dist- packages/ oslo_concurrenc y/lockutils. py", line 445, in inner rpc.dispatcher return f(*args, **kwargs) rpc.dispatcher File "/opt/stack/ manila/ manila/ share/drivers/ service_ instance. py", line 728, in setup_network rpc.dispatcher device_ owner=' manila' ) rpc.dispatcher File "/opt/stack/ manila/ manila/ network/ neutron/ api.py" , line 170, in create_port rpc.dispatcher raise exception. PortLimitExceed ed() rpc.dispatcher PortLimitExceeded: Maximum number of ports exceeded.
2015-05-08 18:51:52.46 TRACE manila.
2015-05-08 18:51:52.46 TRACE manila.
2015-05-08 18:51:52.46 TRACE manila.
2015-05-08 18:51:52.46 TRACE manila.
2015-05-08 18:51:52.46 TRACE manila.
2015-05-08 18:51:52.46 TRACE manila.
2015-05-08 18:51:52.46 TRACE manila.
2015-05-08 18:51:52.46 TRACE manila.
2015-05-08 18:51:52.46 TRACE manila.
2015-05-08 18:51:52.46 TRACE manila.
2015-05-08 18:51:52.46 TRACE manila.
2015-05-08 18:51:52.46 TRACE manila.
2015-05-08 18:51:52.46 TRACE manila.
2015-05-08 18:51:52.46 TRACE manila.
2015-05-08 18:51:52.46 TRACE manila.
2015-05-08 18:51:52.46 TRACE manila.
2015-05-08 18:51:52.46 TRACE manila.
2015-05-08 18:51:52.66 DEBUG oslo_concurrenc
2015-05-08 18:51:52.66 DEBUG oslo_concurrenc
2015-05-08 18:51:52.67 WARNING manila.
2015-05-08 18:51:52.127 DEBUG oslo_concurrenc
2015-05-08 18:51:52.128 ERROR manila.
2015-05-08 18:51:52.176 ERROR oslo_messaging.
2015-05-08 18:51:52.176 TRACE oslo_messaging.
2015-05-08 18:51:52.176 TRACE oslo_messaging.
2015-05-08 18:51:52.176 TRACE oslo_messaging.
2015-05-08 18:51:52.176 TRACE oslo_messaging.
2015-05-08 18:51:52.176 TRACE oslo_messaging.
2015-05-08 18:51:52.176 TRACE oslo_messaging.
2015-05-08 18:51:52.176 TRACE oslo_messaging.
2015-05-08 18:51:52.176 TRACE oslo_messaging.
2015-05-08 18:51:52.176 TRACE oslo_messaging.
2015-05-08 18:51:52.176 TRACE oslo_messaging.
2015-05-08 18:51:52.176 TRACE oslo_messaging.
2015-05-08 18:51:52.176 TRACE oslo_messaging.
2015-05-08 18:51:52.176 TRACE oslo_messaging.
2015-05-08 18:51:52.176 TRACE oslo_messaging.
2015-05-08 18:51:52.176 TRACE oslo_messaging.
2015-05-08 18:51:52.176 TRACE oslo_messaging.
2015-05-08 18:51:52.176 TRACE oslo_messaging.
2015-05-08 18:51:52.176 TRACE oslo_messaging.
2015-05-08 18:51:52.176 TRACE oslo_messaging.
2015-05-08 18:51:52.176 TRACE oslo_messaging.
2015-05-08 18:51:52.176 TRACE oslo_messaging.
2015-05-08 18:51:52.176 TRACE oslo_messaging.
2015-05-08 18:51:52.176 TRACE oslo_messaging.
2015-05-08 18:51:52.176 TRACE oslo_messaging.
2015-05-08 18:51:52.176 TRACE oslo_messaging.
2015-05-08 18:51:52.176 TRACE oslo_messaging.
2015-05-08 18:51:52.176 TRACE oslo_messaging.
2015-05-08 18:51:52.176 TRACE oslo_messaging.
2015-05-08 18:51:52.176 TRACE oslo_messaging.
2015-05-08 18:51:52.176 TRACE oslo_messaging.
2015-05-08 18:51:52.176 TRACE oslo_messaging.
2015-05-08 18:51:52.176 TRACE oslo_messaging.
2015-05-08 18:51:52.176 TRACE oslo_messaging.
2015-05-08 18:51:52.176 TRACE oslo_messaging.
2015-05-08 18:51:52.176 TRACE oslo_messaging.
2015-05-08 18:51:52.176 TRACE oslo_messaging.
2015-05-08 18:51:52.176 TRACE oslo_messaging.
2015-05-08 18:51:52.176 TRACE oslo_messaging.
2015-05-08 18:51:52.176 TRACE oslo_messaging.
2015-05-08 18:51:52.176 TRACE oslo_messaging.