Glare

When glare can't connect to Swift 500 error is raised

Bug #1621232 reported by Mike Fedosin on 2016-09-07

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	Glare	Invalid	Low	Mike Fedosin

Bug Description

When user tries to upload or download file and Glare can't connect to Swift ConnectionError is raised. Glare doesn't catch it and returns 500 error to the user.

2016-09-07 23:06:27.100 ERROR glare.common.wsgi [req-1ccf3cf2-a9cb-489c-825d-5ec90ca6f92d f08733b95dbc4c949e07b822d00fc999 0c86483fe46d4ca3a85b36f5ba9b75bc] Caught error: HTTPConnectionPool(host='10.0.2.15', port=8080): Max retries exceeded with url: /v1/AUTH_478d851108e94937a1d120dff19ff4d9/glance (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7fea690b5110>: Failed to establish a new connection: [Errno 111] ECONNREFUSED',))
2016-09-07 23:06:27.100 TRACE glare.common.wsgi Traceback (most recent call last):
2016-09-07 23:06:27.100 TRACE glare.common.wsgi File "/usr/local/lib/python2.7/dist-packages/glare/common/wsgi.py", line 782, in __call__
2016-09-07 23:06:27.100 TRACE glare.common.wsgi request, **action_args)
2016-09-07 23:06:27.100 TRACE glare.common.wsgi File "/usr/local/lib/python2.7/dist-packages/glare/common/wsgi.py", line 825, in dispatch
2016-09-07 23:06:27.100 TRACE glare.common.wsgi return method(*args, **kwargs)
2016-09-07 23:06:27.100 TRACE glare.common.wsgi File "/usr/local/lib/python2.7/dist-packages/glare/api/v1/resource.py", line 191, in log_decorator
2016-09-07 23:06:27.100 TRACE glare.common.wsgi result = f(self, req, *args, **kwargs)
2016-09-07 23:06:27.100 TRACE glare.common.wsgi File "/usr/local/lib/python2.7/dist-packages/glare/api/v1/resource.py", line 317, in upload_blob
2016-09-07 23:06:27.100 TRACE glare.common.wsgi field_name, data, content_type)
2016-09-07 23:06:27.100 TRACE glare.common.wsgi File "/usr/local/lib/python2.7/dist-packages/glare/locking.py", line 133, in wrapped
2016-09-07 23:06:27.100 TRACE glare.common.wsgi res = f(*a, **k)
2016-09-07 23:06:27.100 TRACE glare.common.wsgi File "/usr/local/lib/python2.7/dist-packages/glare/engine.py", line 303, in upload_blob
2016-09-07 23:06:27.100 TRACE glare.common.wsgi modified_af = af.upload_blob(context, af, field_name, fd, content_type)
2016-09-07 23:06:27.100 TRACE glare.common.wsgi File "/usr/local/lib/python2.7/dist-packages/glare/objects/base.py", line 842, in upload_blob
2016-09-07 23:06:27.100 TRACE glare.common.wsgi cls.db_api.update(context, af.id, {field_name: None})
2016-09-07 23:06:27.100 TRACE glare.common.wsgi File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-09-07 23:06:27.100 TRACE glare.common.wsgi self.force_reraise()
2016-09-07 23:06:27.100 TRACE glare.common.wsgi File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-09-07 23:06:27.100 TRACE glare.common.wsgi six.reraise(self.type_, self.value, self.tb)
2016-09-07 23:06:27.100 TRACE glare.common.wsgi File "/usr/local/lib/python2.7/dist-packages/glare/objects/base.py", line 830, in upload_blob
2016-09-07 23:06:27.100 TRACE glare.common.wsgi blob_id, fd, context, cls._get_max_blob_size(field_name))
2016-09-07 23:06:27.100 TRACE glare.common.wsgi File "/usr/local/lib/python2.7/dist-packages/glare/common/utils.py", line 536, in new_function
2016-09-07 23:06:27.100 TRACE glare.common.wsgi return f(*args, **kwargs)
2016-09-07 23:06:27.100 TRACE glare.common.wsgi File "/usr/local/lib/python2.7/dist-packages/glare/common/store_api.py", line 71, in save_blob_to_store
2016-09-07 23:06:27.100 TRACE glare.common.wsgi 0, store_type, context, verifier)
2016-09-07 23:06:27.100 TRACE glare.common.wsgi File "/usr/local/lib/python2.7/dist-packages/glance_store/backend.py", line 371, in add_to_backend
2016-09-07 23:06:27.100 TRACE glare.common.wsgi verifier)
2016-09-07 23:06:27.100 TRACE glare.common.wsgi File "/usr/local/lib/python2.7/dist-packages/glance_store/backend.py", line 344, in store_add_to_backend
2016-09-07 23:06:27.100 TRACE glare.common.wsgi verifier=verifier)
2016-09-07 23:06:27.100 TRACE glare.common.wsgi File "/usr/local/lib/python2.7/dist-packages/glance_store/capabilities.py", line 225, in op_checker
2016-09-07 23:06:27.100 TRACE glare.common.wsgi return store_op_fun(store, *args, **kwargs)
2016-09-07 23:06:27.100 TRACE glare.common.wsgi File "/usr/local/lib/python2.7/dist-packages/glance_store/_drivers/swift/store.py", line 665, in add
2016-09-07 23:06:27.100 TRACE glare.common.wsgi manager.get_connection())
2016-09-07 23:06:27.100 TRACE glare.common.wsgi File "/usr/local/lib/python2.7/dist-packages/glance_store/_drivers/swift/store.py", line 860, in _create_container_if_missing
2016-09-07 23:06:27.100 TRACE glare.common.wsgi connection.head_container(container)
2016-09-07 23:06:27.100 TRACE glare.common.wsgi File "/usr/local/lib/python2.7/dist-packages/swiftclient/client.py", line 1626, in head_container
2016-09-07 23:06:27.100 TRACE glare.common.wsgi return self._retry(None, head_container, container, headers=headers)
2016-09-07 23:06:27.100 TRACE glare.common.wsgi File "/usr/local/lib/python2.7/dist-packages/swiftclient/client.py", line 1565, in _retry
2016-09-07 23:06:27.100 TRACE glare.common.wsgi service_token=self.service_token, **kwargs)
2016-09-07 23:06:27.100 TRACE glare.common.wsgi File "/usr/local/lib/python2.7/dist-packages/swiftclient/client.py", line 918, in head_container
2016-09-07 23:06:27.100 TRACE glare.common.wsgi conn.request(method, path, '', req_headers)
2016-09-07 23:06:27.100 TRACE glare.common.wsgi File "/usr/local/lib/python2.7/dist-packages/swiftclient/client.py", line 401, in request
2016-09-07 23:06:27.100 TRACE glare.common.wsgi files=files, **self.requests_args)
2016-09-07 23:06:27.100 TRACE glare.common.wsgi File "/usr/local/lib/python2.7/dist-packages/swiftclient/client.py", line 384, in _request
2016-09-07 23:06:27.100 TRACE glare.common.wsgi return self.request_session.request(*arg, **kwarg)
2016-09-07 23:06:27.100 TRACE glare.common.wsgi File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 471, in request
2016-09-07 23:06:27.100 TRACE glare.common.wsgi resp = self.send(prep, **send_kwargs)
2016-09-07 23:06:27.100 TRACE glare.common.wsgi File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 581, in send
2016-09-07 23:06:27.100 TRACE glare.common.wsgi r = adapter.send(request, **kwargs)
2016-09-07 23:06:27.100 TRACE glare.common.wsgi File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 481, in send
2016-09-07 23:06:27.100 TRACE glare.common.wsgi raise ConnectionError(e, request=request)
2016-09-07 23:06:27.100 TRACE glare.common.wsgi ConnectionError: HTTPConnectionPool(host='10.0.2.15', port=8080): Max retries exceeded with url: /v1/AUTH_478d851108e94937a1d120dff19ff4d9/glance (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7fea690b5110>: Failed to establish a new connection: [Errno 111] ECONNREFUSED',))
2016-09-07 23:06:27.100 TRACE glare.common.wsgi

Mike Fedosin (mfedosin) on 2016-09-07

Changed in glare:
status:	New → Confirmed
importance:	Undecided → Low
assignee:	nobody → Mike Fedosin (mfedosin)

Mike Fedosin (mfedosin) on 2017-09-24

Changed in glare:
status:	Confirmed → Invalid

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.