Unable to upload images from external networks

[YG Kumar]

Hi All,

We have a rocky OSA 18.1.2 branch setup. Image uploads to glance from remote networks are failing.
When we try try to upload from within the openstack networks, it is working fine, but from external networks, it is timing out and this is the error we observe in the glance-api logs:

-----------------
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi [req-48abaf55-f91f-4962-b9bc-7dca62ae2e62 e62b971d129f67fee90791696d3b847ed790793378385e8b157f079c4aa025e2 4409fade1aee487bb92350d7cbdd91c3 - 0623aaad3fe64845b14d2ebaca620e16 0623aaad3fe64845b14d2ebaca620e16] Caught error: unable to receive chunked part: IOError: unable to receive chunked part
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi Traceback (most recent call last):
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance/common/wsgi.py", line 1276, in __call__
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi request, **action_args)
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance/common/wsgi.py", line 1319, in dispatch
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi return method(*args, **kwargs)
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance/common/utils.py", line 417, in wrapped
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi return func(self, req, *args, **kwargs)
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance/api/v2/image_data.py", line 279, in upload
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi self._restore(image_repo, image)
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi self.force_reraise()
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi six.reraise(self.type_, self.value, self.tb)
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance/api/v2/image_data.py", line 144, in upload
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi image.set_data(data, size, backend=backend)
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance/domain/proxy.py", line 198, in set_data
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi self.base.set_data(data, size, backend=backend)
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance/notifier.py", line 485, in set_data
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi _send_notification(notify_error, 'image.upload', msg)
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi self.force_reraise()
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi six.reraise(self.type_, self.value, self.tb)
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance/notifier.py", line 432, in set_data
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi self.repo.set_data(data, size, backend=backend)
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance/api/policy.py", line 193, in set_data
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi return self.image.set_data(*args, **kwargs)
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance/quota/__init__.py", line 310, in set_data
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi self.image.set_data(data, size=size, backend=backend)
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance/location.py", line 483, in set_data
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi verifier=verifier)
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance_store/backend.py", line 541, in add_to_backend_with_multihash
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi image_id, data, size, hashing_algo, store, context, verifier)
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance_store/backend.py", line 518, in store_add_to_backend_with_multihash
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi image_id, data, size, hashing_algo, context=context, verifier=verifier)
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance_store/driver.py", line 274, in add_adapter
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi metadata_dict) = store_add_fun(*args, **kwargs)
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance_store/capabilities.py", line 225, in op_checker
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi return store_op_fun(store, *args, **kwargs)
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance_store/_drivers/filesystem.py", line 714, in add
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi raise errors.get(e.errno, e)
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi IOError: unable to receive chunked part
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi
2019-09-17 11:50:52.488 6821 CRITICAL glance [req-48abaf55-f91f-4962-b9bc-7dca62ae2e62 e62b971d129f67fee90791696d3b847ed790793378385e8b157f079c4aa025e2 4409fade1aee487bb92350d7cbdd91c3 - 0623aaad3fe64845b14d2ebaca620e16 0623aaad3fe64845b14d2ebaca620e16] Unhandled error: IOError: write error
2019-09-17 11:50:52.488 6821 ERROR glance IOError: write error
2019-09-17 11:50:52.488 6821 ERROR glance
---------------------

All other api commands r working fine remotely, but only these uploads are failing. Ideally we suppose, openstack should allow remote uploads.

Thanks
Kumar

[YG Kumar (ygk-kmr)]

We dont have any firewall in between................