web-download doesn't work in proxied env

[Peter Sabaini]

I'm trying to import an image via the web-download method[0][1].

When kicking off the import process I'm getting this in the glance-api.log

2021-11-17 12:50:20.586 24884 ERROR glance.async_.taskflow_executor Traceback (most recent call last):
2021-11-17 12:50:20.586 24884 ERROR glance.async_.taskflow_executor File "/usr/lib/python3/dist-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task
2021-11-17 12:50:20.586 24884 ERROR glance.async_.taskflow_executor result = task.execute(**arguments)
2021-11-17 12:50:20.586 24884 ERROR glance.async_.taskflow_executor File "/usr/lib/python3/dist-packages/glance/async_/flows/_internal_plugins/web_download.py", line 116, in execute
2021-11-17 12:50:20.586 24884 ERROR glance.async_.taskflow_executor LOG.error("Task %(task_id)s failed with exception %(error)s",
2021-11-17 12:50:20.586 24884 ERROR glance.async_.taskflow_executor File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2021-11-17 12:50:20.586 24884 ERROR glance.async_.taskflow_executor self.force_reraise()
2021-11-17 12:50:20.586 24884 ERROR glance.async_.taskflow_executor File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2021-11-17 12:50:20.586 24884 ERROR glance.async_.taskflow_executor six.reraise(self.type_, self.value, self.tb)
2021-11-17 12:50:20.586 24884 ERROR glance.async_.taskflow_executor File "/usr/lib/python3/dist-packages/six.py", line 703, in reraise
2021-11-17 12:50:20.586 24884 ERROR glance.async_.taskflow_executor raise value
2021-11-17 12:50:20.586 24884 ERROR glance.async_.taskflow_executor File "/usr/lib/python3/dist-packages/glance/async_/flows/_internal_plugins/web_download.py", line 113, in execute
2021-11-17 12:50:20.586 24884 ERROR glance.async_.taskflow_executor data = script_utils.get_image_data_iter(self.uri)
2021-11-17 12:50:20.586 24884 ERROR glance.async_.taskflow_executor File "/usr/lib/python3/dist-packages/glance/common/scripts/utils.py", line 142, in get_image_data_iter
2021-11-17 12:50:20.586 24884 ERROR glance.async_.taskflow_executor return urllib.request.urlopen(uri)
2021-11-17 12:50:20.586 24884 ERROR glance.async_.taskflow_executor File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
2021-11-17 12:50:20.586 24884 ERROR glance.async_.taskflow_executor return opener.open(url, data, timeout)
2021-11-17 12:50:20.586 24884 ERROR glance.async_.taskflow_executor File "/usr/lib/python3.8/urllib/request.py", line 525, in open
2021-11-17 12:50:20.586 24884 ERROR glance.async_.taskflow_executor response = self._open(req, data)
2021-11-17 12:50:20.586 24884 ERROR glance.async_.taskflow_executor File "/usr/lib/python3.8/urllib/request.py", line 542, in _open
2021-11-17 12:50:20.586 24884 ERROR glance.async_.taskflow_executor result = self._call_chain(self.handle_open, protocol, protocol +
2021-11-17 12:50:20.586 24884 ERROR glance.async_.taskflow_executor File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
2021-11-17 12:50:20.586 24884 ERROR glance.async_.taskflow_executor result = func(*args)
2021-11-17 12:50:20.586 24884 ERROR glance.async_.taskflow_executor File "/usr/lib/python3.8/urllib/request.py", line 1383, in http_open
2021-11-17 12:50:20.586 24884 ERROR glance.async_.taskflow_executor return self.do_open(http.client.HTTPConnection, req)
2021-11-17 12:50:20.586 24884 ERROR glance.async_.taskflow_executor File "/usr/lib/python3.8/urllib/request.py", line 1357, in do_open
2021-11-17 12:50:20.586 24884 ERROR glance.async_.taskflow_executor raise URLError(err)
2021-11-17 12:50:20.586 24884 ERROR glance.async_.taskflow_executor urllib.error.URLError: <urlopen error [Errno 110] ETIMEDOUT>
2021-11-17 12:50:20.586 24884 ERROR glance.async_.taskflow_executor

The model is situated behind a http proxy.

I have set this model-config:

juju model-config | grep http-proxy
apt-http-proxy controller http://foo.proxy.host:3128
http-proxy default ""
juju-http-proxy controller http://foo.proxy.host:3128
snap-http-proxy controller http://foo.proxy.host:3128

It feels like the charm doesn't set up the glance-api process with the required http proxy config.

[0] https://docs.openstack.org/glance/latest/admin/interoperable-image-import.html
[1] https://docs.openstack.org/openstacksdk/latest/user/guides/image.html

[James Page (james-page)]

I dug around on this problem and I can't see an option that allows glance to use a proxy *just* for web-download - i.e. we'd have to set it for the whole glance API process with the associated nasty-ness that introduces when glance needs to talk with something else over HTTP such as keystone.

Options exist for access to a 'glance.store.http.store' but not for the importer.

[James Page (james-page)]

whitelisting and blacklist exists for the web-download importer but no proxy configuration.

This feels like a feature that needs to go into glance rather than being mashed in by the charm in some way that kinda works/maybe works.

[James Page (james-page)]

Added a task for glance directly to get developer opinion on this one.