glance http store is broken with dnspytho 2.6.0

[Abhishek Kekane]

Our gate is broken with new version of dnspython 2.6.0.
You can see nova-ceph-multistore job started failing since today [1]

Last time it passed the dnspython 2.5.0 was used.

[1] https://zuul.opendev.org/t/openstack/builds?job_name=nova-ceph-multistore&skip=0

Stacktrace in g-api:

Feb 22 12:51:43.010574 np0036837391 glance-api[105921]: ERROR glance.async_.flows._internal_plugins.web_download [-] Task 3ff71b99-5784-4fbd-aa33-c287cc7de816 failed with exception <urlopen error [Errno -3] Lookup timed out>: urllib.error.URLError: <urlopen error [Errno -3] Lookup timed out>
Feb 22 12:51:43.017897 np0036837391 glance-api[105921]: WARNING glance.async_.taskflow_executor [-] Task 'api_image_import-WebDownload-3ff71b99-5784-4fbd-aa33-c287cc7de816' (959ff8ac-9ba5-406b-b130-8d9a9846b97e) transitioned into state 'FAILURE' from state 'RUNNING'
Feb 22 12:51:43.017897 np0036837391 glance-api[105921]: 4 predecessors (most recent first):
Feb 22 12:51:43.017897 np0036837391 glance-api[105921]: Flow 'api_image_import'
Feb 22 12:51:43.017897 np0036837391 glance-api[105921]: |__Atom 'api_image_import-ImageLock-3ff71b99-5784-4fbd-aa33-c287cc7de816' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {}, 'provides': None}
Feb 22 12:51:43.017897 np0036837391 glance-api[105921]: |__Atom 'api_image_import_retry' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {}, 'provides': [(None, {})]}
Feb 22 12:51:43.017897 np0036837391 glance-api[105921]: |__Flow 'api_image_import': urllib.error.URLError: <urlopen error [Errno -3] Lookup timed out>
Feb 22 12:51:43.017897 np0036837391 glance-api[105921]: ERROR glance.async_.taskflow_executor Traceback (most recent call last):
Feb 22 12:51:43.017897 np0036837391 glance-api[105921]: ERROR glance.async_.taskflow_executor File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/support/greendns.py", line 463, in resolve
Feb 22 12:51:43.017897 np0036837391 glance-api[105921]: ERROR glance.async_.taskflow_executor return _proxy.query(name, rdtype, raise_on_no_answer=raises,
Feb 22 12:51:43.017897 np0036837391 glance-api[105921]: ERROR glance.async_.taskflow_executor File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/support/greendns.py", line 419, in query
Feb 22 12:51:43.017897 np0036837391 glance-api[105921]: ERROR glance.async_.taskflow_executor return end()
Feb 22 12:51:43.017897 np0036837391 glance-api[105921]: ERROR glance.async_.taskflow_executor File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/support/greendns.py", line 398, in end
Feb 22 12:51:43.017897 np0036837391 glance-api[105921]: ERROR glance.async_.taskflow_executor raise result[1]
Feb 22 12:51:43.017897 np0036837391 glance-api[105921]: ERROR glance.async_.taskflow_executor File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/support/greendns.py", line 379, in step
Feb 22 12:51:43.017897 np0036837391 glance-api[105921]: ERROR glance.async_.taskflow_executor a = fun(*args, **kwargs)
Feb 22 12:51:43.017897 np0036837391 glance-api[105921]: ERROR glance.async_.taskflow_executor File "/opt/stack/data/venv/lib/python3.10/site-packages/dns/resolver.py", line 1364, in query
Feb 22 12:51:43.017897 np0036837391 glance-api[105921]: ERROR glance.async_.taskflow_executor return self.resolve(
Feb 22 12:51:43.017897 np0036837391 glance-api[105921]: ERROR glance.async_.taskflow_executor File "/opt/stack/data/venv/lib/python3.10/site-packages/dns/resolver.py", line 1321, in resolve
Feb 22 12:51:43.017897 np0036837391 glance-api[105921]: ERROR glance.async_.taskflow_executor timeout = self._compute_timeout(start, lifetime, resolution.errors)
Feb 22 12:51:43.017897 np0036837391 glance-api[105921]: ERROR glance.async_.taskflow_executor File "/opt/stack/data/venv/lib/python3.10/site-packages/dns/resolver.py", line 1075, in _compute_timeout
Feb 22 12:51:43.017897 np0036837391 glance-api[105921]: ERROR glance.async_.taskflow_executor raise LifetimeTimeout(timeout=duration, errors=errors)
Feb 22 12:51:43.017897 np0036837391 glance-api[105921]: ERROR glance.async_.taskflow_executor dns.resolver.LifetimeTimeout: The resolution lifetime expired after 5.102 seconds: Server Do53:127.0.0.1@53 answered udp() got an unexpected keyword argument 'ignore_errors'; Server Do53:127.0.0.1@53 answered udp() got an unexpected keyword argument 'ignore_errors'; Server Do53:127.0.0.1@53 answered udp() got an unexpected keyword argument 'ignore_errors'; Server Do53:127.0.0.1@53 answered udp() got an unexpected keyword argument 'ignore_errors'; Server Do53:127.0.0.1@53 answered udp() got an unexpected keyword argument 'ignore_errors'; Server Do53:127.0.0.1@53 answered udp() got an unexpected keyword argument 'ignore_errors'

[Cyril Roelandt (cyril-roelandt)]

https://review.opendev.org/c/openstack/requirements/+/909923 should fix it

[Abhishek Kekane (abhishek-kekane)]

Fix proposed: https://review.opendev.org/c/openstack/requirements/+/909923