Horizon is unable to retrieve Cinder API versions when it has a self-signed SSL certificate

[David Moreau Simard]

With "OPENSTACK_SSL_NO_VERIFY = True" in local_settings.py, Horizon is able to communicate with a Cinder API instance that is using a self-signed certificate.

However, before communicating with the Cinder API, it first uses cinderclient to retrieve available API versions: https://github.com/openstack/horizon/blob/d5b7feb5d4bf622905d717cd20fc83fd136c8a8c/openstack_dashboard/api/cinder.py#L263

The get_server_version method from cinderclient doesn't support an "insecure" or "verify" argument and the request it does to retrieve the API versions is currently always secure: https://github.com/openstack/python-cinderclient/blob/63b36a901bfaf2508a9c3cda1d8dafb8769f2340/cinderclient/client.py#L75-L109

Even with DEBUG logging enabled, it was not trivial to understand what was going on because the only thing that gets written to the logs is something that looks like this:

=====
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): 172.29.236.100:8776
Call to list enabled services failed. This is likely due to a problem communicating with the Cinder endpoint. Consistency Group panel will not be displayed.
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): 172.29.236.100:8776
Call to list enabled services failed. This is likely due to a problem communicating with the Cinder endpoint. Consistency Group Snapshot panel will not be displayed.
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): 172.29.236.100:8776
Call to list enabled services failed. This is likely due to a problem communicating with the Cinder endpoint. Volume Group panel will not be displayed.
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): 172.29.236.100:8776
Call to list enabled services failed. This is likely due to a problem communicating with the Cinder endpoint. Volume Group Snapshot panel will not be displayed.
=====

I had to manually add some tracing to get the actual exception:

=====
Traceback (most recent call last):
  File "/openstack/venvs/horizon-18.1.3/lib/python2.7/site-packages/openstack_dashboard/dashboards/project/volumes/views.py", line 63, in _get_volumes
    sort_dir=sort_dir, paginate=True)
  File "/openstack/venvs/horizon-18.1.3/lib/python2.7/site-packages/openstack_dashboard/api/cinder.py", line 319, in volume_list_paged
    c_client = _cinderclient_with_generic_groups(request)
  File "/openstack/venvs/horizon-18.1.3/lib/python2.7/site-packages/openstack_dashboard/api/cinder.py", line 271, in _cinderclient_with_generic_groups
    version = get_microversion(request, 'groups')
  File "/openstack/venvs/horizon-18.1.3/lib/python2.7/site-packages/openstack_dashboard/api/cinder.py", line 265, in get_microversion
    min_ver, max_ver = cinder_client.get_server_version(cinder_url)
  File "/openstack/venvs/horizon-18.1.3/lib/python2.7/site-packages/cinderclient/client.py", line 109, in get_server_version
    response = requests.get(version_url)
  File "/openstack/venvs/horizon-18.1.3/lib/python2.7/site-packages/requests/api.py", line 72, in get
    return request('get', url, params=params, **kwargs)
  File "/openstack/venvs/horizon-18.1.3/lib/python2.7/site-packages/requests/api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "/openstack/venvs/horizon-18.1.3/lib/python2.7/site-packages/requests/sessions.py", line 512, in request
    resp = self.send(prep, **send_kwargs)
  File "/openstack/venvs/horizon-18.1.3/lib/python2.7/site-packages/requests/sessions.py", line 622, in send
    r = adapter.send(request, **kwargs)
  File "/openstack/venvs/horizon-18.1.3/lib/python2.7/site-packages/requests/adapters.py", line 511, in send
    raise SSLError(e, request=request)
SSLError: HTTPSConnectionPool(host='172.29.236.100', port=8776): Max retries exceeded with url: / (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",),))
=====

It seems like the fix would be to add an "insecure" parameter to cinderclient's get_server_version method and then use that parameter from Horizon's "get_microversion" method for the Cinder API.

[David Moreau Simard (dmsimard)]

There is a similar issue in Nova because it has it's own implementation of get_server_version: https://github.com/openstack/nova/blob/78d6aca9a6cfb25ac180e28dc519fb76d22a2314/nova/volume/cinder.py#L104-L146

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/643405

[OpenStack Infra (hudson-openstack)]

Change abandoned by Nikita Gerasimov (<email address hidden>) on branch: master
Review: https://review.openstack.org/643405

[Marc Methot (mb-methot)]

Basically duplicate of https://bugs.launchpad.net/horizon/+bug/1744670
Should be fixed by:
- https://github.com/openstack/horizon/commit/285c51f7e1b27a786fa9684abdc489998285b4e2
- https://review.opendev.org/#/c/675894/

Version bump of cinderclient which is required:
- https://review.opendev.org/#/c/675891/

Diff:
< python-cinderclient>=4.0.1 # Apache-2.0
> python-cinderclient>=5.0.0 # Apache-2.0

Cheers,
Marc Methot