Comment 0 for bug 1716344

Revision history for this message
Christoph Fiehe (fiehe) wrote : Nova-API sometimes uses Keystone's public endpoint

I have setup a fresh HA deployment of OpenStack Pike on Ubuntu 16.04. I recognized in the logs that Nova sometimes fails during vm creation with the following exception:

2017-09-11 09:31:28.909 5604 ERROR nova.api.openstack.identity [req-6efab9e1-78f5-4e85-8247-686ff4f3568c dddfba8e02f746799a6408a523e6cd25 ed2d2efd86dd40e7a45491d8502318d3 - default default] Unable to contact keystone to verify project_id: SSLError: SSL exception connecting to https://os-cloud.materna.com:5000/v3/projects/ed2d2efd86dd40e7a45491d8502318d3: ("bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')],)",)
2017-09-11 09:31:28.909 5604 ERROR nova.api.openstack.identity Traceback (most recent call last):
2017-09-11 09:31:28.909 5604 ERROR nova.api.openstack.identity File "/usr/lib/python2.7/dist-packages/nova/api/openstack/identity.py", line 42, in verify_project_id
2017-09-11 09:31:28.909 5604 ERROR nova.api.openstack.identity raise_exc=False)
2017-09-11 09:31:28.909 5604 ERROR nova.api.openstack.identity File "/usr/lib/python2.7/dist-packages/keystoneauth1/session.py", line 845, in get
2017-09-11 09:31:28.909 5604 ERROR nova.api.openstack.identity return self.request(url, 'GET', **kwargs)
2017-09-11 09:31:28.909 5604 ERROR nova.api.openstack.identity File "/usr/lib/python2.7/dist-packages/positional/__init__.py", line 101, in inner
2017-09-11 09:31:28.909 5604 ERROR nova.api.openstack.identity return wrapped(*args, **kwargs)
2017-09-11 09:31:28.909 5604 ERROR nova.api.openstack.identity File "/usr/lib/python2.7/dist-packages/keystoneauth1/session.py", line 703, in request
2017-09-11 09:31:28.909 5604 ERROR nova.api.openstack.identity resp = send(**kwargs)
2017-09-11 09:31:28.909 5604 ERROR nova.api.openstack.identity File "/usr/lib/python2.7/dist-packages/keystoneauth1/session.py", line 765, in _send_request
2017-09-11 09:31:28.909 5604 ERROR nova.api.openstack.identity raise exceptions.SSLError(msg)
2017-09-11 09:31:28.909 5604 ERROR nova.api.openstack.identity SSLError: SSL exception connecting to https://os-cloud.materna.com:5000/v3/projects/ed2d2efd86dd40e7a45491d8502318d3: ("bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')],)",)
2017-09-11 09:31:28.909 5604 ERROR nova.api.openstack.identity

Keystone's public endpoint should only visible to external clients. All internal OpenStack services should use the internalURL for authentication purposes. I think my configuration is correct. The "auth_url" point to Keystone's internal URL, whereas "auth_uri" points to Keystone's public endpoint. The strange thing is, that sometimes after a service restart, Nova uses the Keystone's internal URL and sometimes the Keystone's public URL. I want to avoid https based communication for the internal cloud services.

$ openstack endpoint list | grep keystone
| 00a22bfee72141ddadacd0e357161075 | RegionOne | keystone | identity | True | internal | http://os-identity.materna.com:5000/v3 |
| 7178e534cb4e4c5e9a67066ff3e9c433 | RegionOne | keystone | identity | True | public | https://os-cloud.materna.com:5000/v3 |
| f5ed3bba70274d7fa03f2ceaab96c3c9 | RegionOne | keystone | identity | True | admin | http://os-identity.materna.com:35357/v3 |

################
nova.conf
################
...
[keystone_authtoken]
auth_type = password
auth_uri = http://os-cloud.materna.com:5000
auth_url = http://os-identity:35357
memcached_servers = os-memcache:11211
password = novapass
project_domain_name = default
project_name = service
user_domain_name = default
username = nova
...

Using the option "insecure = True" is a workaround to avoid that Nova sometimes fails when the service uses Keystone's public https endpoint.

Can someone please have a look?