insecure=True not documented outside of keystoneclient.middleware.auth_token

[Bhuvan Arumugam]

We use self signed certificate with all openstack services. It all worked so far, but break once keystoneclient v0.4.0 was released last week.

As per this commit, keystoneclient by default use insecure=False.
https://github.com/openstack/python-keystoneclient/commit/20e166fd8a943ee3f91ba362a47e9c14c7cc5f4c

This break self-signed instances. The openstack components {nova, glance, neutron} are unable to communicate with keystone. We don't use horion or swift. I presume they are broken as well. The keystone client is happy though if we use --insecure flag, while using it directly.

Ideally, we should introduce new config parameter keystone_api_insecure. The insecure flag in keystoneclient should be defined based on this parameter. This should be fixed in all openstack services, nova, glance & neutron.

[barumugam@build tempest]$ keystone --insecure tenant-list
+----------------------------+----------------------------+---------+
| id | name | enabled |
+----------------------------+----------------------------+---------+
| csi-tenant-tempest | csi-tenant-tempest | True |
+----------------------------+----------------------------+---------+

[barumugam@build tempest]$ nova --insecure list
ERROR: Unauthorized (HTTP 401)

Nova log:

2013-10-13 00:01:56,680 (keystoneclient.middleware.auth_token): ERROR auth_token _http_request HTTP connection exception: [Errno 1] _ssl.c:490: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
2013-10-13 00:01:56,682 (keystoneclient.middleware.auth_token): DEBUG auth_token _validate_user_token Token validation failure.
Traceback (most recent call last):
  File "/usr/local/csi/share/csi-nova.venv/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py", line 808, in _validate_user_token
    verified = self.verify_signed_token(user_token)
  File "/usr/local/csi/share/csi-nova.venv/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py", line 1165, in verify_signed_token
    if self.is_signed_token_revoked(signed_text):
  File "/usr/local/csi/share/csi-nova.venv/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py", line 1127, in is_signed_token_revoked
    revocation_list = self.token_revocation_list
  File "/usr/local/csi/share/csi-nova.venv/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py", line 1217, in token_revocation_list
    self.token_revocation_list = self.fetch_revocation_list()
  File "/usr/local/csi/share/csi-nova.venv/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py", line 1235, in fetch_revocation_list
    additional_headers=headers)
  File "/usr/local/csi/share/csi-nova.venv/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py", line 739, in _json_request
    response = self._http_request(method, path, **kwargs)
  File "/usr/local/csi/share/csi-nova.venv/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py", line 704, in _http_request
    raise NetworkError('Unable to communicate with keystone')

[Jamie Lennox (jamielennox)]

Mailing List Thread: http://lists.openstack.org/pipermail/openstack-dev/2013-October/016627.html

[Dolph Mathews (dolph)]

Can you share you auth_token configuration?

[Dolph Mathews (dolph)]

This doesn't appear to be relevant to nova itself

[Davanum Srinivas (DIMS) (dims-v)]

@Bhuvan,

Did you try adding insecure=true flag in /etc/nova/api-paste.ini? in the "[filter:authtoken]" section?

[Bhuvan Arumugam (bhuvan)]

@dims-v, that is it.
I didn't set [filter:authtoken].insecure=True in /etc/nova/api-paste.ini. With this flag, openstack services could communicate with self-signed keystone instance.

Since this is a change in behavior with keystoneclient, we should document it somewhere here:
http://docs.openstack.org/developer/python-keystoneclient/using-api.html

... something in these lines:

If you use keystone with self signed certificate, the openstack services wouldn't be able to verify the certificate. To disable certificate verification, you should define insecure=True in [filter:authtoken] section in /etc/nova/api-paste.ini file. The flag should be defined for other components as well.

[Dolph Mathews (dolph)]

Agree that this is a doc bug, although keystoneclient doesn't currently document usage for auth_token at all, beyond help attributes via oslo.config / argparse (in this case... `cfg.BoolOpt('insecure', default=False, help='Verify HTTPS connections.')` ). openstack-manuals does have coverage on this topic though.

[Jamie Lennox (jamielennox)]

keystoneclient does hae some documentation for auth_token. It is in the python-keystoneclient/doc/source/middlewarearchitecture.rst document, but i'm not sure where this gets published to.

[Dolph Mathews (dolph)]

Jamie: see bug 1240202. Added keystone to this bug so that the canonical version can be revised-

  https://github.com/openstack/keystone/blob/master/doc/source/middlewarearchitecture.rst#configuration-options

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/63541

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/63541
Committed: https://git.openstack.org/cgit/openstack/openstack-manuals/commit/?id=fda2f0459a5d9d48d40d607ba447db1204458e75
Submitter: Jenkins
Branch: master

commit fda2f0459a5d9d48d40d607ba447db1204458e75
Author: Chandan Kumar <email address hidden>
Date: Sat Dec 21 04:08:41 2013 +0530

    Added docs for disabling SSL certificates in openstack services

    Closes-Bug:#1239894

    Change-Id: If3ad6cb364dcccd224db1da738b7f189856b465f
    backport:none

[Steve Martinelli (stevemar)]

refer to http://docs.openstack.org/developer/keystonemiddleware/middlewarearchitecture.html, we no longer have a version of this in keystone server docs

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/openstack-manuals 15.0.0 release.