horizon can`t login when enable tls report could not find cert file

Bug #1867121 reported by yj.bai
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
kolla-ansible
Fix Released
Medium
yj.bai
Ussuri
Fix Released
Medium
yj.bai

Bug Description

[Mon Mar 09 13:46:51.594567 2020] [:error] [pid 56] DEBUG:keystoneauth.identity.v3.base:Making authentication request to https://overcloud.internal.vim1.local:5000/v3/auth/tokens
[Mon Mar 09 13:46:51.633357 2020] [:error] [pid 56] Internal Server Error: /auth/login/
[Mon Mar 09 13:46:51.633403 2020] [:error] [pid 56] Traceback (most recent call last):
[Mon Mar 09 13:46:51.633409 2020] [:error] [pid 56] File "/var/lib/kolla/venv/lib/python2.7/site-packages/django/core/handlers/exception.py", line 41, in inner
[Mon Mar 09 13:46:51.633415 2020] [:error] [pid 56] response = get_response(request)
[Mon Mar 09 13:46:51.633420 2020] [:error] [pid 56] File "/var/lib/kolla/venv/lib/python2.7/site-packages/django/core/handlers/base.py", line 249, in _legacy_get_response
[Mon Mar 09 13:46:51.633426 2020] [:error] [pid 56] response = self._get_response(request)
[Mon Mar 09 13:46:51.633431 2020] [:error] [pid 56] File "/var/lib/kolla/venv/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, in _get_response
[Mon Mar 09 13:46:51.633436 2020] [:error] [pid 56] response = self.process_exception_by_middleware(e, request)
[Mon Mar 09 13:46:51.633441 2020] [:error] [pid 56] File "/var/lib/kolla/venv/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, in _get_response
[Mon Mar 09 13:46:51.633447 2020] [:error] [pid 56] response = wrapped_callback(request, *callback_args, **callback_kwargs)
[Mon Mar 09 13:46:51.633452 2020] [:error] [pid 56] File "/var/lib/kolla/venv/lib/python2.7/site-packages/django/views/decorators/debug.py", line 76, in sensitive_post_parameters_wrapper
[Mon Mar 09 13:46:51.633456 2020] [:error] [pid 56] return view(request, *args, **kwargs)
[Mon Mar 09 13:46:51.633461 2020] [:error] [pid 56] File "/var/lib/kolla/venv/lib/python2.7/site-packages/django/utils/decorators.py", line 149, in _wrapped_view
[Mon Mar 09 13:46:51.633465 2020] [:error] [pid 56] response = view_func(request, *args, **kwargs)
[Mon Mar 09 13:46:51.633470 2020] [:error] [pid 56] File "/var/lib/kolla/venv/lib/python2.7/site-packages/django/views/decorators/cache.py", line 57, in _wrapped_view_func
[Mon Mar 09 13:46:51.633475 2020] [:error] [pid 56] response = view_func(request, *args, **kwargs)
[Mon Mar 09 13:46:51.633479 2020] [:error] [pid 56] File "/var/lib/kolla/venv/lib/python2.7/site-packages/openstack_auth/views.py", line 106, in login
[Mon Mar 09 13:46:51.633484 2020] [:error] [pid 56] **kwargs)
[Mon Mar 09 13:46:51.633488 2020] [:error] [pid 56] File "/var/lib/kolla/venv/lib/python2.7/site-packages/django/contrib/auth/views.py", line 54, in inner
[Mon Mar 09 13:46:51.633493 2020] [:error] [pid 56] return func(*args, **kwargs)
[Mon Mar 09 13:46:51.633498 2020] [:error] [pid 56] File "/var/lib/kolla/venv/lib/python2.7/site-packages/django/contrib/auth/views.py", line 150, in login
[Mon Mar 09 13:46:51.633503 2020] [:error] [pid 56] )(request)
[Mon Mar 09 13:46:51.633507 2020] [:error] [pid 56] File "/var/lib/kolla/venv/lib/python2.7/site-packages/django/views/generic/base.py", line 68, in view
[Mon Mar 09 13:46:51.633512 2020] [:error] [pid 56] return self.dispatch(request, *args, **kwargs)
[Mon Mar 09 13:46:51.633516 2020] [:error] [pid 56] File "/var/lib/kolla/venv/lib/python2.7/site-packages/django/utils/decorators.py", line 67, in _wrapper
[Mon Mar 09 13:46:51.633521 2020] [:error] [pid 56] return bound_func(*args, **kwargs)
[Mon Mar 09 13:46:51.633526 2020] [:error] [pid 56] File "/var/lib/kolla/venv/lib/python2.7/site-packages/django/views/decorators/debug.py", line 76, in sensitive_post_parameters_wrapper
[Mon Mar 09 13:46:51.633530 2020] [:error] [pid 56] return view(request, *args, **kwargs)
[Mon Mar 09 13:46:51.633537 2020] [:error] [pid 56] File "/var/lib/kolla/venv/lib/python2.7/site-packages/django/utils/decorators.py", line 63, in bound_func
[Mon Mar 09 13:46:51.633556 2020] [:error] [pid 56] return func.__get__(self, type(self))(*args2, **kwargs2)
[Mon Mar 09 13:46:51.633563 2020] [:error] [pid 56] File "/var/lib/kolla/venv/lib/python2.7/site-packages/django/utils/decorators.py", line 67, in _wrapper
[Mon Mar 09 13:46:51.633580 2020] [:error] [pid 56] return bound_func(*args, **kwargs)
[Mon Mar 09 13:46:51.633586 2020] [:error] [pid 56] File "/var/lib/kolla/venv/lib/python2.7/site-packages/django/utils/decorators.py", line 149, in _wrapped_view
[Mon Mar 09 13:46:51.633593 2020] [:error] [pid 56] response = view_func(request, *args, **kwargs)
[Mon Mar 09 13:46:51.633600 2020] [:error] [pid 56] File "/var/lib/kolla/venv/lib/python2.7/site-packages/django/utils/decorators.py", line 63, in bound_func
[Mon Mar 09 13:46:51.633606 2020] [:error] [pid 56] return func.__get__(self, type(self))(*args2, **kwargs2)
[Mon Mar 09 13:46:51.633611 2020] [:error] [pid 56] File "/var/lib/kolla/venv/lib/python2.7/site-packages/django/utils/decorators.py", line 67, in _wrapper
[Mon Mar 09 13:46:51.633617 2020] [:error] [pid 56] return bound_func(*args, **kwargs)
[Mon Mar 09 13:46:51.633624 2020] [:error] [pid 56] File "/var/lib/kolla/venv/lib/python2.7/site-packages/django/views/decorators/cache.py", line 57, in _wrapped_view_func
[Mon Mar 09 13:46:51.633631 2020] [:error] [pid 56] response = view_func(request, *args, **kwargs)
[Mon Mar 09 13:46:51.633638 2020] [:error] [pid 56] File "/var/lib/kolla/venv/lib/python2.7/site-packages/django/utils/decorators.py", line 63, in bound_func
[Mon Mar 09 13:46:51.633644 2020] [:error] [pid 56] return func.__get__(self, type(self))(*args2, **kwargs2)
[Mon Mar 09 13:46:51.633651 2020] [:error] [pid 56] File "/var/lib/kolla/venv/lib/python2.7/site-packages/django/contrib/auth/views.py", line 90, in dispatch
[Mon Mar 09 13:46:51.633657 2020] [:error] [pid 56] return super(LoginView, self).dispatch(request, *args, **kwargs)
[Mon Mar 09 13:46:51.633663 2020] [:error] [pid 56] File "/var/lib/kolla/venv/lib/python2.7/site-packages/django/views/generic/base.py", line 88, in dispatch
[Mon Mar 09 13:46:51.633668 2020] [:error] [pid 56] return handler(request, *args, **kwargs)
[Mon Mar 09 13:46:51.633673 2020] [:error] [pid 56] File "/var/lib/kolla/venv/lib/python2.7/site-packages/django/views/generic/edit.py", line 182, in post
[Mon Mar 09 13:46:51.633679 2020] [:error] [pid 56] if form.is_valid():
[Mon Mar 09 13:46:51.633708 2020] [:error] [pid 56] File "/var/lib/kolla/venv/lib/python2.7/site-packages/django/forms/forms.py", line 183, in is_valid
[Mon Mar 09 13:46:51.633716 2020] [:error] [pid 56] return self.is_bound and not self.errors
[Mon Mar 09 13:46:51.633720 2020] [:error] [pid 56] File "/var/lib/kolla/venv/lib/python2.7/site-packages/django/forms/forms.py", line 175, in errors
[Mon Mar 09 13:46:51.633724 2020] [:error] [pid 56] self.full_clean()
[Mon Mar 09 13:46:51.633728 2020] [:error] [pid 56] File "/var/lib/kolla/venv/lib/python2.7/site-packages/django/forms/forms.py", line 385, in full_clean
[Mon Mar 09 13:46:51.633732 2020] [:error] [pid 56] self._clean_form()
[Mon Mar 09 13:46:51.633736 2020] [:error] [pid 56] File "/var/lib/kolla/venv/lib/python2.7/site-packages/django/forms/forms.py", line 412, in _clean_form
[Mon Mar 09 13:46:51.633740 2020] [:error] [pid 56] cleaned_data = self.clean()
[Mon Mar 09 13:46:51.633880 2020] [:error] [pid 56] File "/var/lib/kolla/venv/lib/python2.7/site-packages/django/views/decorators/debug.py", line 36, in sensitive_variables_wrapper
[Mon Mar 09 13:46:51.633886 2020] [:error] [pid 56] return func(*func_args, **func_kwargs)
[Mon Mar 09 13:46:51.633890 2020] [:error] [pid 56] File "/var/lib/kolla/venv/lib/python2.7/site-packages/openstack_auth/forms.py", line 138, in clean
[Mon Mar 09 13:46:51.633894 2020] [:error] [pid 56] auth_url=region)
[Mon Mar 09 13:46:51.633898 2020] [:error] [pid 56] File "/var/lib/kolla/venv/lib/python2.7/site-packages/django/contrib/auth/__init__.py", line 70, in authenticate
[Mon Mar 09 13:46:51.633903 2020] [:error] [pid 56] user = _authenticate_with_backend(backend, backend_path, request, credentials)
[Mon Mar 09 13:46:51.633907 2020] [:error] [pid 56] File "/var/lib/kolla/venv/lib/python2.7/site-packages/django/contrib/auth/__init__.py", line 115, in _authenticate_with_backend
[Mon Mar 09 13:46:51.633918 2020] [:error] [pid 56] return backend.authenticate(*args, **credentials)
[Mon Mar 09 13:46:51.633922 2020] [:error] [pid 56] File "/var/lib/kolla/venv/lib/python2.7/site-packages/openstack_auth/backend.py", line 123, in authenticate
[Mon Mar 09 13:46:51.633926 2020] [:error] [pid 56] unscoped_auth_ref = plugin.get_access_info(unscoped_auth)
[Mon Mar 09 13:46:51.633930 2020] [:error] [pid 56] File "/var/lib/kolla/venv/lib/python2.7/site-packages/openstack_auth/plugin/base.py", line 126, in get_access_info
[Mon Mar 09 13:46:51.633935 2020] [:error] [pid 56] unscoped_auth_ref = keystone_auth.get_access(session)
[Mon Mar 09 13:46:51.633941 2020] [:error] [pid 56] File "/var/lib/kolla/venv/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 134, in get_access
[Mon Mar 09 13:46:51.633947 2020] [:error] [pid 56] self.auth_ref = self.get_auth_ref(session)
[Mon Mar 09 13:46:51.633954 2020] [:error] [pid 56] File "/var/lib/kolla/venv/lib/python2.7/site-packages/keystoneauth1/identity/v3/base.py", line 177, in get_auth_ref
[Mon Mar 09 13:46:51.633960 2020] [:error] [pid 56] authenticated=False, log=False, **rkwargs)
[Mon Mar 09 13:46:51.633967 2020] [:error] [pid 56] File "/var/lib/kolla/venv/lib/python2.7/site-packages/keystoneauth1/session.py", line 848, in post
[Mon Mar 09 13:46:51.633973 2020] [:error] [pid 56] return self.request(url, 'POST', **kwargs)
[Mon Mar 09 13:46:51.633979 2020] [:error] [pid 56] File "/var/lib/kolla/venv/lib/python2.7/site-packages/keystoneauth1/session.py", line 698, in request
[Mon Mar 09 13:46:51.633985 2020] [:error] [pid 56] resp = send(**kwargs)
[Mon Mar 09 13:46:51.633990 2020] [:error] [pid 56] File "/var/lib/kolla/venv/lib/python2.7/site-packages/keystoneauth1/session.py", line 756, in _send_request
[Mon Mar 09 13:46:51.633996 2020] [:error] [pid 56] resp = self.session.request(method, url, **kwargs)
[Mon Mar 09 13:46:51.634001 2020] [:error] [pid 56] File "/var/lib/kolla/venv/lib/python2.7/site-packages/requests/sessions.py", line 508, in request
[Mon Mar 09 13:46:51.634039 2020] [:error] [pid 56] resp = self.send(prep, **send_kwargs)
[Mon Mar 09 13:46:51.634046 2020] [:error] [pid 56] File "/var/lib/kolla/venv/lib/python2.7/site-packages/requests/sessions.py", line 618, in send
[Mon Mar 09 13:46:51.634051 2020] [:error] [pid 56] r = adapter.send(request, **kwargs)
[Mon Mar 09 13:46:51.634056 2020] [:error] [pid 56] File "/var/lib/kolla/venv/lib/python2.7/site-packages/requests/adapters.py", line 407, in send
[Mon Mar 09 13:46:51.634062 2020] [:error] [pid 56] self.cert_verify(conn, request.url, verify, cert)
[Mon Mar 09 13:46:51.634067 2020] [:error] [pid 56] File "/var/lib/kolla/venv/lib/python2.7/site-packages/requests/adapters.py", line 226, in cert_verify
[Mon Mar 09 13:46:51.634073 2020] [:error] [pid 56] "invalid path: {0}".format(cert_loc))
[Mon Mar 09 13:46:51.634078 2020] [:error] [pid 56] IOError: Could not find a suitable TLS CA certificate bundle, invalid path: /etc/pki/ca-trust/source/anchors/kolla-customca-haproxy-internal.crt
[Mon Mar 09 13:46:53.496161 2020] [:error] [pid 56] Not Found: /favicon.ico

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kolla-ansible (master)

Fix proposed to branch: master
Review: https://review.opendev.org/712630

Changed in kolla-ansible:
assignee: nobody → yj.bai (baiyj)
status: New → In Progress
Changed in kolla-ansible:
assignee: yj.bai (baiyj) → Radosław Piliszek (yoctozepto)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kolla-ansible (master)

Reviewed: https://review.opendev.org/712630
Committed: https://git.openstack.org/cgit/openstack/kolla-ansible/commit/?id=ed2df25ebcdd6bd1acde2facbbef763791e705c3
Submitter: Zuul
Branch: master

commit ed2df25ebcdd6bd1acde2facbbef763791e705c3
Author: yj.bai <bai.yongjun@99cloud.net>
Date: Thu Mar 12 17:39:47 2020 +0800

    Copy ca certificates also to horizon container

    Add copy ca file to horizon container.
    because:
    Could not find a suitable TLS CA certificate bundle,
    invalid path: /etc/pki/ca-trust/source/anchors/kolla-customca-haproxy-internal.crt

    Closes-Bug: #1867121

    Change-Id: I64d4dbeebd53048705005b61eb3c5b2104e8f2ed
    Signed-off-by: yj.bai <bai.yongjun@99cloud.net>

Changed in kolla-ansible:
status: In Progress → Fix Released
Mark Goddard (mgoddard)
Changed in kolla-ansible:
importance: Undecided → Medium
Revision history for this message
Mithun (mithunsunku) wrote :

Is the restart of the horizon container required after copying CA certificate ?

Revision history for this message
Mark Goddard (mgoddard) wrote :

A restart is required, but it should be performed automatically.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.