Mirantis OpenStack

User cannot retrieve Project in Horizon with Keystone v3

Bug #1503765 reported by Laurent Perrin
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Mirantis OpenStack
Invalid
High
Laurent Perrin
Mirantis OpenStack 8.0

Bug Description

Version:
MOS 7.0

When Horizon is configured to use Keystone v3 and OPENSTACK_KEYSTONE_MULTIDOMAIN_SUPPORT = True, users cannot retrieve the project list in Horizon drop-down menu. The list is empty so they cannot switch project.

In the Horizon logs /var/log/horizon/horizon.log:
2015-10-07 16:22:20,614 22196 ERROR openstack_auth.user Unable to retrieve project list.
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/openstack_auth/user.py", line 307, in authorized_tenants
    is_federated=self.is_federated)
  File "/usr/lib/python2.7/dist-packages/openstack_auth/utils.py", line 145, in wrapper
    result = func(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/openstack_auth/utils.py", line 251, in get_project_list
    projects = client.projects.list(user=kwargs.get('user_id'))
  File "/usr/lib/python2.7/dist-packages/keystoneclient/utils.py", line 318, in inner
    return func(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/keystoneclient/v3/projects.py", line 106, in list
    **kwargs)
  File "/usr/lib/python2.7/dist-packages/keystoneclient/base.py", line 73, in func
    return f(*args, **new_kwargs)
  File "/usr/lib/python2.7/dist-packages/keystoneclient/base.py", line 361, in list
    self.collection_key)
  File "/usr/lib/python2.7/dist-packages/keystoneclient/base.py", line 113, in _list
    resp, body = self.client.get(url, **kwargs)
  File "/usr/lib/python2.7/dist-packages/keystoneclient/adapter.py", line 170, in get
    return self.request(url, 'GET', **kwargs)
  File "/usr/lib/python2.7/dist-packages/keystoneclient/adapter.py", line 206, in request
    resp = super(LegacyJsonAdapter, self).request(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/keystoneclient/adapter.py", line 95, in request
    return self.session.request(url, method, **kwargs)
  File "/usr/lib/python2.7/dist-packages/keystoneclient/utils.py", line 318, in inner
    return func(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/keystoneclient/session.py", line 397, in request
    raise exceptions.from_response(resp, method, url)
NotFound: The resource could not be found. (HTTP 404)
2015-10-07 16:22:20,722 22196 ERROR openstack_auth.user Unable to retrieve project list.
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/openstack_auth/user.py", line 307, in authorized_tenants
    is_federated=self.is_federated)
  File "/usr/lib/python2.7/dist-packages/openstack_auth/utils.py", line 145, in wrapper
    result = func(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/openstack_auth/utils.py", line 251, in get_project_list
    projects = client.projects.list(user=kwargs.get('user_id'))
  File "/usr/lib/python2.7/dist-packages/keystoneclient/utils.py", line 318, in inner
    return func(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/keystoneclient/v3/projects.py", line 106, in list
    **kwargs)
  File "/usr/lib/python2.7/dist-packages/keystoneclient/base.py", line 73, in func
    return f(*args, **new_kwargs)
  File "/usr/lib/python2.7/dist-packages/keystoneclient/base.py", line 361, in list
    self.collection_key)
  File "/usr/lib/python2.7/dist-packages/keystoneclient/base.py", line 113, in _list
    resp, body = self.client.get(url, **kwargs)
  File "/usr/lib/python2.7/dist-packages/keystoneclient/adapter.py", line 170, in get
    return self.request(url, 'GET', **kwargs)
  File "/usr/lib/python2.7/dist-packages/keystoneclient/adapter.py", line 206, in request
    resp = super(LegacyJsonAdapter, self).request(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/keystoneclient/adapter.py", line 95, in request
    return self.session.request(url, method, **kwargs)
  File "/usr/lib/python2.7/dist-packages/keystoneclient/utils.py", line 318, in inner
    return func(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/keystoneclient/session.py", line 397, in request
    raise exceptions.from_response(resp, method, url)
NotFound: The resource could not be found. (HTTP 404)
2015-10-07 16:22:20,735 22196 ERROR openstack_auth.user Unable to retrieve project list.
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/openstack_auth/user.py", line 307, in authorized_tenants
    is_federated=self.is_federated)
  File "/usr/lib/python2.7/dist-packages/openstack_auth/utils.py", line 145, in wrapper
    result = func(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/openstack_auth/utils.py", line 251, in get_project_list
    projects = client.projects.list(user=kwargs.get('user_id'))
  File "/usr/lib/python2.7/dist-packages/keystoneclient/utils.py", line 318, in inner
    return func(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/keystoneclient/v3/projects.py", line 106, in list
    **kwargs)
  File "/usr/lib/python2.7/dist-packages/keystoneclient/base.py", line 73, in func
    return f(*args, **new_kwargs)
  File "/usr/lib/python2.7/dist-packages/keystoneclient/base.py", line 361, in list
    self.collection_key)
  File "/usr/lib/python2.7/dist-packages/keystoneclient/base.py", line 113, in _list
    resp, body = self.client.get(url, **kwargs)
  File "/usr/lib/python2.7/dist-packages/keystoneclient/adapter.py", line 170, in get
    return self.request(url, 'GET', **kwargs)
  File "/usr/lib/python2.7/dist-packages/keystoneclient/adapter.py", line 206, in request
    resp = super(LegacyJsonAdapter, self).request(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/keystoneclient/adapter.py", line 95, in request
    return self.session.request(url, method, **kwargs)
  File "/usr/lib/python2.7/dist-packages/keystoneclient/utils.py", line 318, in inner
    return func(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/keystoneclient/session.py", line 397, in request
    raise exceptions.from_response(resp, method, url)
NotFound: The resource could not be found. (HTTP 404)

And same 404 error in the keystone logs /var/log/apache2/keystone_wsgi_main_access.log :
192.168.174.194 - - [07/Oct/2015:16:22:20 +0000] "GET /users/4c7bc1537202418fbffc2fe3d6de5846/projects HTTP/1.1" 404 93 947 "-" "python-keystoneclient"
192.168.174.194 - - [07/Oct/2015:16:22:20 +0000] "GET /users/4c7bc1537202418fbffc2fe3d6de5846/projects HTTP/1.1" 404 93 970 "-" "python-keystoneclient"
192.168.174.194 - - [07/Oct/2015:16:22:20 +0000] "GET /users/4c7bc1537202418fbffc2fe3d6de5846/projects HTTP/1.1" 404 93 928 "-" "python-keystoneclient"

Tags: horizon
Roman Podoliaka (rpodolyaka)
Changed in mos:
milestone: none → 8.0
assignee: nobody → MOS Horizon (mos-horizon)
importance: Undecided → Medium
status: New → Confirmed
Timur Sufiev (tsufiev-x)
Changed in mos:
importance: Medium → High
Revision history for this message
Timur Sufiev (tsufiev-x) wrote :

Laurent, I'm trying to reproduce the bug. Can you confirm that you both had OPENSTACK_API_VERSIONS['identity'] set to 3 and OPENSTACK_KEYSTONE_URL with '/v3' suffix? I had the similar issue when I just added '/v3' suffix, but didn't specified right OPENSTACK_API_VERSIONS['identity'].

Changed in mos:
assignee: MOS Horizon (mos-horizon) → Timur Sufiev (tsufiev-x)
Revision history for this message
Laurent Perrin (perrin-laurent) wrote :
Download full text (3.2 KiB)

Yes the OPENSTACK_API_VERSIONS['identity'] was to 3 and OPENSTACK_KEYSTONE_URL with '/v3' suffix.

But I ran some new tests.

To enable both keystone v2 and v3, I updated the keystone endpoints in the SQL database, using this configuration:
internal | http://192.168.174.196:5000
public | https://public.fuel.local:5000
admin | http://192.168.174.196:35357

With this configuration, the projects are not listed with Horizon configured for v2 or v3 (same 404 error):
Horizon Keystone v2 / generic endpoint :
192.168.174.196 - - [13/Oct/2015:08:55:10 +0000] "GET /v2.0/tenants HTTP/1.1" 200 238 21618 "-" "python-keystoneclient"
192.168.174.196 - - [13/Oct/2015:08:55:10 +0000] "POST /v2.0/tokens HTTP/1.1" 200 4851 24458 "-" "python-keystoneclient"
192.168.174.196 - - [13/Oct/2015:08:55:12 +0000] "GET /v3/auth/tokens HTTP/1.1" 200 8489 77744 "-" "python-keystoneclient"
192.168.174.196 - - [13/Oct/2015:08:55:12 +0000] "GET /tenants HTTP/1.1" 404 93 950 "-" "python-keystoneclient"
192.168.174.196 - - [13/Oct/2015:08:55:12 +0000] "GET /tenants HTTP/1.1" 404 93 1425 "-" "python-keystoneclient"
192.168.174.196 - - [13/Oct/2015:08:55:12 +0000] "GET /tenants HTTP/1.1" 404 93 945 "-" "python-keystoneclient"

Horizon Keystone v3 / generic endpoint :
192.168.174.196 - - [13/Oct/2015:10:22:04 +0000] "GET /v3/users/f466e48b3e0d46feb916349802888144/projects HTTP/1.1" 200 636 638133 "-" "python-keystoneclient"
192.168.174.196 - - [13/Oct/2015:10:22:05 +0000] "POST /v3/auth/tokens HTTP/1.1" 201 8491 679145 "-" "python-keystoneclient"
192.168.174.196 - - [13/Oct/2015:10:22:10 +0000] "GET /v3/auth/tokens HTTP/1.1" 200 8491 631669 "-" "python-keystoneclient"
192.168.174.196 - - [13/Oct/2015:10:22:11 +0000] "GET /users/f466e48b3e0d46feb916349802888144/projects HTTP/1.1" 404 93 1132 "-" "python-keystoneclient"
192.168.174.196 - - [13/Oct/2015:10:22:11 +0000] "GET /users/f466e48b3e0d46feb916349802888144/projects HTTP/1.1" 404 93 1103 "-" "python-keystoneclient"
192.168.174.196 - - [13/Oct/2015:10:22:11 +0000] "GET /users/f466e48b3e0d46feb916349802888144/projects HTTP/1.1" 404 93 1215 "-" "python-keystoneclient"

But if I configure the Keystone endpoint for only v3 like this:
internal | http://192.168.174.196:5000/v3
public | https://public.fuel.local:5000/v3
admin | http://192.168.174.196:35357/v3

The projects are listed correctly:
Horizon Keystone v3 / Endpoint v3
192.168.174.196 - - [13/Oct/2015:09:41:40 +0000] "OPTIONS / HTTP/1.0" 300 605 1642 "-" "-"
192.168.174.196 - - [13/Oct/2015:09:41:50 +0000] "POST /v3/auth/tokens HTTP/1.1" 201 297 34867 "-" "python-keystoneclient"
192.168.174.196 - - [13/Oct/2015:09:41:50 +0000] "OPTIONS / HTTP/1.0" 300 605 1373 "-" "-"
192.168.174.196 - - [13/Oct/2015:09:41:50 +0000] "GET /v3/users/f466e48b3e0d46feb916349802888144/projects HTTP/1.1" 200 636 22771 "-" "python-keystoneclient"
192.168.174.196 - - [13/Oct/2015:09:41:50 +0000] "POST /v3/auth/tokens HTTP/1.1" 201 8500 28813 "-" "python-keystoneclient"
192.168.174.196 - - [13/Oct/2015:09:41:51 +0000] "GET /v3/auth/tokens HTTP/1.1" 200 8500 8970 "-" "python-keystoneclient"
192.168.174.196 - - [13/Oct/2015:09:41:51 +0000] "GET /v3/users/f466e48b3e0d46feb916349802888...

Read more...

Revision history for this message
Timur Sufiev (tsufiev-x) wrote :

Waiting for additional info, Incomplete until then.

Changed in mos:
status: Confirmed → Incomplete
assignee: Timur Sufiev (tsufiev-x) → Laurent Perrin (perrin-laurent)
status: Incomplete → Confirmed
assignee: Laurent Perrin (perrin-laurent) → Timur Sufiev (tsufiev-x)
Revision history for this message
Timur Sufiev (tsufiev-x) wrote :

Seems we updated the bug simultaneously. Looking through the new update.

Revision history for this message
Timur Sufiev (tsufiev-x) wrote :

Laurent, please also provide your initial Horizon configuration (OPENSTACK_API_VERSIONS['identity'] and OPENSTACK_KEYSTONE_URL) as well initial Keystone endpoints when you first discovered the bug.

Revision history for this message
Laurent Perrin (perrin-laurent) wrote :

When I discovered the bug, the horizon configuration was this:
OPENSTACK_API_VERSIONS = {
     "identity": 3
}
OPENSTACK_KEYSTONE_URL = "http://192.168.174.196:5000/v3"

The endpoints were these ones:
internal | http://192.168.174.196:5000
public | https://public.fuel.local:5000
admin | http://192.168.174.196:35357

Revision history for this message
Timur Sufiev (tsufiev-x) wrote :

Laurent, I've just deployed fresh environment from Mirantis Openstack 7.0 release ISO, here is its output for Keystone endpoints:

root@node-1:~# keystone endpoint-list | grep 5000
| b7c12ffa3bc846e7ad57bcb0a539e182 | RegionOne | https://public.fuel.local:5000/v2.0 | http://192.168.0.2:5000/v2.0 | http://192.168.0.2:35357/v2.0 | 7740c2ec1b384d3488d1f9df7e7e0ac3 |

It seems your MOS is a bit customized.

Also, I changed Horizon config to have these parameters:
OPENSTACK_KEYSTONE_MULTIDOMAIN_SUPPORT = True
OPENSTACK_API_VERSIONS['identity'] = 3 and OPENSTACK_KEYSTONE_URL = "192.168.0.2:5000/v3" and the project switcher worked fine for me (see the screenshot below).

Changing status again to Incomplete until more steps to reproduce are known.

Revision history for this message
Timur Sufiev (tsufiev-x) wrote :
Changed in mos:
status: Confirmed → Incomplete
assignee: Timur Sufiev (tsufiev-x) → Laurent Perrin (perrin-laurent)
Revision history for this message
Laurent Perrin (perrin-laurent) wrote :

Timur, I confirm that with standard endpoints, it works.

From what I have read, it was necessary to expose a v3 keystone endpoint to use the keystone v3 api.

Sorry for this wrong report.

Timur Sufiev (tsufiev-x)
tags: added: horizon
Roman Podoliaka (rpodolyaka)
Changed in mos:
status: Incomplete → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.