Contrail Ocata :: R4.0.1.0 build 30 :: keystone v3 :: horizon fails to allow user to login to newly created domain.

[Ritam Gangopadhyay]

Setup:- R4.0.1.0 build 29 Ocata multi node setup.

Horizon accessible on VIP - 10.204.217.184

nodei19 10.204.217.131 openstack
nodec28 10.204.217.13 controller, analytics, analyticsdb
nodec10 10.204.217.176 controller, analytics, analyticsdb
nodec33 10.204.217.168 controller, analytics, analyticsdb
nodeg37 10.204.217.77 lb
nodei17 10.204.217.129 compute
nodei20 10.204.217.132 compute

******************************************************
******************************************************
******************************************************
******************************************************

Steps:-

1. create new domain
2. switch/set new domain context
3. create user under new domain
4. create project under new domain
5. add user to project with admin and member role
6. add user to domain with admin and member role
7. log out of horizon.
8. log in to horizon with new domain and new user

******************************************************
******************************************************
******************************************************
******************************************************

ERROR LOG:-

/var/lib/docker/volumes/kolla_logs/_data/horizon/horizon.log

[Fri Sep 15 13:26:36.313184 2017] [wsgi:error] [pid 143:tid 140074279245568] Login successful for user "admin", remote address 192.168.100.15.
[Fri Sep 15 13:27:27.437038 2017] [wsgi:error] [pid 144:tid 140074279245568] Creating domain with name "test-dm1"
[Fri Sep 15 13:27:51.715980 2017] [wsgi:error] [pid 145:tid 140074279245568] Creating user with name "u1"
[Fri Sep 15 13:28:23.782572 2017] [wsgi:error] [pid 142:tid 140074279245568] Updating domain with name "test-dm1"
[Fri Sep 15 13:28:27.842695 2017] [wsgi:error] [pid 145:tid 140074279245568] Logging out user "admin".
[Fri Sep 15 13:28:38.781263 2017] [wsgi:error] [pid 144:tid 140074279245568] Login successful for user "u1", remote address 192.168.100.15.
[Fri Sep 15 13:28:39.230834 2017] [wsgi:error] [pid 144:tid 140074279245568] Pure project admin doesn't have a domain token
[Fri Sep 15 13:28:39.231571 2017] [wsgi:error] [pid 144:tid 140074279245568] Internal Server Error: /identity/
[Fri Sep 15 13:28:39.231589 2017] [wsgi:error] [pid 144:tid 140074279245568] Traceback (most recent call last):
[Fri Sep 15 13:28:39.231595 2017] [wsgi:error] [pid 144:tid 140074279245568] File "/usr/lib/python2.7/dist-packages/django/core/handlers/base.py", line 132, in get_response
[Fri Sep 15 13:28:39.231600 2017] [wsgi:error] [pid 144:tid 140074279245568] response = wrapped_callback(request, *callback_args, **callback_kwargs)
[Fri Sep 15 13:28:39.231604 2017] [wsgi:error] [pid 144:tid 140074279245568] File "/usr/share/openstack-dashboard/horizon/decorators.py", line 36, in dec
[Fri Sep 15 13:28:39.231608 2017] [wsgi:error] [pid 144:tid 140074279245568] return view_func(request, *args, **kwargs)
[Fri Sep 15 13:28:39.231612 2017] [wsgi:error] [pid 144:tid 140074279245568] File "/usr/share/openstack-dashboard/horizon/decorators.py", line 52, in dec
[Fri Sep 15 13:28:39.231616 2017] [wsgi:error] [pid 144:tid 140074279245568] return view_func(request, *args, **kwargs)
[Fri Sep 15 13:28:39.231620 2017] [wsgi:error] [pid 144:tid 140074279245568] File "/usr/share/openstack-dashboard/horizon/decorators.py", line 36, in dec
[Fri Sep 15 13:28:39.231624 2017] [wsgi:error] [pid 144:tid 140074279245568] return view_func(request, *args, **kwargs)
[Fri Sep 15 13:28:39.231628 2017] [wsgi:error] [pid 144:tid 140074279245568] File "/usr/lib/python2.7/dist-packages/django/views/generic/base.py", line 71, in view
[Fri Sep 15 13:28:39.231632 2017] [wsgi:error] [pid 144:tid 140074279245568] return self.dispatch(request, *args, **kwargs)
[Fri Sep 15 13:28:39.231636 2017] [wsgi:error] [pid 144:tid 140074279245568] File "/usr/lib/python2.7/dist-packages/django/views/generic/base.py", line 89, in dispatch
[Fri Sep 15 13:28:39.231640 2017] [wsgi:error] [pid 144:tid 140074279245568] return handler(request, *args, **kwargs)
[Fri Sep 15 13:28:39.231643 2017] [wsgi:error] [pid 144:tid 140074279245568] File "/usr/share/openstack-dashboard/horizon/tables/views.py", line 219, in get
[Fri Sep 15 13:28:39.231647 2017] [wsgi:error] [pid 144:tid 140074279245568] handled = self.construct_tables()
[Fri Sep 15 13:28:39.231651 2017] [wsgi:error] [pid 144:tid 140074279245568] File "/usr/share/openstack-dashboard/horizon/tables/views.py", line 210, in construct_tables
[Fri Sep 15 13:28:39.231655 2017] [wsgi:error] [pid 144:tid 140074279245568] handled = self.handle_table(table)
[Fri Sep 15 13:28:39.231658 2017] [wsgi:error] [pid 144:tid 140074279245568] File "/usr/share/openstack-dashboard/horizon/tables/views.py", line 123, in handle_table
[Fri Sep 15 13:28:39.231662 2017] [wsgi:error] [pid 144:tid 140074279245568] data = self._get_data_dict()
[Fri Sep 15 13:28:39.231666 2017] [wsgi:error] [pid 144:tid 140074279245568] File "/usr/share/openstack-dashboard/horizon/tables/views.py", line 248, in _get_data_dict
[Fri Sep 15 13:28:39.231670 2017] [wsgi:error] [pid 144:tid 140074279245568] self._data = {self.table_class._meta.name: self.get_data()}
[Fri Sep 15 13:28:39.231690 2017] [wsgi:error] [pid 144:tid 140074279245568] File "/usr/share/openstack-dashboard/openstack_dashboard/dashboards/identity/projects/views.py", line 135, in get_data
[Fri Sep 15 13:28:39.231697 2017] [wsgi:error] [pid 144:tid 140074279245568] t.domain_name = domain_lookup.get(t.domain_id)
[Fri Sep 15 13:28:39.231701 2017] [wsgi:error] [pid 144:tid 140074279245568] AttributeError: 'NoneType' object has no attribute 'get'

[Ritam Gangopadhyay (ritam)]

This is only seen from horizon UI and not from contrail AND/OR openstack api. So downgrading the priority.

[Jeba Paulaiyan (jebap)]

As per Ritam this might not be an issue with build #32. Ritam to verify and close.

[Ritam Gangopadhyay (ritam)]

Recreated with R4.0.1.0 build 32 ocata. Setup is in error state and locked.

Horizon login details for the issue - 10.204.217.184 :-

Domain - ritamd
User - ritam
Password - ritam

Setup:-

nodei19 10.204.217.131 openstack
nodec28 10.204.217.13 controller, analytics, analyticsdb
nodec10 10.204.217.176 controller, analytics, analyticsdb
nodec33 10.204.217.168 controller, analytics, analyticsdb
nodeg37 10.204.217.77 lb
nodei17 10.204.217.129 compute
nodei20 10.204.217.132 compute

[Manoj (manojgn)]

After debugging we found that the issue is not with horizon but with keystone itself. It is the basic definition of cloud-admin in the keystone policy.json file.

When horizon issues call to list domains the following rules is activated
"identity:list_domains": "rule:cloud_admin",

cloud-admin has a definition as follows in the policy.json file
"cloud_admin": "role:admin and (token.is_admin_project:True or domain_id:default)",

is_admin_project is returned as false and hence keystone rejects the access and throws and error "You are not authorized to perform the requested action: identity:list_domains" and horizon logs out.

[Manoj (manojgn)]

From: Biswajit Mandal <email address hidden>
Date: Sunday, October 8, 2017 at 10:31 AM
To: Ritam Gangopadhyay <email address hidden>, Rudra Rugge <email address hidden>, Anish Mehta <email address hidden>
Cc: Sudheendra Rao <email address hidden>, Naga Kiran K Y S <email address hidden>, Manoj Naik <email address hidden>
Subject: Re: 4.0.2 release next Friday (Fixes due by Tuesday)

The issue is as Manoj Specified, while login list_domains is failing due to cloud_admin role is not matching with the rule, is_admin_project: True is failing.
The cloud_admin rule is defined as:

"role:admin and (token.is_admin_project:True or domain_id:admin_domain_id)"

As per the release notes and bug fixes as described below:
https://docs.openstack.org/releasenotes/keystone/ocata.html

There was a typo in policy.json as it was fixed in https://bugs.launchpad.net/keystone/+bug/1547684, we tried policy.json changing as specified in this bug fix, but still the issue did not solve.

I see that as part of https://bugs.launchpad.net/keystone/+bug/1651989 there was a discussion that the fix as was done in Bug 1547684 did not sole completely, so there was one more fix to change in policy.json as part of https://bugs.launchpad.net/keystone/+bug/1651989/comments/14 which also did not solve as our case. And also there was revert/commit of the same CL as part of some other bug, so there may be some issue exists in ocata.

So, I think for now to unblock it, can we have cloud_admin role as defined in mitaka case?
"cloud_admin": "rule:admin_required or is_admin:1"

With Regards,
Biswajit

[Manoj (manojgn)]

On further analysis it is found that in horizon as well there is a keystone_policy.json file defined in openstack_dashboard/settings.py which needs to match the policy.json in keystone.

So copying the copying the policy.json to keystone_policy.json and restarting horizon and keystone should fix this issue.

[Ritam Gangopadhyay (ritam)]

Yes, doing this fixes the issue, verified.