magnum unable to authorise request in Openstack queens

[jayesh chaudhari]

I have Openstack queens setup. I have followed steps in openstack intsllation guide. https://docs.openstack.org/magnum/queens/install/install.html
But when I try to verify steps its failing as magnum-api shows errors default domain not found. All other services working as expected.
<pre><code>2020-04-24 10:21:45.758 169545 WARNING keystonemiddleware.auth_token [-] Identity response: {"error": {"message": "Could not find domain: default. (Disable insecure_debug mode to suppress these details.)", "code": 401, "title": "Unauthorized"}}: Unauthorized: Could not find domain: default. (Disable insecure_debug mode to suppress these details.) (HTTP 401) (Request-ID: req-15867afa-7a4c-4e69-9b15-9a7c287a5b11)
2020-04-24 10:21:45.786 169545 WARNING keystonemiddleware.auth_token [-] Identity response: {"error": {"message": "Could not find domain: default. (Disable insecure_debug mode to suppress these details.)", "code": 401, "title": "Unauthorized"}}: Unauthorized: Could not find domain: default. (Disable insecure_debug mode to suppress these details.) (HTTP 401) (Request-ID: req-6b837f1c-b7bb-472c-b6af-110aace8d996)
2020-04-24 10:21:45.786 169545 CRITICAL keystonemiddleware.auth_token [-] Unable to validate token: Identity server rejected authorization necessary to fetch token data: ServiceError: Identity server rejected authorization necessary to fetch token data
2020-04-24 10:21:45.788 169545 INFO werkzeug [-] 10.43.10.8 - - [24/Apr/2020 10:21:45] "GET /v1/mservices HTTP/1.1" 503 -</code></pre>

Even for any api in magnum it shows 503 service not available.
My magnum.conf file as follows :
<pre><code>root@controller1:~# egrep "^\[|^[a-zA-Z0-9]" /etc/magnum/magnum.conf
[DEFAULT]
transport_url = rabbit://openstack:########@controller:5672
pybasedir = /usr/lib/python2.7/dist-packages/magnum
bindir = /usr/bin
state_path = /var/lib/magnum
host = controller1
[api]
port = 9511
host = 0.0.0.0
[barbican_client]
[certificates]
cert_manager_type = x509keypair
[cinder]
[cinder_client]
region_name = RegionOne
[cluster]
[cluster_heat]
[cluster_template]
[conductor]
topic = magnum-conductor
conductor_life_check_timeout = 4
[cors]
[database]
connection = mysql+pymysql://magnum:########@controller/magnum
[docker]
[docker_registry]
[drivers]
[glance_client]
[heat_client]
[keystone_auth]
[keystone_authtoken]
www_authenticate_uri = http://controller:5000
auth_uri = http://controller:5000
auth_version = v3
auth_protocol = http
memcached_servers = controller:11211
project_domain_id = default
project_name = service
user_domain_id = default
password = ########
username = magnum
auth_url = http://controller:35357
auth_type = password
admin_user = magnum
admin_password = ########
admin_tenant_name = service
[magnum_client]
[matchmaker_redis]
[neutron_client]
[nova_client]
[oslo_concurrency]
lock_path = /var/lock/magnum
[oslo_messaging_amqp]
[oslo_messaging_kafka]
[oslo_messaging_notifications]
driver = messaging
[oslo_messaging_rabbit]
rabbit_host = localhost
rabbit_port = 5672
rabbit_userid = guest
rabbit_password =
[oslo_messaging_zmq]
[oslo_policy]
[profiler]
[quotas]
[trust]
trustee_domain_name = magnum
trustee_domain_admin_name = magnum_domain_admin
trustee_domain_admin_password = ########
trustee_keystone_interface = public
[x509]</code></pre>

When I try to list magnum service list getting following error
<pre>openstack coe service list --debug</pre>
<pre><code>REQ: curl -g -i -X GET http://controller:9511/v1/mservices -H "X-Auth-Token: {SHA1}dceee385a0529cf587d444ae52f779103a42a186" -H "Content-Type: application/json" -H "Accept: application/json" -H "User-Agent: None"
Starting new HTTP connection (1): controller
http://controller:9511 "GET /v1/mservices HTTP/1.1" 503 170
RESP: [503] content-length: 170 content-type: application/json x-openstack-request-id: req-d0df3806-5860-477e-a56f-63101c374623 server: Werkzeug/0.10.4 Python/2.7.12 date: Fri, 24 Apr 2020 05:18:17 GMT connection: keep-alive
RESP BODY: {"message": "The server is currently unavailable. Please try again at a later time.<br /><br />\n\n\n", "code": "503 Service Unavailable", "title": "Service Unavailable"}

GET call to container-infra for http://controller:9511/v1/mservices used request id req-d0df3806-5860-477e-a56f-63101c374623
'errors'
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/cliff/app.py", line 400, in run_subcommand
    result = cmd.run(parsed_args)
  File "/usr/lib/python2.7/dist-packages/osc_lib/command/command.py", line 41, in run
    return super(Command, self).run(parsed_args)
  File "/usr/lib/python2.7/dist-packages/cliff/display.py", line 116, in run
    column_names, data = self.take_action(parsed_args)
  File "/usr/lib/python2.7/dist-packages/magnumclient/osc/v1/mservices.py", line 35, in take_action
    services = client.mservices.list()
  File "/usr/lib/python2.7/dist-packages/magnumclient/v1/mservices.py", line 68, in list
    return self._list(self._path(path), "mservices")
  File "/usr/lib/python2.7/dist-packages/magnumclient/common/base.py", line 121, in _list
    resp, body = self.api.json_request('GET', url)
  File "/usr/lib/python2.7/dist-packages/magnumclient/common/httpclient.py", line 368, in json_request
    resp = self._http_request(url, method, **kwargs)
  File "/usr/lib/python2.7/dist-packages/magnumclient/common/httpclient.py", line 349, in _http_request
    error_json = _extract_error_json(resp.content)
  File "/usr/lib/python2.7/dist-packages/magnumclient/common/httpclient.py", line 55, in _extract_error_json
    error_body = body_json['errors'][0]
KeyError: 'errors'
clean_up ListService: 'errors'
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/osc_lib/shell.py", line 134, in run
    ret_val = super(OpenStackShell, self).run(argv)
  File "/usr/lib/python2.7/dist-packages/cliff/app.py", line 279, in run
    result = self.run_subcommand(remainder)
  File "/usr/lib/python2.7/dist-packages/osc_lib/shell.py", line 169, in run_subcommand
    ret_value = super(OpenStackShell, self).run_subcommand(argv)
  File "/usr/lib/python2.7/dist-packages/cliff/app.py", line 400, in run_subcommand
    result = cmd.run(parsed_args)
  File "/usr/lib/python2.7/dist-packages/osc_lib/command/command.py", line 41, in run
    return super(Command, self).run(parsed_args)
  File "/usr/lib/python2.7/dist-packages/cliff/display.py", line 116, in run
    column_names, data = self.take_action(parsed_args)
  File "/usr/lib/python2.7/dist-packages/magnumclient/osc/v1/mservices.py", line 35, in take_action
    services = client.mservices.list()
  File "/usr/lib/python2.7/dist-packages/magnumclient/v1/mservices.py", line 68, in list
    return self._list(self._path(path), "mservices")
  File "/usr/lib/python2.7/dist-packages/magnumclient/common/base.py", line 121, in _list
    resp, body = self.api.json_request('GET', url)
  File "/usr/lib/python2.7/dist-packages/magnumclient/common/httpclient.py", line 368, in json_request
    resp = self._http_request(url, method, **kwargs)
  File "/usr/lib/python2.7/dist-packages/magnumclient/common/httpclient.py", line 349, in _http_request
    error_json = _extract_error_json(resp.content)
  File "/usr/lib/python2.7/dist-packages/magnumclient/common/httpclient.py", line 55, in _extract_error_json
    error_body = body_json['errors'][0]
KeyError: 'errors'

END return value: 1
</code></pre>

[hanxiao (xiao0)]

I have get the same issue. The cause is that the user password is incorrect in my environment. Reset the user password to solve this issue.

## Reset user password
```
openstack user password --password <MAGNUM_PASS> magnum
openstack user password --password <MAGNUM_DOMAIN_ADMIN_PASS> magnum_domain_admin
```

## My environment
OS: Rocky Linux 9.2
OpenStack: zed
Magnum: zed

## cat /etc/magnum/magnum.conf
```bash
[DEFAULT]
transport_url = rabbit://openstack:<RABBIT_PASS>@controller:5672/
pybasedir = /usr/lib/python3.9/site-packages/magnum/
host = 192.168.40.11
state_path = /var/lib/magnum
debug = True
log_dir = /var/log/magnum

[api]
host = 192.168.40.11
port = 9511
enabled_ssl = False

[barbican_client]
region_name = RegionOne

[certificates]
cert_manager_type = barbican
storage_path = /var/lib/magnum/certificates/

[cinder]

[cinder_client]
region_name = RegionOne
endpoint_type = publicURL
api_version = 3

[cluster]

[cluster_heat]

[cluster_template]

[conductor]

[cors]

[database]
connection = mysql+pymysql://magnum:<MAGNUM_DB_PASS>@controller:3306/magnum

[docker]

[docker_registry]

[drivers]

[glance_client]

[heat_client]
region_name = RegionOne

[keystone_auth]
www_authenticate_uri = http://controller:5000/v3/
auth_url = http://controller:5000/v3/
memcached_servers = controller:11211
auth_type = password
project_domain_id = default
user_domain_id = default
project_name = service
username = magnum
password = <MAGNUM_PASS>

[keystone_authtoken]
region_name = RegionOne
interface = public
auth_version = v3
www_authenticate_uri = http://controller:5000/v3/
auth_url = http://controller:5000/v3/
memcached_servers = controller:11211
auth_type = password
project_domain_id = default
user_domain_id = default
project_name = service
username = magnum
password = <MAGNUM_PASS>
admin_user = magnum
admin_password = <MAGNUM_PASS>
admin_tenant_name = service

[kubernetes]

[magnum_client]
region_name = RegionOne

[neutron_client]

[nova_client]

[octavia_client]

[oslo_concurrency]
lock_path = $state_path/tmp

[oslo_messaging_amqp]

[oslo_messaging_kafka]

[oslo_messaging_notifications]
driver = messaging

[oslo_messaging_rabbit]

[oslo_policy]

[profiler]

[quotas]

[trust]
cluster_user_trust = True
trustee_domain_name = magnum
trustee_domain_admin_name = magnum_domain_admin
trustee_domain_admin_domain_name = magnum
trustee_domain_admin_password = <MAGNUM_DOMAIN_ADMIN_PASS>
trustee_keystone_interface = public

[x509]

```