Unable to talk to nova properly

[venkatamahesh]

To reproduce just run devstack with default settings:

Most interactions involving nova are failing after that.

Original description:

When we try to create flavor using dashboard it is giving error :
Danger: There was an error submitting the form. Please try again

[Cindy Lu (clu-m)]

Hi Venkata, can you give more context? What if you refresh the browser?

[venkatamahesh (venkatamaheshkotha)]

I am trying to create a flavor and when I entered the flavor specifications it is throwing this error

[Itxaka Serrano (itxaka)]

Venkata, can you provide the horizon logs to see what the error is?

Also please provide the versions for horizon. Thanks!

[Rob Cresswell (robcresswell-deactivatedaccount)]

We need logs and Horizon version. There is no obvious issue on master.

[Dr. Jens Harbott (j-harbott)]

I can reprouce this failure running a Chef based deployment of Ubuntu Cloud archive Newton on Xenial.
I'm also seeing "Error: Unable to retrieve flavor list." when accessing the admin/flavors tab.

Version is openstack-dashboard=3:10.0.0-0ubuntu1~cloud0.

Traceback from the create flavor:

[Tue Jan 24 10:55:22.880955 2017] [wsgi:error] [pid 11124:tid 140408635807488] Internal Server Error: /admin/flavors/create/
[Tue Jan 24 10:55:22.880990 2017] [wsgi:error] [pid 11124:tid 140408635807488] Traceback (most recent call last):
[Tue Jan 24 10:55:22.880999 2017] [wsgi:error] [pid 11124:tid 140408635807488] File "/usr/lib/python2.7/dist-packages/django/core/handlers/base.py", line 132, in get_response
[Tue Jan 24 10:55:22.881007 2017] [wsgi:error] [pid 11124:tid 140408635807488] response = wrapped_callback(request, *callback_args, **callback_kwargs)
[Tue Jan 24 10:55:22.881014 2017] [wsgi:error] [pid 11124:tid 140408635807488] File "/usr/share/openstack-dashboard/openstack_dashboard/wsgi/../../horizon/decorators.py", line 36, in dec
[Tue Jan 24 10:55:22.881021 2017] [wsgi:error] [pid 11124:tid 140408635807488] return view_func(request, *args, **kwargs)
[Tue Jan 24 10:55:22.881029 2017] [wsgi:error] [pid 11124:tid 140408635807488] File "/usr/share/openstack-dashboard/openstack_dashboard/wsgi/../../horizon/decorators.py", line 52, in dec
[Tue Jan 24 10:55:22.881036 2017] [wsgi:error] [pid 11124:tid 140408635807488] return view_func(request, *args, **kwargs)
[Tue Jan 24 10:55:22.881042 2017] [wsgi:error] [pid 11124:tid 140408635807488] File "/usr/share/openstack-dashboard/openstack_dashboard/wsgi/../../horizon/decorators.py", line 36, in dec
[Tue Jan 24 10:55:22.881050 2017] [wsgi:error] [pid 11124:tid 140408635807488] return view_func(request, *args, **kwargs)
[Tue Jan 24 10:55:22.881056 2017] [wsgi:error] [pid 11124:tid 140408635807488] File "/usr/share/openstack-dashboard/openstack_dashboard/wsgi/../../horizon/decorators.py", line 84, in dec
[Tue Jan 24 10:55:22.881063 2017] [wsgi:error] [pid 11124:tid 140408635807488] return view_func(request, *args, **kwargs)
[Tue Jan 24 10:55:22.881069 2017] [wsgi:error] [pid 11124:tid 140408635807488] File "/usr/lib/python2.7/dist-packages/django/views/generic/base.py", line 71, in view
[Tue Jan 24 10:55:22.881076 2017] [wsgi:error] [pid 11124:tid 140408635807488] return self.dispatch(request, *args, **kwargs)
[Tue Jan 24 10:55:22.881084 2017] [wsgi:error] [pid 11124:tid 140408635807488] File "/usr/lib/python2.7/dist-packages/django/views/generic/base.py", line 89, in dispatch
[Tue Jan 24 10:55:22.881090 2017] [wsgi:error] [pid 11124:tid 140408635807488] return handler(request, *args, **kwargs)
[Tue Jan 24 10:55:22.881096 2017] [wsgi:error] [pid 11124:tid 140408635807488] File "/usr/share/openstack-dashboard/openstack_dashboard/wsgi/../../horizon/workflows/views.py", line 169, in post
[Tue Jan 24 10:55:22.881102 2017] [wsgi:error] [pid 11124:tid 140408635807488] context = self.get_context_data(**kwargs)
[Tue Jan 24 10:55:22.881108 2017] [wsgi:error] [pid 11124:tid 140408635807488] File "/usr/share/openstack-dashboard/openstack_dashboard/wsgi/../../horizon/workfl...

[Dr. Jens Harbott (j-harbott)]

For the "Unable to retrieve flavor list." I'm seeing this in the horizon log after setting Debug=True:

[Tue Jan 24 11:46:22.128831 2017] [wsgi:error] [pid 6856:tid 140119210809088] Property `management_url` is deprecated for SessionClient. It should be set via `endpoint_override` variable while class initialization.
[Tue Jan 24 11:46:22.129507 2017] [wsgi:error] [pid 6856:tid 140119210809088] DEBUG:keystoneauth.session:REQ: curl -g -i --insecure -X GET http://127.0.0.1:5000/v3 -H "Accept: application/json" -H "User-Agent: python-novaclient keystoneauth1/2.18.0 python-requests/2.12.5 CPython/2.7.12"
[Tue Jan 24 11:46:22.240292 2017] [wsgi:error] [pid 6856:tid 140119210809088] DEBUG:keystoneauth.session:RESP: [200] Date: Tue, 24 Jan 2017 11:46:22 GMT Server: Apache Vary: X-Auth-Token X-Distribution: Ubuntu x-openstack-request-id: req-90b89e66-817d-4aa7-aa24-2814d15b3ed6 Content-Length: 248 Keep-Alive: timeout=5, max=100 Connection: Keep-Alive Content-Type: application/json
[Tue Jan 24 11:46:22.240322 2017] [wsgi:error] [pid 6856:tid 140119210809088] RESP BODY: {"version": {"status": "stable", "updated": "2016-10-06T00:00:00Z", "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v3+json"}], "id": "v3.7", "links": [{"href": "http://127.0.0.1:5000/v3/", "rel": "self"}]}}
[Tue Jan 24 11:46:22.240328 2017] [wsgi:error] [pid 6856:tid 140119210809088]
[Tue Jan 24 11:46:22.240447 2017] [wsgi:error] [pid 6856:tid 140119210809088] DEBUG:keystoneauth.session:GET call to None for http://127.0.0.1:5000/v3 used request id req-90b89e66-817d-4aa7-aa24-2814d15b3ed6
[Tue Jan 24 11:46:22.241110 2017] [wsgi:error] [pid 6856:tid 140119210809088] DEBUG:keystoneauth.identity.v3.base:Making authentication request to http://127.0.0.1:5000/v3/auth/tokens
[Tue Jan 24 11:46:22.269882 2017] [wsgi:error] [pid 6856:tid 140119210809088] DEBUG:keystoneauth.session:Request returned failure status: 400
[Tue Jan 24 11:46:22.270453 2017] [wsgi:error] [pid 6856:tid 140119210809088] Recoverable error: Expecting to find domain in user - the server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. (HTTP 400) (Request-ID: req-33625908-c2ab-49a3-b146-76bbd57ce52e)
[Tue Jan 24 11:46:22.277146 2017] [wsgi:error] [pid 6856:tid 140119210809088] DEBUG:keystoneauth.session:REQ: curl -g -i --insecure -X GET http://127.0.0.1:5000/v3/users/3e21a9dd5c2742d69d8a3aeddf8ce378/projects -H "User-Agent: python-keystoneclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}d150621fece0f7fd13a1e2e7bbebbdd1f7730252"
[Tue Jan 24 11:46:22.345641 2017] [wsgi:error] [pid 6856:tid 140119210809088] DEBUG:keystoneauth.session:RESP: [200] Date: Tue, 24 Jan 2017 11:46:22 GMT Server: Apache Vary: X-Auth-Token X-Distribution: Ubuntu x-openstack-request-id: req-0ca43e7b-7c58-4143-817e-623709fc460c Content-Length: 440 Keep-Alive: timeout=5, max=100 Connection: Keep-Alive Content-Type: application/json
[Tue Jan 24 11:46:22.345661 2017] [wsgi:error] [pid 6856:tid 140119210809088] RESP BODY: {"links": {"self": "http://127.0.0.1:5000/v3/users/3e21a9dd5c2742d69d8a3aeddf8ce378/projects", "previous": nul...

[Dr. Jens Harbott (j-harbott)]

So it turns out that this is another duplicate of 1654183, fixed in python-novaclient git master currently.

For the Chef deployment, there was a global pip based python-openstackclient installation that unfortunately installed the latest python-novaclient, too, and made it leak into other services.

[Jeremy Nauck (jedihamster)]

I tried to replace python-openstackclient with 7.1.0 from the Git master branch and still see the same issue. I opened bug report 1661456 (https://bugs.launchpad.net/horizon/+bug/1661456) with some more information.