[SRU] exceptions.from_response with webob 1.6.0 results in "AttributeError: 'unicode' object has no attribute 'get'"

[Phil Hopkins]

[Impact] [Testcase]
Running on Ubuntu 14.04.
After installing nova from source in either the Liberty release or Mitaka, with WebOb 1.6.0, running any nova command generated this error:
root@openstack-ubu-controller:~# nova service-list
ERROR (AttributeError): 'unicode' object has no attribute 'get'

The equivalent openstack commands work correctly. After downgrading WebOb to 1.5.1 AND restarting the nova-api service everything works.

Detailed output from nova -debug service-list with the error:

root@openstack-ubu-controller:~# nova --debug service-list
DEBUG (extension:157) found extension EntryPoint.parse('v2token = keystoneauth1.loading._plugins.identity.v2:Token')
DEBUG (extension:157) found extension EntryPoint.parse('admin_token = keystoneauth1.loading._plugins.admin_token:AdminToken')
DEBUG (extension:157) found extension EntryPoint.parse('v3oidcauthcode = keystoneauth1.loading._plugins.identity.v3:OpenIDConnectAuthorizationCode')
DEBUG (extension:157) found extension EntryPoint.parse('v2password = keystoneauth1.loading._plugins.identity.v2:Password')
DEBUG (extension:157) found extension EntryPoint.parse('v3password = keystoneauth1.loading._plugins.identity.v3:Password')
DEBUG (extension:157) found extension EntryPoint.parse('v3oidcpassword = keystoneauth1.loading._plugins.identity.v3:OpenIDConnectPassword')
DEBUG (extension:157) found extension EntryPoint.parse('token = keystoneauth1.loading._plugins.identity.generic:Token')
DEBUG (extension:157) found extension EntryPoint.parse('v3token = keystoneauth1.loading._plugins.identity.v3:Token')
DEBUG (extension:157) found extension EntryPoint.parse('password = keystoneauth1.loading._plugins.identity.generic:Password')
DEBUG (session:248) REQ: curl -g -i -X GET http://10.0.1.3:5000/v2.0 -H "Accept: application/json" -H "User-Agent: keystoneauth1/2.3.0 python-requests/2.9.1 CPython/2.7.6"
INFO (connectionpool:207) Starting new HTTP connection (1): 10.0.1.3
DEBUG (connectionpool:387) "GET /v2.0 HTTP/1.1" 200 334
DEBUG (session:277) RESP: [200] Content-Length: 334 Vary: X-Auth-Token Keep-Alive: timeout=5, max=100 Server: Apache/2.4.7 (Ubuntu) Connection: Keep-Alive Date: Fri, 18 Mar 2016 12:41:58 GMT Content-Type: application/json x-openstack-request-id: req-a0c68cd5-ea29-4391-942f-130cc69d15f8
RESP BODY: {"version": {"status": "stable", "updated": "2014-04-17T00:00:00Z", "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v2.0+json"}], "id": "v2.0", "links": [{"href": "http://10.0.1.3:5000/v2.0/", "rel": "self"}, {"href": "http://docs.openstack.org/", "type": "text/html", "rel": "describedby"}]}}

DEBUG (v2:63) Making authentication request to http://10.0.1.3:5000/v2.0/tokens
DEBUG (connectionpool:387) "POST /v2.0/tokens HTTP/1.1" 200 2465
DEBUG (session:248) REQ: curl -g -i -X GET http://10.0.1.3:8774/v1.1/b77d640e127e488fb42a7c0716ba53a5 -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}381893576ad46c62b587f4963d769b89441b919a"
INFO (connectionpool:207) Starting new HTTP connection (1): 10.0.1.3
DEBUG (connectionpool:387) "GET /v1.1/b77d640e127e488fb42a7c0716ba53a5 HTTP/1.1" 404 112
DEBUG (session:277) RESP: [404] Date: Fri, 18 Mar 2016 12:41:59 GMT Connection: keep-alive Content-Type: application/json; charset=UTF-8 Content-Length: 112 X-Compute-Request-Id: req-f10a2016-9a88-48fd-af1d-5f800fc9e11a
RESP BODY: {"message": "The resource could not be found.<br /><br />\n\n\n", "code": "404 Not Found", "title": "Not Found"}

DEBUG (shell:894) 'unicode' object has no attribute 'get'
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/novaclient/shell.py", line 892, in main
    OpenStackComputeShell().main(argv)
  File "/usr/local/lib/python2.7/dist-packages/novaclient/shell.py", line 726, in main
    api_version = api_versions.discover_version(self.cs, api_version)
  File "/usr/local/lib/python2.7/dist-packages/novaclient/api_versions.py", line 267, in discover_version
    client)
  File "/usr/local/lib/python2.7/dist-packages/novaclient/api_versions.py", line 248, in _get_server_version_range
    version = client.versions.get_current()
  File "/usr/local/lib/python2.7/dist-packages/novaclient/v2/versions.py", line 83, in get_current
    return self._get_current()
  File "/usr/local/lib/python2.7/dist-packages/novaclient/v2/versions.py", line 57, in _get_current
    return self._get(url, "version")
  File "/usr/local/lib/python2.7/dist-packages/novaclient/base.py", line 297, in _get
    _resp, body = self.api.client.get(url)
  File "/usr/local/lib/python2.7/dist-packages/keystoneauth1/adapter.py", line 173, in get
    return self.request(url, 'GET', **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/novaclient/client.py", line 92, in request
    raise exceptions.from_response(resp, body, url, method)
  File "/usr/local/lib/python2.7/dist-packages/novaclient/exceptions.py", line 274, in from_response
    message = error.get('message')
AttributeError: 'unicode' object has no attribute 'get'
ERROR (AttributeError): 'unicode' object has no attribute 'get'

Since nova-api must be restarted for this to either take effect or go away (with the WebOb downgrade) this appears to be an issue with the nova-api process. The failing message from the api service is 112 bytes long while the successful one (using downgraded WebOb) is 52 bytes long.

[Regression Potential]
The regression potential is minimal. The patch is cherry-picked without change from stable/newton to stable/mitaka, and I've already tested it in a ppa.

[Matt Riedemann (mriedem)]

I think we can confirm this, we're currently testing webob 1.5.1 in the gate but the upper-constraints proposal to include webob 1.6.0 is failing on the same thing:

http://logs.openstack.org/49/288849/15/check/gate-tempest-dsvm-full/4722c5a/logs/devstacklog.txt.gz#_2016-03-18_07_06_03_369

[Matt Riedemann (mriedem)]

The problem is in python-novaclient.

[Matt Riedemann (mriedem)]

Looks like this is the breaking change in webob 1.6.0:

https://github.com/Pylons/webob/pull/139

As you can see in the response body:

RESP BODY: {"message": "The resource could not be found.<br /><br />\n\n\n", "code": "404 Not Found", "title": "Not Found"}

The error message has html in it.

[Matt Riedemann (mriedem)]

Also wondering about https://github.com/Pylons/webob/pull/230

[Matt Riedemann (mriedem)]

The failure is here:

https://github.com/openstack/python-novaclient/blob/3.3.0/novaclient/exceptions.py#L298

Apparently our assumptions about the body there are wrong now.

[Matt Riedemann (mriedem)]

So given the error debug above, the body is this:

{"message": "The resource could not be found.<br /><br />\n\n\n", "code": "404 Not Found", "title": "Not Found"}

If you take that python-novaclient code, you get:

error = body[list(body)[0]]

>>> body[list(body)[0]]
'The resource could not be found.<br /><br />\n\n\n'

So error is going to be a str rather than a dict. I'm not sure what this was before.

[Matt Riedemann (mriedem)]

From what I can tell, this should also be a problem in openstackclient:

https://github.com/openstack/python-openstackclient/blob/2.2.0/openstackclient/common/exceptions.py#L125

And cinderclient:

https://github.com/openstack/python-cinderclient/blob/master/cinderclient/exceptions.py#L183

Unless it has something to do with the content-type on the request/response?

[Matt Riedemann (mriedem)]

Yeah, this is the issue:

https://github.com/Pylons/webob/commit/87c8749a57c1ff2442db2d74d9fb86935b7b201e

Which was written for glance, and of course glanceclient handles this properly:

https://github.com/openstack/python-glanceclient/blob/2.0.0/glanceclient/exc.py#L157

[Phil Hopkins (phil-hopkins-a)]

If you downgrade to WebOb 1.5.1 the problem will not go away until nova-api is restarted, so there is some involvement from the nova api server also.

[Matt Riedemann (mriedem)]

I've opened an issue against webob:

https://github.com/Pylons/webob/issues/235

I doubt they are going to change it since it's fixing a long standing behavior issue, but it was a backward incompatible change and probably should have gone out in a major version 2.0 release.

But we'll probably need to cap webob<1.6.0 until the various clients are working with this.

[Matt Riedemann (mriedem)]

Here is what a 404 response looks like with novaclient and webob 1.5.1:

DEBUG (session:248) REQ: curl -g -i -X GET http://9.5.126.108:8774/v2.1/e4f68bae118f46fa986e3fd54cbacb1e/flavors/test -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-OpenStack-Nova-API-Version: 2.25" -H "X-Auth-Token: {SHA1}8b87e089d79ad483b178bdf673e9639ffd86d529"
DEBUG (connectionpool:387) "GET /v2.1/e4f68bae118f46fa986e3fd54cbacb1e/flavors/test HTTP/1.1" 404 77
DEBUG (session:277) RESP: [404] Content-Length: 77 X-Compute-Request-Id: req-d9df03b0-4150-4b53-8157-7560ccf39f75 Vary: X-OpenStack-Nova-API-Version Connection: keep-alive X-Openstack-Nova-Api-Version: 2.25 Date: Fri, 18 Mar 2016 22:30:46 GMT Content-Type: application/json; charset=UTF-8
RESP BODY: {"itemNotFound": {"message": "Flavor test could not be found.", "code": 404}}

So the body is a dict of dicts, where I guess the error message keys to the body, which is how the code expected it to work in novaclient.

>>> body = {"itemNotFound": {"message": "Flavor test could not be found.", "code
": 404}}
>>> body[list(body)[0]]
{'message': 'Flavor test could not be found.', 'code': 404}
>>>

But now we get something like:

{"message": "The resource could not be found.<br /><br />\n\n\n", "code": "404 Not Found", "title": "Not Found"}

Note that even with webob 1.5.1 the content-type in the response is application/json, so it doesn't really help us to key off that to handle this. We basically can't trust the format until we require at least webob>=1.6.0.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/294850

[Matt Riedemann (mriedem)]

python-novaclient patch is here: https://review.openstack.org/#/c/294850/

[Matt Riedemann (mriedem)]

Looks like the apiclient in oslo-incubator handled this awhile ago:

https://github.com/openstack/oslo-incubator/commit/fd8dc0c3ffe38fe64a3c841d87ea820045221770

Which is now used in keystoneauth. And python-keystoneclient is using that code from keystoneauth.

[YAMAMOTO Takashi (yamamoto)]

https://bugs.launchpad.net/networking-midonet/+bug/1558923 seems like the same issue

[Bert JW Regeer (bertjwregeer)]

Hey all,

Sorry about the breakage, but I have a couple of questions, especially since Nova's Fault() should be doing the right thing (but clearly it is not):

See:

https://github.com/Pylons/webob/issues/235#issuecomment-198643200

If someone can help me understand the flow on the nova API side then I can make sure that it is a bug in WebOb, because according to my reading WebOb shouldn't even be hitting the code path that generates it's internal JSON response.

[Bert JW Regeer (bertjwregeer)]

I have run all tests on the nova codebase using tox -e py27,functional with WebOb 1.6.0 and I can not replicate any failure's. Please provide a stacktrace that shows WebOb's exception's generate_response function being called (which is what is causing the wrong JSON to be returned).

[Bert JW Regeer (bertjwregeer)]

Referring to #17:

https://github.com/Pylons/webob/blob/1.6-branch/webob/exc.py#L325

This is the function that has to be called to generate the wrong JSON response. With the Fault wrapping that nova does, I can't figure out how it would ever be called. See the Github issue link: https://github.com/Pylons/webob/issues/235 for more information on what I have tried and the analysis.

[Ian Cordasco (icordasc)]

Bert, thanks for the offer to help.

So Nova uses this FaultWrapper middleware (https://github.com/openstack/nova/blob/3b2895b960630934f827918d50577f9167110cab/nova/api/openstack/__init__.py#L106) to intercept all exceptions and problems so it can re-wrap them in its Fault class. I wonder if something changed that stopped calling the Fault instance. It seems like we're expecting all of the work that is causing this to happen in the Fault's __call__ (https://github.com/openstack/nova/blob/2bc8128d7793cc72ca2e146de3a092e1fef5033b/nova/api/openstack/wsgi.py#L1061) method and I wonder if something stopped calling it.

Note also that the FaultWrapper middleware *returns* the exception instead of re-raising the exception wrapped in the Fault. Could either of those (or both) be related to this problem?

[Ian Cordasco (icordasc)]

For what it's wroth, the paste file looks like this:

https://github.com/openstack/nova/blob/2bc8128d7793cc72ca2e146de3a092e1fef5033b/etc/nova/api-paste.ini#L45 which uses the faultwrap filter defined here: https://github.com/openstack/nova/blob/2bc8128d7793cc72ca2e146de3a092e1fef5033b/etc/nova/api-paste.ini#L62

[Alex Xu (xuhj)]

Nova return fault at here https://github.com/openstack/nova/blob/stable/mitaka/nova/wsgi.py#L460 for any resource which didn't register to the route mapper

If we try to request with '-H "Accept: text/plain"' explicitly and with webob 1.6.0, we will get same response as before"

curl -g -i -X GET http://192.168.2.176:8774/v2.1/b90b53ed87d74e19806da34dbaa056c9/ser -H "User-Agent: python-novaclient" -H "Accept: text/plain" -H "X-Auth-Token: 3d30d32d4c3f4c36b268656f070a7e4a"HTTP/1.1 404 Not Found
Content-Length: 52
Content-Type: text/plain; charset=UTF-8
X-Compute-Request-Id: req-12de9dc1-ff6a-4f77-81a3-079a1d4c0221
Date: Tue, 22 Mar 2016 09:58:47 GMT

404 Not Found

The resource could not be found.

[Alex Xu (xuhj)]

@Matt, I think we will return the same response even we upgrade webob to 1.6 in the case of "GET /v2.1/e4f68bae118f46fa986e3fd54cbacb1e/flavors/test", due to this is registered resource. The response changed only for the case like "GET /v2.1/b90b53ed87d74e19806da34dbaa056c9/not_registered_resource"

[OpenStack Infra (hudson-openstack)]

Change abandoned by YAMAMOTO Takashi (<email address hidden>) on branch: master
Review: https://review.openstack.org/294527
Reason: turned out to be unnecessary

[YAMAMOTO Takashi (yamamoto)]

midonet v2015.06 fix: https://review.gerrithub.io/#/c/269940/

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/298086

[OpenStack Infra (hudson-openstack)]

Change abandoned by Wenzhi Yu ([e-mail]) on branch: master
Review: https://review.openstack.org/298086
Reason: 'from_response' is never called in code base, should remove it.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/298617

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/298617
Committed: https://git.openstack.org/cgit/openstack/python-openstackclient/commit/?id=848d5312fffa358a789a46a8657af3a92cb3ef6f
Submitter: Jenkins
Branch: master

commit 848d5312fffa358a789a46a8657af3a92cb3ef6f
Author: Wenzhi Yu <[e-mail]>
Date: Tue Mar 29 17:20:05 2016 +0800

    Remove unused method 'from_response'

    'openstackclient.common.exceptions.from_response' method is never called
    in openstackclient code base, so we should remove it.

    Change-Id: I04254a4e66863942e6c273d77bbd66ce2ce7804c
    Related-Bug: #1559072

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/294850
Committed: https://git.openstack.org/cgit/openstack/python-novaclient/commit/?id=fa377e7fca44e81247639efebd7cae5cb377e57e
Submitter: Jenkins
Branch: master

commit fa377e7fca44e81247639efebd7cae5cb377e57e
Author: Matt Riedemann <email address hidden>
Date: Fri Mar 18 19:17:47 2016 -0400

    Handle error response for webob>=1.6.0

    WebOb change https://github.com/Pylons/webob/pull/230 changed
    the way in which the error response body is formatted such that
    it's no longer a nested dict. So we have to handle both the
    old convention of an error message key to the response body error
    dict and the new way with just the error body dict.

    This was reported upstream:

    https://github.com/Pylons/webob/issues/235

    But given this was apparently implemented as a long-overdue change
    in WebOb the behavior is not likely to change.

    Change-Id: If653a247d842786d2824b4b3a5c0cde1383ed7ab
    Closes-Bug: #1559072

[YAMAMOTO Takashi (yamamoto)]

all relevant patches are merged

[Davanum Srinivas (DIMS) (dims-v)]

This issue was fixed in the openstack/python-novaclient 3.4.0 release.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/307021

[OpenStack Infra (hudson-openstack)]

Change abandoned by Alexis Lee (<email address hidden>) on branch: stable/mitaka
Review: https://review.openstack.org/307021
Reason: cool, thank you

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[James Page (james-page)]

Fixes in 3.4.0; Marking yakkety task as fix released.

[James Page (james-page)]

Unable to reproduce on Xenial; Xenial has webob 1.5.1 so I don't think its likely to be impacted.

[James Page (james-page)]

Re-opening Xenial task - accessing a cloud using webob 1.6.0 with older novaclient from 16.04 will fail with this problem; bearing this in mind we might want to consider resurrecting the stable/mitaka novaclient fix as well.

[Ryan Beisner (1chb1n)]

In trying to create a nova flavor from a Xenial machine against a Xenial-Newton deployment, I see:

00:25:25.978 + nova flavor-create m1.cirros 6 64 1 1
00:25:27.740 ERROR (AttributeError): 'unicode' object has no attribute 'get'

[Ryan Beisner (1chb1n)]

jenkins@juju-osci1-machine-8:~/temp$ apt-cache policy python-novaclient
python-novaclient:
  Installed: 2:3.3.1-2
  Candidate: 2:3.3.1-2
  Version table:
 *** 2:3.3.1-2 500
        500 http://nova.clouds.archive.ubuntu.com/ubuntu xenial/main amd64 Packages
        100 /var/lib/dpkg/status

[Corey Bryant (corey.bryant)]

I just tested python-novaclient 2:3.3.1-2 on xenial with the abandoned upstream patch successfully.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/378671

[Corey Bryant (corey.bryant)]

I've re-proposed the stable/mitaka patch for this since this is an issue when the stable/mitaka python-novaclient is used with newton+ nova deployments.

[Chris J Arges (arges)]

Hello Phil, or anyone else affected,

Accepted python-novaclient into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/python-novaclient/2:3.3.1-2ubuntu1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[OpenStack Infra (hudson-openstack)]

Change abandoned by Corey Bryant (<email address hidden>) on branch: stable/mitaka
Review: https://review.openstack.org/378671
Reason: Abandoned in favor of: https://review.openstack.org/307021

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/307021
Committed: https://git.openstack.org/cgit/openstack/python-novaclient/commit/?id=70c35e6e3d2aed9f37d5101dafd22c5905ec8583
Submitter: Jenkins
Branch: stable/mitaka

commit 70c35e6e3d2aed9f37d5101dafd22c5905ec8583
Author: Matt Riedemann <email address hidden>
Date: Fri Mar 18 19:17:47 2016 -0400

    Handle error response for webob>=1.6.0

    WebOb change https://github.com/Pylons/webob/pull/230 changed
    the way in which the error response body is formatted such that
    it's no longer a nested dict. So we have to handle both the
    old convention of an error message key to the response body error
    dict and the new way with just the error body dict.

    This was reported upstream:

    https://github.com/Pylons/webob/issues/235

    But given this was apparently implemented as a long-overdue change
    in WebOb the behavior is not likely to change.

    Change-Id: If653a247d842786d2824b4b3a5c0cde1383ed7ab
    Closes-Bug: #1559072

[Sean McGinnis (sean-mcginnis)]

This has been fixed in webob 1.6.1:

http://docs.webob.org/en/stable/changes.html

[Corey Bryant (corey.bryant)]

Fix tested successfully on xenial-proposed and mitaka-proposed.

[Launchpad Janitor (janitor)]

This bug was fixed in the package python-novaclient - 2:3.3.1-2ubuntu1

---------------
python-novaclient (2:3.3.1-2ubuntu1) xenial; urgency=medium

  * d/p/enable-1.6.0-webob.patch: Cherry pick from stable/mitaka gerrit review
    to enable use of python-novaclient with newton+ nova deployments that have
    python-webob >= 1.6.0 (LP: #1559072).

 -- Corey Bryant <email address hidden> Wed, 28 Sep 2016 10:44:36 -0400

[Chris J Arges (arges)]

The verification of the Stable Release Update for python-novaclient has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Eric Harney (eharney)]

cinderclient patch: https://review.openstack.org/#/c/407422/

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/407422
Committed: https://git.openstack.org/cgit/openstack/python-cinderclient/commit/?id=19befa69651a41534c7d25b1a0d3a0cd407afb44
Submitter: Jenkins
Branch: master

commit 19befa69651a41534c7d25b1a0d3a0cd407afb44
Author: Akira KAMIO <email address hidden>
Date: Tue Dec 6 14:10:38 2016 +0900

    Handle error response for webob>=1.6.0

    WebOb change https://github.com/Pylons/webob/pull/230 changed
    the way in which the error response body is formatted such that
    it's no longer a nested dict. So we have to handle both the
    old convention of an error message key to the response body error
    dict and the new way with just the error body dict.

    This was reported upstream:

    https://github.com/Pylons/webob/issues/235

    But given this was apparently implemented as a long-overdue change
    in WebOb the behavior is not likely to change.Handle error response for
    webob>=1.6.0

    Change-Id: I7d589415aa024588faf77c8234ac026110f6c3cd
    Closes-Bug: #1559072

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/python-cinderclient 1.10.0 release.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/462204

[Simon Leinen (simon-leinen)]

A comment about the new review 462204: I requested a backport of the cinderclient patch (407422) to Newton. We recently upgraded to Newton, and a user is now unable to delete Heat stacks that contain Cinder volumes that have already been removed. Applying the cinderclient patch fixes this problem.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/462204
Committed: https://git.openstack.org/cgit/openstack/python-cinderclient/commit/?id=b20d9fd6563c1ef295e8a260742f2b3e0b8ee327
Submitter: Jenkins
Branch: stable/newton

commit b20d9fd6563c1ef295e8a260742f2b3e0b8ee327
Author: Akira KAMIO <email address hidden>
Date: Tue Dec 6 14:10:38 2016 +0900

    Handle error response for webob>=1.6.0

    WebOb change https://github.com/Pylons/webob/pull/230 changed
    the way in which the error response body is formatted such that
    it's no longer a nested dict. So we have to handle both the
    old convention of an error message key to the response body error
    dict and the new way with just the error body dict.

    This was reported upstream:

    https://github.com/Pylons/webob/issues/235

    But given this was apparently implemented as a long-overdue change
    in WebOb the behavior is not likely to change.Handle error response for
    webob>=1.6.0

    Change-Id: I7d589415aa024588faf77c8234ac026110f6c3cd
    Closes-Bug: #1559072
    (cherry picked from commit 19befa69651a41534c7d25b1a0d3a0cd407afb44)

[Volodymyr Litovka (doka.ua)]

Openstack: stable/ocata
Ubuntu 16.04.02 Xenial Xerus

$ openstack server list
+--------------------------------------+--------+-----------+
| ID | Name | Status |
+--------------------------------------+--------+-----------+
| 54b2465f-6a5d-467e-8815-42c436852ba6 | jex-n1 | ACTIVE |
| a0442c11-dc5b-4d8f-9e77-a83b59782b7c | jex-n2 | SUSPENDED |
+--------------------------------------+--------+-----------+

$ openstack server rebuild jex-n1
'unicode' object has no attribute 'get'

====== --debug -vvvv =======
GET call to compute for http://controller:8774/v2.1/servers?name=jex-n1 used request id req-def51157-4adb-4c37-9ccf-77a8124fa939
REQ: curl -g -i -X GET http://controller:8774/v2.1/servers/54b2465f-6a5d-467e-8815-42c436852ba6 -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}7970e0bb2968bd69cc6f49026ba350e55da7b105"
http://controller:8774 "GET /v2.1/servers/54b2465f-6a5d-467e-8815-42c436852ba6 HTTP/1.1" 200 1782
RESP: [200] Content-Length: 1782 Content-Type: application/json Openstack-Api-Version: compute 2.1 X-Openstack-Nova-Api-Version: 2.1 Vary: OpenStack-API-Version, X-OpenStack-Nova-API-Version X-Compute-Request-Id: req-4e02c928-6dbc-4f79-9cc9-1ca0ad8e69e3 Date: Mon, 05 Jun 2017 10:44:16 GMT
RESP BODY: {"server": {"OS-EXT-STS:task_state": null, "addresses": {"e-net": [{"OS-EXT-IPS-MAC:mac_addr": "d0:1c:a0:04:a4:5d", "version": 4, "addr": "51.255.0.248", "OS-EXT-IPS:type": "fixed"}], "jex-net": [{"OS-EXT-IPS-MAC:mac_addr": "d0:1c:a0:74:f8:57", "version": 4, "addr": "1.1.1.10", "OS-EXT-IPS:type": "fixed"}], "dummy-net": [{"OS-EXT-IPS-MAC:mac_addr": "d0:1c:a0:8c:ae:f2", "version": 4, "addr": "25.0.0.7", "OS-EXT-IPS:type": "fixed"}]}, "links": [{"href": "http://controller:8774/v2.1/servers/54b2465f-6a5d-467e-8815-42c436852ba6", "rel": "self"}, {"href": "http://controller:8774/servers/54b2465f-6a5d-467e-8815-42c436852ba6", "rel": "bookmark"}], "image": "", "OS-EXT-STS:vm_state": "active", "OS-EXT-SRV-ATTR:instance_name": "instance-0000016f", "OS-SRV-USG:launched_at": "2017-06-05T08:53:58.000000", "flavor": {"id": "d0ff4bc5-df38-4f20-8908-afc516d594e6", "links": [{"href": "http://controller:8774/flavors/d0ff4bc5-df38-4f20-8908-afc516d594e6", "rel": "bookmark"}]}, "id": "54b2465f-6a5d-467e-8815-42c436852ba6", "security_groups": [{"name": "default"}, {"name": "jex-esg"}], "user_id": "0d01892c43b0498198c4716d510c6667", "OS-DCF:diskConfig": "MANUAL", "accessIPv4": "", "accessIPv6": "", "progress": 0, "OS-EXT-STS:power_state": 1, "OS-EXT-AZ:availability_zone": "nova", "metadata": {}, "status": "ACTIVE", "updated": "2017-06-05T08:53:59Z", "hostId": "4c5f24a68ef88f97beb99c3e34d5ea6cba6d3fabe3dc5a6487ed8415", "OS-EXT-SRV-ATTR:host": "ardbeg", "OS-SRV-USG:terminated_at": null, "key_name": null, "OS-EXT-SRV-ATTR:hypervisor_hostname": "compute0.cloud.local", "name": "jex-n1", "created": "2017-06-05T08:53:42Z", "tenant_id": "d8051a3ff3ad4c4bb380f828992b8178", "os-extended-volumes:volumes_attached": [{"id": "d76e23e8-c6cb-462e-9ac0-81b2d733fc77"}], "config_drive": ""}}
GET call to compute for http://controller:8774/v2.1/servers/54b2465f-6a5d-467e-8815-42c436852ba6 used re...

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/python-novaclient 3.3.3 release.

[Corey Bryant (corey.bryant)]

yakkety is EOL

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Corey Bryant (corey.bryant)]

I've uploaded python-cinderclient to the xenial review queue and directly to the newton cloud archive staging PPA.

[Brian Murray (brian-murray)]

Hello Phil, or anyone else affected,

Accepted python-cinderclient into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/python-cinderclient/1:1.6.0-2ubuntu1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[James Page (james-page)]

Hello Phil, or anyone else affected,

Accepted python-cinderclient into newton-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:newton-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-newton-needed to verification-newton-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-newton-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Saverio Proto (zioproto)]

I tested the package on Newton. It LGTM. Thank you

[James Page (james-page)]

Hello Phil, or anyone else affected,

Accepted python-cinderclient into mitaka-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:mitaka-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-mitaka-needed to verification-mitaka-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-mitaka-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[James Page (james-page)]

@zioproto

Please could you update the bug tags inline with the acceptance comments when you've completed testing.

[James Page (james-page)]

The verification of the Stable Release Update for python-cinderclient has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[James Page (james-page)]

This bug was fixed in the package python-cinderclient - 1:1.9.0-0ubuntu1~cloud2
---------------

 python-cinderclient (1:1.9.0-0ubuntu1~cloud2) xenial-newton; urgency=medium
 .
   * No change. Upload new version as last version was rejected (LP: #1559072).

[Corey Bryant (corey.bryant)]

I've verified that python-cinderclient is fixed on xenial-proposed: http://paste.ubuntu.com/25283545/

Regression tests have also passed successfully.

[Corey Bryant (corey.bryant)]

I've also verified against the same cloud (with web>=1.6.0) via a trusty machine with the mitaka-proposed cloud archive: http://paste.ubuntu.com/25283602/

[Launchpad Janitor (janitor)]

This bug was fixed in the package python-cinderclient - 1:1.6.0-2ubuntu1

---------------
python-cinderclient (1:1.6.0-2ubuntu1) xenial; urgency=medium

  * d/p/review-462204.patch: Handle error response for webob>=1.6.0
    (LP: #1559072).

 -- Saverio Proto <email address hidden> Mon, 26 Jun 2017 11:55:31 +0000

[Ryan Beisner (1chb1n)]

The verification of the Stable Release Update for python-cinderclient has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Ryan Beisner (1chb1n)]

This bug was fixed in the package python-cinderclient - 1:1.6.0-2ubuntu1~cloud0
---------------

 python-cinderclient (1:1.6.0-2ubuntu1~cloud0) trusty-mitaka; urgency=medium
 .
   * New update for the Ubuntu Cloud Archive.
 .
 python-cinderclient (1:1.6.0-2ubuntu1) xenial; urgency=medium
 .
   * d/p/review-462204.patch: Handle error response for webob>=1.6.0
     (LP: #1559072).

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/541170

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/541170
Committed: https://git.openstack.org/cgit/openstack/python-troveclient/commit/?id=0b28b04ebbfaed981f4e00e69e4c7a8e0ed75395
Submitter: Zuul
Branch: master

commit 0b28b04ebbfaed981f4e00e69e4c7a8e0ed75395
Author: Zhao Chao <email address hidden>
Date: Tue Feb 6 15:31:42 2018 +0800

    Handle error response for webob>=1.6.0

    WebOb change https://github.com/Pylons/webob/pull/230 changed
    the way in which the error response body is formatted such that
    it's no longer a nested dict. So we have to handle both the
    old convention of an error message key to the response body error
    dict and the new way with just the error body dict.

    This was reported upstream:
    https://github.com/Pylons/webob/issues/235

    But given this was apparently implemented as a long-overdue change
    in WebOb the behavior is not likely to change.

    Change-Id: Id8bfde6277640de48768bbe5a5e4252b97253ed2
    Closes-Bug: #1559072
    Co-Authored-By: Matt Riedemann <email address hidden>
    Signed-off-by: Zhao Chao <email address hidden>

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/python-troveclient 2.15.0 release.