[SRU] exceptions.from_response with webob 1.6.0 results in "AttributeError: 'unicode' object has no attribute 'get'"

Bug #1559072 reported by Phil Hopkins on 2016-03-18
52
This bug affects 9 people
Affects Status Importance Assigned to Milestone
Ubuntu Cloud Archive
Status tracked in Pike
Mitaka
High
Saverio Proto
Newton
High
Saverio Proto
Ocata
High
Unassigned
Pike
High
Unassigned
networking-midonet
Critical
YAMAMOTO Takashi
python-cinderclient
Undecided
Akira KAMIO
python-novaclient
High
Matt Riedemann
python-openstackclient
Undecided
Wenzhi Yu
python-cinderclient (Ubuntu)
High
Unassigned
Xenial
High
Saverio Proto
Yakkety
Undecided
Unassigned
Zesty
High
Unassigned
python-novaclient (Ubuntu)
High
Unassigned
Xenial
High
Unassigned
Yakkety
High
Unassigned
Zesty
Undecided
Unassigned

Bug Description

[Impact] [Testcase]
Running on Ubuntu 14.04.
After installing nova from source in either the Liberty release or Mitaka, with WebOb 1.6.0, running any nova command generated this error:
root@openstack-ubu-controller:~# nova service-list
ERROR (AttributeError): 'unicode' object has no attribute 'get'

The equivalent openstack commands work correctly. After downgrading WebOb to 1.5.1 AND restarting the nova-api service everything works.

Detailed output from nova -debug service-list with the error:

root@openstack-ubu-controller:~# nova --debug service-list
DEBUG (extension:157) found extension EntryPoint.parse('v2token = keystoneauth1.loading._plugins.identity.v2:Token')
DEBUG (extension:157) found extension EntryPoint.parse('admin_token = keystoneauth1.loading._plugins.admin_token:AdminToken')
DEBUG (extension:157) found extension EntryPoint.parse('v3oidcauthcode = keystoneauth1.loading._plugins.identity.v3:OpenIDConnectAuthorizationCode')
DEBUG (extension:157) found extension EntryPoint.parse('v2password = keystoneauth1.loading._plugins.identity.v2:Password')
DEBUG (extension:157) found extension EntryPoint.parse('v3password = keystoneauth1.loading._plugins.identity.v3:Password')
DEBUG (extension:157) found extension EntryPoint.parse('v3oidcpassword = keystoneauth1.loading._plugins.identity.v3:OpenIDConnectPassword')
DEBUG (extension:157) found extension EntryPoint.parse('token = keystoneauth1.loading._plugins.identity.generic:Token')
DEBUG (extension:157) found extension EntryPoint.parse('v3token = keystoneauth1.loading._plugins.identity.v3:Token')
DEBUG (extension:157) found extension EntryPoint.parse('password = keystoneauth1.loading._plugins.identity.generic:Password')
DEBUG (session:248) REQ: curl -g -i -X GET http://10.0.1.3:5000/v2.0 -H "Accept: application/json" -H "User-Agent: keystoneauth1/2.3.0 python-requests/2.9.1 CPython/2.7.6"
INFO (connectionpool:207) Starting new HTTP connection (1): 10.0.1.3
DEBUG (connectionpool:387) "GET /v2.0 HTTP/1.1" 200 334
DEBUG (session:277) RESP: [200] Content-Length: 334 Vary: X-Auth-Token Keep-Alive: timeout=5, max=100 Server: Apache/2.4.7 (Ubuntu) Connection: Keep-Alive Date: Fri, 18 Mar 2016 12:41:58 GMT Content-Type: application/json x-openstack-request-id: req-a0c68cd5-ea29-4391-942f-130cc69d15f8
RESP BODY: {"version": {"status": "stable", "updated": "2014-04-17T00:00:00Z", "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v2.0+json"}], "id": "v2.0", "links": [{"href": "http://10.0.1.3:5000/v2.0/", "rel": "self"}, {"href": "http://docs.openstack.org/", "type": "text/html", "rel": "describedby"}]}}

DEBUG (v2:63) Making authentication request to http://10.0.1.3:5000/v2.0/tokens
DEBUG (connectionpool:387) "POST /v2.0/tokens HTTP/1.1" 200 2465
DEBUG (session:248) REQ: curl -g -i -X GET http://10.0.1.3:8774/v1.1/b77d640e127e488fb42a7c0716ba53a5 -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}381893576ad46c62b587f4963d769b89441b919a"
INFO (connectionpool:207) Starting new HTTP connection (1): 10.0.1.3
DEBUG (connectionpool:387) "GET /v1.1/b77d640e127e488fb42a7c0716ba53a5 HTTP/1.1" 404 112
DEBUG (session:277) RESP: [404] Date: Fri, 18 Mar 2016 12:41:59 GMT Connection: keep-alive Content-Type: application/json; charset=UTF-8 Content-Length: 112 X-Compute-Request-Id: req-f10a2016-9a88-48fd-af1d-5f800fc9e11a
RESP BODY: {"message": "The resource could not be found.<br /><br />\n\n\n", "code": "404 Not Found", "title": "Not Found"}

DEBUG (shell:894) 'unicode' object has no attribute 'get'
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/novaclient/shell.py", line 892, in main
    OpenStackComputeShell().main(argv)
  File "/usr/local/lib/python2.7/dist-packages/novaclient/shell.py", line 726, in main
    api_version = api_versions.discover_version(self.cs, api_version)
  File "/usr/local/lib/python2.7/dist-packages/novaclient/api_versions.py", line 267, in discover_version
    client)
  File "/usr/local/lib/python2.7/dist-packages/novaclient/api_versions.py", line 248, in _get_server_version_range
    version = client.versions.get_current()
  File "/usr/local/lib/python2.7/dist-packages/novaclient/v2/versions.py", line 83, in get_current
    return self._get_current()
  File "/usr/local/lib/python2.7/dist-packages/novaclient/v2/versions.py", line 57, in _get_current
    return self._get(url, "version")
  File "/usr/local/lib/python2.7/dist-packages/novaclient/base.py", line 297, in _get
    _resp, body = self.api.client.get(url)
  File "/usr/local/lib/python2.7/dist-packages/keystoneauth1/adapter.py", line 173, in get
    return self.request(url, 'GET', **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/novaclient/client.py", line 92, in request
    raise exceptions.from_response(resp, body, url, method)
  File "/usr/local/lib/python2.7/dist-packages/novaclient/exceptions.py", line 274, in from_response
    message = error.get('message')
AttributeError: 'unicode' object has no attribute 'get'
ERROR (AttributeError): 'unicode' object has no attribute 'get'

Since nova-api must be restarted for this to either take effect or go away (with the WebOb downgrade) this appears to be an issue with the nova-api process. The failing message from the api service is 112 bytes long while the successful one (using downgraded WebOb) is 52 bytes long.

[Regression Potential]
The regression potential is minimal. The patch is cherry-picked without change from stable/newton to stable/mitaka, and I've already tested it in a ppa.

Changed in nova:
importance: Undecided → Medium
tags: added: api
Matt Riedemann (mriedem) wrote :

I think we can confirm this, we're currently testing webob 1.5.1 in the gate but the upper-constraints proposal to include webob 1.6.0 is failing on the same thing:

http://logs.openstack.org/49/288849/15/check/gate-tempest-dsvm-full/4722c5a/logs/devstacklog.txt.gz#_2016-03-18_07_06_03_369

Changed in nova:
status: New → Confirmed
Changed in python-novaclient:
status: New → Confirmed
importance: Undecided → High
Changed in nova:
status: Confirmed → Invalid
Matt Riedemann (mriedem) wrote :

The problem is in python-novaclient.

Matt Riedemann (mriedem) wrote :

Looks like this is the breaking change in webob 1.6.0:

https://github.com/Pylons/webob/pull/139

As you can see in the response body:

RESP BODY: {"message": "The resource could not be found.<br /><br />\n\n\n", "code": "404 Not Found", "title": "Not Found"}

The error message has html in it.

Matt Riedemann (mriedem) wrote :
Matt Riedemann (mriedem) wrote :

The failure is here:

https://github.com/openstack/python-novaclient/blob/3.3.0/novaclient/exceptions.py#L298

Apparently our assumptions about the body there are wrong now.

Matt Riedemann (mriedem) wrote :

So given the error debug above, the body is this:

{"message": "The resource could not be found.<br /><br />\n\n\n", "code": "404 Not Found", "title": "Not Found"}

If you take that python-novaclient code, you get:

error = body[list(body)[0]]

>>> body[list(body)[0]]
'The resource could not be found.<br /><br />\n\n\n'

So error is going to be a str rather than a dict. I'm not sure what this was before.

Matt Riedemann (mriedem) wrote :

From what I can tell, this should also be a problem in openstackclient:

https://github.com/openstack/python-openstackclient/blob/2.2.0/openstackclient/common/exceptions.py#L125

And cinderclient:

https://github.com/openstack/python-cinderclient/blob/master/cinderclient/exceptions.py#L183

Unless it has something to do with the content-type on the request/response?

Matt Riedemann (mriedem) wrote :
Phil Hopkins (phil-hopkins-a) wrote :

If you downgrade to WebOb 1.5.1 the problem will not go away until nova-api is restarted, so there is some involvement from the nova api server also.

Matt Riedemann (mriedem) wrote :

I've opened an issue against webob:

https://github.com/Pylons/webob/issues/235

I doubt they are going to change it since it's fixing a long standing behavior issue, but it was a backward incompatible change and probably should have gone out in a major version 2.0 release.

But we'll probably need to cap webob<1.6.0 until the various clients are working with this.

no longer affects: keystoneauth
no longer affects: nova
summary: - WebOb 1.6.0 causes nova api problems
+ Error handling with from_response with webob 1.6.0 results in
+ "AttributeError: 'unicode' object has no attribute 'get'"
summary: - Error handling with from_response with webob 1.6.0 results in
- "AttributeError: 'unicode' object has no attribute 'get'"
+ exceptions.from_response with webob 1.6.0 results in "AttributeError:
+ 'unicode' object has no attribute 'get'"

Here is what a 404 response looks like with novaclient and webob 1.5.1:

DEBUG (session:248) REQ: curl -g -i -X GET http://9.5.126.108:8774/v2.1/e4f68bae118f46fa986e3fd54cbacb1e/flavors/test -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-OpenStack-Nova-API-Version: 2.25" -H "X-Auth-Token: {SHA1}8b87e089d79ad483b178bdf673e9639ffd86d529"
DEBUG (connectionpool:387) "GET /v2.1/e4f68bae118f46fa986e3fd54cbacb1e/flavors/test HTTP/1.1" 404 77
DEBUG (session:277) RESP: [404] Content-Length: 77 X-Compute-Request-Id: req-d9df03b0-4150-4b53-8157-7560ccf39f75 Vary: X-OpenStack-Nova-API-Version Connection: keep-alive X-Openstack-Nova-Api-Version: 2.25 Date: Fri, 18 Mar 2016 22:30:46 GMT Content-Type: application/json; charset=UTF-8
RESP BODY: {"itemNotFound": {"message": "Flavor test could not be found.", "code": 404}}

So the body is a dict of dicts, where I guess the error message keys to the body, which is how the code expected it to work in novaclient.

>>> body = {"itemNotFound": {"message": "Flavor test could not be found.", "code
": 404}}
>>> body[list(body)[0]]
{'message': 'Flavor test could not be found.', 'code': 404}
>>>

But now we get something like:

{"message": "The resource could not be found.<br /><br />\n\n\n", "code": "404 Not Found", "title": "Not Found"}

Note that even with webob 1.5.1 the content-type in the response is application/json, so it doesn't really help us to key off that to handle this. We basically can't trust the format until we require at least webob>=1.6.0.

Fix proposed to branch: master
Review: https://review.openstack.org/294850

Changed in python-novaclient:
assignee: nobody → Matt Riedemann (mriedem)
status: Confirmed → In Progress
Matt Riedemann (mriedem) wrote :

Looks like the apiclient in oslo-incubator handled this awhile ago:

https://github.com/openstack/oslo-incubator/commit/fd8dc0c3ffe38fe64a3c841d87ea820045221770

Which is now used in keystoneauth. And python-keystoneclient is using that code from keystoneauth.

Wenzhi Yu (yuywz) on 2016-03-19
Changed in python-openstackclient:
assignee: nobody → Wenzhi Yu (yuywz)
status: New → In Progress
Bert JW Regeer (bertjwregeer) wrote :

Hey all,

Sorry about the breakage, but I have a couple of questions, especially since Nova's Fault() should be doing the right thing (but clearly it is not):

See:

https://github.com/Pylons/webob/issues/235#issuecomment-198643200

If someone can help me understand the flow on the nova API side then I can make sure that it is a bug in WebOb, because according to my reading WebOb shouldn't even be hitting the code path that generates it's internal JSON response.

Changed in networking-midonet:
assignee: nobody → YAMAMOTO Takashi (yamamoto)
importance: Undecided → Critical
milestone: none → 2.0.0
status: New → In Progress
Bert JW Regeer (bertjwregeer) wrote :

I have run all tests on the nova codebase using tox -e py27,functional with WebOb 1.6.0 and I can not replicate any failure's. Please provide a stacktrace that shows WebOb's exception's generate_response function being called (which is what is causing the wrong JSON to be returned).

Bert JW Regeer (bertjwregeer) wrote :

Referring to #17:

https://github.com/Pylons/webob/blob/1.6-branch/webob/exc.py#L325

This is the function that has to be called to generate the wrong JSON response. With the Fault wrapping that nova does, I can't figure out how it would ever be called. See the Github issue link: https://github.com/Pylons/webob/issues/235 for more information on what I have tried and the analysis.

Ian Cordasco (icordasc) wrote :

Bert, thanks for the offer to help.

So Nova uses this FaultWrapper middleware (https://github.com/openstack/nova/blob/3b2895b960630934f827918d50577f9167110cab/nova/api/openstack/__init__.py#L106) to intercept all exceptions and problems so it can re-wrap them in its Fault class. I wonder if something changed that stopped calling the Fault instance. It seems like we're expecting all of the work that is causing this to happen in the Fault's __call__ (https://github.com/openstack/nova/blob/2bc8128d7793cc72ca2e146de3a092e1fef5033b/nova/api/openstack/wsgi.py#L1061) method and I wonder if something stopped calling it.

Note also that the FaultWrapper middleware *returns* the exception instead of re-raising the exception wrapped in the Fault. Could either of those (or both) be related to this problem?

Alex Xu (xuhj) wrote :

Nova return fault at here https://github.com/openstack/nova/blob/stable/mitaka/nova/wsgi.py#L460 for any resource which didn't register to the route mapper

If we try to request with '-H "Accept: text/plain"' explicitly and with webob 1.6.0, we will get same response as before"

curl -g -i -X GET http://192.168.2.176:8774/v2.1/b90b53ed87d74e19806da34dbaa056c9/ser -H "User-Agent: python-novaclient" -H "Accept: text/plain" -H "X-Auth-Token: 3d30d32d4c3f4c36b268656f070a7e4a"HTTP/1.1 404 Not Found
Content-Length: 52
Content-Type: text/plain; charset=UTF-8
X-Compute-Request-Id: req-12de9dc1-ff6a-4f77-81a3-079a1d4c0221
Date: Tue, 22 Mar 2016 09:58:47 GMT

404 Not Found

The resource could not be found.

Alex Xu (xuhj) wrote :

@Matt, I think we will return the same response even we upgrade webob to 1.6 in the case of "GET /v2.1/e4f68bae118f46fa986e3fd54cbacb1e/flavors/test", due to this is registered resource. The response changed only for the case like "GET /v2.1/b90b53ed87d74e19806da34dbaa056c9/not_registered_resource"

Change abandoned by YAMAMOTO Takashi (<email address hidden>) on branch: master
Review: https://review.openstack.org/294527
Reason: turned out to be unnecessary

Change abandoned by Wenzhi Yu (wenzhi_yu@163.com) on branch: master
Review: https://review.openstack.org/298086
Reason: 'from_response' is never called in code base, should remove it.

Changed in python-openstackclient:
status: In Progress → Invalid

Reviewed: https://review.openstack.org/298617
Committed: https://git.openstack.org/cgit/openstack/python-openstackclient/commit/?id=848d5312fffa358a789a46a8657af3a92cb3ef6f
Submitter: Jenkins
Branch: master

commit 848d5312fffa358a789a46a8657af3a92cb3ef6f
Author: Wenzhi Yu <wenzhi_yu@163.com>
Date: Tue Mar 29 17:20:05 2016 +0800

    Remove unused method 'from_response'

    'openstackclient.common.exceptions.from_response' method is never called
    in openstackclient code base, so we should remove it.

    Change-Id: I04254a4e66863942e6c273d77bbd66ce2ce7804c
    Related-Bug: #1559072

Reviewed: https://review.openstack.org/294850
Committed: https://git.openstack.org/cgit/openstack/python-novaclient/commit/?id=fa377e7fca44e81247639efebd7cae5cb377e57e
Submitter: Jenkins
Branch: master

commit fa377e7fca44e81247639efebd7cae5cb377e57e
Author: Matt Riedemann <email address hidden>
Date: Fri Mar 18 19:17:47 2016 -0400

    Handle error response for webob>=1.6.0

    WebOb change https://github.com/Pylons/webob/pull/230 changed
    the way in which the error response body is formatted such that
    it's no longer a nested dict. So we have to handle both the
    old convention of an error message key to the response body error
    dict and the new way with just the error body dict.

    This was reported upstream:

    https://github.com/Pylons/webob/issues/235

    But given this was apparently implemented as a long-overdue change
    in WebOb the behavior is not likely to change.

    Change-Id: If653a247d842786d2824b4b3a5c0cde1383ed7ab
    Closes-Bug: #1559072

Changed in python-novaclient:
status: In Progress → Fix Released

all relevant patches are merged

Changed in networking-midonet:
status: In Progress → Fix Released

This issue was fixed in the openstack/python-novaclient 3.4.0 release.

Cao ShuFeng (caosf-fnst) on 2016-04-11
Changed in python-cinderclient:
assignee: nobody → Cao ShuFeng (caosf-fnst)
Cao ShuFeng (caosf-fnst) on 2016-04-11
Changed in python-cinderclient:
assignee: Cao ShuFeng (caosf-fnst) → nobody

Change abandoned by Alexis Lee (<email address hidden>) on branch: stable/mitaka
Review: https://review.openstack.org/307021
Reason: cool, thank you

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in ubuntu:
status: New → Confirmed
Scott Moser (smoser) on 2016-06-01
no longer affects: ubuntu
no longer affects: Ubuntu Yakkety
no longer affects: Ubuntu Xenial
Changed in python-novaclient (Ubuntu):
status: New → Confirmed
Changed in python-novaclient (Ubuntu Xenial):
status: New → Confirmed
importance: Undecided → High
Changed in python-novaclient (Ubuntu Yakkety):
importance: Undecided → High
James Page (james-page) wrote :

Fixes in 3.4.0; Marking yakkety task as fix released.

Changed in python-novaclient (Ubuntu Yakkety):
status: Confirmed → Fix Released
James Page (james-page) wrote :

Unable to reproduce on Xenial; Xenial has webob 1.5.1 so I don't think its likely to be impacted.

Changed in python-novaclient (Ubuntu Xenial):
status: Confirmed → Invalid
James Page (james-page) wrote :

Re-opening Xenial task - accessing a cloud using webob 1.6.0 with older novaclient from 16.04 will fail with this problem; bearing this in mind we might want to consider resurrecting the stable/mitaka novaclient fix as well.

Changed in python-novaclient (Ubuntu Xenial):
status: Invalid → New
Ryan Beisner (1chb1n) wrote :

In trying to create a nova flavor from a Xenial machine against a Xenial-Newton deployment, I see:

00:25:25.978 + nova flavor-create m1.cirros 6 64 1 1
00:25:27.740 ERROR (AttributeError): 'unicode' object has no attribute 'get'

Ryan Beisner (1chb1n) wrote :

jenkins@juju-osci1-machine-8:~/temp$ apt-cache policy python-novaclient
python-novaclient:
  Installed: 2:3.3.1-2
  Candidate: 2:3.3.1-2
  Version table:
 *** 2:3.3.1-2 500
        500 http://nova.clouds.archive.ubuntu.com/ubuntu xenial/main amd64 Packages
        100 /var/lib/dpkg/status

Corey Bryant (corey.bryant) wrote :

I just tested python-novaclient 2:3.3.1-2 on xenial with the abandoned upstream patch successfully.

I've re-proposed the stable/mitaka patch for this since this is an issue when the stable/mitaka python-novaclient is used with newton+ nova deployments.

summary: - exceptions.from_response with webob 1.6.0 results in "AttributeError:
- 'unicode' object has no attribute 'get'"
+ [SRU] exceptions.from_response with webob 1.6.0 results in
+ "AttributeError: 'unicode' object has no attribute 'get'"
Changed in python-novaclient (Ubuntu Xenial):
status: New → Fix Committed
status: Fix Committed → New
description: updated

Hello Phil, or anyone else affected,

Accepted python-novaclient into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/python-novaclient/2:3.3.1-2ubuntu1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in python-novaclient (Ubuntu Xenial):
status: New → Fix Committed
tags: added: verification-needed

Change abandoned by Corey Bryant (<email address hidden>) on branch: stable/mitaka
Review: https://review.openstack.org/378671
Reason: Abandoned in favor of: https://review.openstack.org/307021

Reviewed: https://review.openstack.org/307021
Committed: https://git.openstack.org/cgit/openstack/python-novaclient/commit/?id=70c35e6e3d2aed9f37d5101dafd22c5905ec8583
Submitter: Jenkins
Branch: stable/mitaka

commit 70c35e6e3d2aed9f37d5101dafd22c5905ec8583
Author: Matt Riedemann <email address hidden>
Date: Fri Mar 18 19:17:47 2016 -0400

    Handle error response for webob>=1.6.0

    WebOb change https://github.com/Pylons/webob/pull/230 changed
    the way in which the error response body is formatted such that
    it's no longer a nested dict. So we have to handle both the
    old convention of an error message key to the response body error
    dict and the new way with just the error body dict.

    This was reported upstream:

    https://github.com/Pylons/webob/issues/235

    But given this was apparently implemented as a long-overdue change
    in WebOb the behavior is not likely to change.

    Change-Id: If653a247d842786d2824b4b3a5c0cde1383ed7ab
    Closes-Bug: #1559072

tags: added: in-stable-mitaka
Sean McGinnis (sean-mcginnis) wrote :

This has been fixed in webob 1.6.1:

http://docs.webob.org/en/stable/changes.html

Changed in python-cinderclient:
status: New → Invalid
Corey Bryant (corey.bryant) wrote :

Fix tested successfully on xenial-proposed and mitaka-proposed.

tags: added: verification-done
removed: verification-needed
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package python-novaclient - 2:3.3.1-2ubuntu1

---------------
python-novaclient (2:3.3.1-2ubuntu1) xenial; urgency=medium

  * d/p/enable-1.6.0-webob.patch: Cherry pick from stable/mitaka gerrit review
    to enable use of python-novaclient with newton+ nova deployments that have
    python-webob >= 1.6.0 (LP: #1559072).

 -- Corey Bryant <email address hidden> Wed, 28 Sep 2016 10:44:36 -0400

Changed in python-novaclient (Ubuntu Xenial):
status: Fix Committed → Fix Released

The verification of the Stable Release Update for python-novaclient has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Akira KAMIO (kamio) on 2016-12-06
Changed in python-cinderclient:
assignee: nobody → Akira KAMIO (akamio)
Changed in python-cinderclient:
status: Invalid → In Progress
Eric Harney (eharney) wrote :

Reviewed: https://review.openstack.org/407422
Committed: https://git.openstack.org/cgit/openstack/python-cinderclient/commit/?id=19befa69651a41534c7d25b1a0d3a0cd407afb44
Submitter: Jenkins
Branch: master

commit 19befa69651a41534c7d25b1a0d3a0cd407afb44
Author: Akira KAMIO <email address hidden>
Date: Tue Dec 6 14:10:38 2016 +0900

    Handle error response for webob>=1.6.0

    WebOb change https://github.com/Pylons/webob/pull/230 changed
    the way in which the error response body is formatted such that
    it's no longer a nested dict. So we have to handle both the
    old convention of an error message key to the response body error
    dict and the new way with just the error body dict.

    This was reported upstream:

    https://github.com/Pylons/webob/issues/235

    But given this was apparently implemented as a long-overdue change
    in WebOb the behavior is not likely to change.Handle error response for
    webob>=1.6.0

    Change-Id: I7d589415aa024588faf77c8234ac026110f6c3cd
    Closes-Bug: #1559072

Changed in python-cinderclient:
status: In Progress → Fix Released

This issue was fixed in the openstack/python-cinderclient 1.10.0 release.

Simon Leinen (simon-leinen) wrote :

A comment about the new review 462204: I requested a backport of the cinderclient patch (407422) to Newton. We recently upgraded to Newton, and a user is now unable to delete Heat stacks that contain Cinder volumes that have already been removed. Applying the cinderclient patch fixes this problem.

Reviewed: https://review.openstack.org/462204
Committed: https://git.openstack.org/cgit/openstack/python-cinderclient/commit/?id=b20d9fd6563c1ef295e8a260742f2b3e0b8ee327
Submitter: Jenkins
Branch: stable/newton

commit b20d9fd6563c1ef295e8a260742f2b3e0b8ee327
Author: Akira KAMIO <email address hidden>
Date: Tue Dec 6 14:10:38 2016 +0900

    Handle error response for webob>=1.6.0

    WebOb change https://github.com/Pylons/webob/pull/230 changed
    the way in which the error response body is formatted such that
    it's no longer a nested dict. So we have to handle both the
    old convention of an error message key to the response body error
    dict and the new way with just the error body dict.

    This was reported upstream:

    https://github.com/Pylons/webob/issues/235

    But given this was apparently implemented as a long-overdue change
    in WebOb the behavior is not likely to change.Handle error response for
    webob>=1.6.0

    Change-Id: I7d589415aa024588faf77c8234ac026110f6c3cd
    Closes-Bug: #1559072
    (cherry picked from commit 19befa69651a41534c7d25b1a0d3a0cd407afb44)

tags: added: in-stable-newton
Volodymyr Litovka (doka.ua) wrote :
Download full text (5.0 KiB)

Openstack: stable/ocata
Ubuntu 16.04.02 Xenial Xerus

$ openstack server list
+--------------------------------------+--------+-----------+
| ID | Name | Status |
+--------------------------------------+--------+-----------+
| 54b2465f-6a5d-467e-8815-42c436852ba6 | jex-n1 | ACTIVE |
| a0442c11-dc5b-4d8f-9e77-a83b59782b7c | jex-n2 | SUSPENDED |
+--------------------------------------+--------+-----------+

$ openstack server rebuild jex-n1
'unicode' object has no attribute 'get'

====== --debug -vvvv =======
GET call to compute for http://controller:8774/v2.1/servers?name=jex-n1 used request id req-def51157-4adb-4c37-9ccf-77a8124fa939
REQ: curl -g -i -X GET http://controller:8774/v2.1/servers/54b2465f-6a5d-467e-8815-42c436852ba6 -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}7970e0bb2968bd69cc6f49026ba350e55da7b105"
http://controller:8774 "GET /v2.1/servers/54b2465f-6a5d-467e-8815-42c436852ba6 HTTP/1.1" 200 1782
RESP: [200] Content-Length: 1782 Content-Type: application/json Openstack-Api-Version: compute 2.1 X-Openstack-Nova-Api-Version: 2.1 Vary: OpenStack-API-Version, X-OpenStack-Nova-API-Version X-Compute-Request-Id: req-4e02c928-6dbc-4f79-9cc9-1ca0ad8e69e3 Date: Mon, 05 Jun 2017 10:44:16 GMT
RESP BODY: {"server": {"OS-EXT-STS:task_state": null, "addresses": {"e-net": [{"OS-EXT-IPS-MAC:mac_addr": "d0:1c:a0:04:a4:5d", "version": 4, "addr": "51.255.0.248", "OS-EXT-IPS:type": "fixed"}], "jex-net": [{"OS-EXT-IPS-MAC:mac_addr": "d0:1c:a0:74:f8:57", "version": 4, "addr": "1.1.1.10", "OS-EXT-IPS:type": "fixed"}], "dummy-net": [{"OS-EXT-IPS-MAC:mac_addr": "d0:1c:a0:8c:ae:f2", "version": 4, "addr": "25.0.0.7", "OS-EXT-IPS:type": "fixed"}]}, "links": [{"href": "http://controller:8774/v2.1/servers/54b2465f-6a5d-467e-8815-42c436852ba6", "rel": "self"}, {"href": "http://controller:8774/servers/54b2465f-6a5d-467e-8815-42c436852ba6", "rel": "bookmark"}], "image": "", "OS-EXT-STS:vm_state": "active", "OS-EXT-SRV-ATTR:instance_name": "instance-0000016f", "OS-SRV-USG:launched_at": "2017-06-05T08:53:58.000000", "flavor": {"id": "d0ff4bc5-df38-4f20-8908-afc516d594e6", "links": [{"href": "http://controller:8774/flavors/d0ff4bc5-df38-4f20-8908-afc516d594e6", "rel": "bookmark"}]}, "id": "54b2465f-6a5d-467e-8815-42c436852ba6", "security_groups": [{"name": "default"}, {"name": "jex-esg"}], "user_id": "0d01892c43b0498198c4716d510c6667", "OS-DCF:diskConfig": "MANUAL", "accessIPv4": "", "accessIPv6": "", "progress": 0, "OS-EXT-STS:power_state": 1, "OS-EXT-AZ:availability_zone": "nova", "metadata": {}, "status": "ACTIVE", "updated": "2017-06-05T08:53:59Z", "hostId": "4c5f24a68ef88f97beb99c3e34d5ea6cba6d3fabe3dc5a6487ed8415", "OS-EXT-SRV-ATTR:host": "ardbeg", "OS-SRV-USG:terminated_at": null, "key_name": null, "OS-EXT-SRV-ATTR:hypervisor_hostname": "compute0.cloud.local", "name": "jex-n1", "created": "2017-06-05T08:53:42Z", "tenant_id": "d8051a3ff3ad4c4bb380f828992b8178", "os-extended-volumes:volumes_attached": [{"id": "d76e23e8-c6cb-462e-9ac0-81b2d733fc77"}], "config_drive": ""}}
GET call to compute for http://controller:8774/v2.1/servers/54b2465f-6a5d-467e-8815-42c436852ba6 used re...

Read more...

This issue was fixed in the openstack/python-novaclient 3.3.3 release.

Corey Bryant (corey.bryant) wrote :

yakkety is EOL

Changed in python-cinderclient (Ubuntu):
status: New → Fix Released
Changed in python-cinderclient (Ubuntu Yakkety):
status: New → Won't Fix
Changed in python-cinderclient (Ubuntu Xenial):
assignee: nobody → Saverio Pepe (saverio)
assignee: Saverio Pepe (saverio) → Saverio Proto (zioproto)
status: New → In Progress
importance: Undecided → High
Changed in python-cinderclient (Ubuntu):
importance: Undecided → High
Changed in python-cinderclient (Ubuntu Zesty):
status: New → Fix Released
importance: Undecided → High
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in python-novaclient (Ubuntu Zesty):
status: New → Confirmed
Changed in python-novaclient (Ubuntu Zesty):
status: Confirmed → Fix Released
Corey Bryant (corey.bryant) wrote :

I've uploaded python-cinderclient to the xenial review queue and directly to the newton cloud archive staging PPA.

Hello Phil, or anyone else affected,

Accepted python-cinderclient into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/python-cinderclient/1:1.6.0-2ubuntu1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in python-cinderclient (Ubuntu Xenial):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-xenial
removed: verification-done
James Page (james-page) wrote :

Hello Phil, or anyone else affected,

Accepted python-cinderclient into newton-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:newton-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-newton-needed to verification-newton-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-newton-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

tags: added: verification-newton-needed
Saverio Proto (zioproto) wrote :

I tested the package on Newton. It LGTM. Thank you

James Page (james-page) wrote :

Hello Phil, or anyone else affected,

Accepted python-cinderclient into mitaka-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:mitaka-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-mitaka-needed to verification-mitaka-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-mitaka-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

tags: added: verification-mitaka-needed
James Page (james-page) wrote :

@zioproto

Please could you update the bug tags inline with the acceptance comments when you've completed testing.

Saverio Proto (zioproto) on 2017-07-25
tags: added: verification-newton-done
removed: verification-newton-needed

The verification of the Stable Release Update for python-cinderclient has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

James Page (james-page) wrote :

This bug was fixed in the package python-cinderclient - 1:1.9.0-0ubuntu1~cloud2
---------------

 python-cinderclient (1:1.9.0-0ubuntu1~cloud2) xenial-newton; urgency=medium
 .
   * No change. Upload new version as last version was rejected (LP: #1559072).

Corey Bryant (corey.bryant) wrote :

I've verified that python-cinderclient is fixed on xenial-proposed: http://paste.ubuntu.com/25283545/

Regression tests have also passed successfully.

tags: added: verification-done-xenial
removed: verification-needed-xenial
Corey Bryant (corey.bryant) wrote :

I've also verified against the same cloud (with web>=1.6.0) via a trusty machine with the mitaka-proposed cloud archive: http://paste.ubuntu.com/25283602/

tags: added: verification-done verification-mitaka-done
removed: verification-mitaka-needed verification-needed
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package python-cinderclient - 1:1.6.0-2ubuntu1

---------------
python-cinderclient (1:1.6.0-2ubuntu1) xenial; urgency=medium

  * d/p/review-462204.patch: Handle error response for webob>=1.6.0
    (LP: #1559072).

 -- Saverio Proto <email address hidden> Mon, 26 Jun 2017 11:55:31 +0000

Changed in python-cinderclient (Ubuntu Xenial):
status: Fix Committed → Fix Released
Ryan Beisner (1chb1n) wrote :

The verification of the Stable Release Update for python-cinderclient has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Ryan Beisner (1chb1n) wrote :

This bug was fixed in the package python-cinderclient - 1:1.6.0-2ubuntu1~cloud0
---------------

 python-cinderclient (1:1.6.0-2ubuntu1~cloud0) trusty-mitaka; urgency=medium
 .
   * New update for the Ubuntu Cloud Archive.
 .
 python-cinderclient (1:1.6.0-2ubuntu1) xenial; urgency=medium
 .
   * d/p/review-462204.patch: Handle error response for webob>=1.6.0
     (LP: #1559072).

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.