oslo.serialization 2.21.2 breaks glance

Bug #1728368 reported by Dirk Mueller
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Glance
Fix Released
High
John L. Villalovos
oslo.serialization
Invalid
Undecided
Unassigned

Bug Description

glance unittests break with 2.21.2, work fine with 2.21.1:

======================================================================
FAIL: glance.tests.unit.v1.test_registry_api.TestRegistryAPI.test_update_all_image_members
----------------------------------------------------------------------
Traceback (most recent call last):
  File "glance/tests/unit/v1/test_registry_api.py", line 1629, in test_update_all_image_members
    content_type='json')
  File "glance/tests/utils.py", line 564, in get_api_response_ext
    self.assertEqual(res.status_int, http_resp)
  File "/data/dmueller/src/Cloud/glance/.tox/venv/lib/python2.7/site-packages/testtools/testcase.py", line 411, in assertEqual
    self.assertThat(observed, matcher, message)
  File "/data/dmueller/src/Cloud/glance/.tox/venv/lib/python2.7/site-packages/testtools/testcase.py", line 498, in assertThat
    raise mismatch_error
testtools.matchers._impl.MismatchError: 200 != 204

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to oslo.serialization (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/534736

Revision history for this message
John L. Villalovos (happycamp) wrote :
Download full text (7.4 KiB)

So in the unit test run using: $ tox -vvv -e py27 -- -r glance.tests.unit.v1.test_registry_api.TestRegistryAPI.test_update_all_image_members

I see that it is trying to serialize an HTTPNoContent exception and I also see a warning that in oslo.serialization 3.0 this will raise a ValueError() instead of printing a warning.

Also see it here: http://logs.openstack.org/72/533872/6/check/openstack-tox-py27/4709e32/job-output.txt.gz#_2018-01-16_13_10_38_931593

<HTTPNoContent at 0x7fe744f7fe60 204 No Content>

  File "/usr/lib64/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/home/devstack/openstack/glance/.tox/py27/lib/python2.7/site-packages/subunit/run.py", line 149, in <module>
    main()
  File "/home/devstack/openstack/glance/.tox/py27/lib/python2.7/site-packages/subunit/run.py", line 145, in main
    stdout=stdout, exit=False)
  File "/home/devstack/openstack/glance/.tox/py27/lib/python2.7/site-packages/testtools/run.py", line 187, in __init__
    self.runTests()
  File "/home/devstack/openstack/glance/.tox/py27/lib/python2.7/site-packages/testtools/run.py", line 221, in runTests
    self.result = testRunner.run(self.test)
  File "/home/devstack/openstack/glance/.tox/py27/lib/python2.7/site-packages/subunit/run.py", line 70, in run
    test(result)
  File "/usr/lib64/python2.7/unittest/suite.py", line 70, in __call__
    return self.run(*args, **kwds)
  File "/usr/lib64/python2.7/unittest/suite.py", line 108, in run
    test(result)
  File "/home/devstack/openstack/glance/.tox/py27/lib/python2.7/site-packages/unittest2/suite.py", line 87, in __call__
    return self.run(*args, **kwds)
  File "/home/devstack/openstack/glance/.tox/py27/lib/python2.7/site-packages/unittest2/suite.py", line 126, in run
    test(result)
  File "/home/devstack/openstack/glance/.tox/py27/lib/python2.7/site-packages/unittest2/suite.py", line 87, in __call__
    return self.run(*args, **kwds)
  File "/home/devstack/openstack/glance/.tox/py27/lib/python2.7/site-packages/unittest2/suite.py", line 126, in run
    test(result)
  File "/home/devstack/openstack/glance/.tox/py27/lib/python2.7/site-packages/unittest2/suite.py", line 87, in __call__
    return self.run(*args, **kwds)
  File "/home/devstack/openstack/glance/.tox/py27/lib/python2.7/site-packages/unittest2/suite.py", line 126, in run
    test(result)
  File "/home/devstack/openstack/glance/.tox/py27/lib/python2.7/site-packages/unittest2/case.py", line 673, in __call__
    return self.run(*args, **kwds)
  File "/home/devstack/openstack/glance/.tox/py27/lib/python2.7/site-packages/testtools/testcase.py", line 671, in run
    return run_test.run(result)
  File "/home/devstack/openstack/glance/.tox/py27/lib/python2.7/site-packages/testtools/runtest.py", line 80, in run
    return self._run_one(actual_result)
  File "/home/devstack/openstack/glance/.tox/py27/lib/python2.7/site-packages/testtools/runtest.py", line 94, in _run_one
    return self._run_prepared_result(ExtendedToOriginalDecorator(result))
  File "/home/devstack/openstack/glance/.tox/py27/lib/python2.7/site-pa...

Read more...

Revision history for this message
John L. Villalovos (happycamp) wrote :

This might fix the issue:

diff --git a/glance/common/wsgi.py b/glance/common/wsgi.py
index 6c8853da..1d763890 100644
--- a/glance/common/wsgi.py
+++ b/glance/common/wsgi.py
@@ -1142,6 +1142,8 @@ class JSONResponseSerializer(object):
             return obj.to_dict()
         if isinstance(obj, multidict.MultiDict):
             return obj.mixed()
+ if isinstance(obj, Exception):
+ return obj
         return jsonutils.to_primitive(obj)

     def to_json(self, data):

Revision history for this message
John L. Villalovos (happycamp) wrote :

So there is a test patch:

https://review.openstack.org/#/c/535166/

And the unit tests pass with it.

Changed in glance:
assignee: nobody → John L. Villalovos (happycamp)
status: New → In Progress
Revision history for this message
John L. Villalovos (happycamp) wrote :
Revision history for this message
John L. Villalovos (happycamp) wrote :

The reason for the previous comment is this:

https://review.openstack.org/#/c/535166/4/glance/common/wsgi.py@1149

So I was digging into this more, but going to stop as 11:30PM my time.

When this returns the obj, it actually causes a Circular reference detected exception to occur. And even without this change and the old oslo.serialization the same exception happens.

So far some reason a CircularReference exception works and returning a string representation of the HTTPNoContent fails the unit tests.

Weird!

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on oslo.serialization (master)

Change abandoned by ChangBo Guo(gcb) (<email address hidden>) on branch: master
Review: https://review.openstack.org/534736

Changed in oslo.serialization:
status: New → Invalid
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to glance (master)

Reviewed: https://review.openstack.org/535166
Committed: https://git.openstack.org/cgit/openstack/glance/commit/?id=6e397345b1b4f5dc9e67cebde817155693f5cf51
Submitter: Zuul
Branch: master

commit 6e397345b1b4f5dc9e67cebde817155693f5cf51
Author: John L. Villalovos <email address hidden>
Date: Wed Jan 17 21:01:29 2018 -0800

    Resolve unit test failures with going to oslo.serialization 2.3.0

    openstack/oslo.serialization made a change in commit
    c1a7079c26d27a2e46cca26963d3d9aa040bdbe8 which changed how
    jsonutils.to_primitive() handles Exception objects. Previously the
    actual Exception object would be returned but now it returns a 'repr'
    of the object. This was done because CircularReferences were occuring
    in some use cases with Exceptions and also in version 3.0 of
    oslo.serialization they were going to raise a ValueError() for
    unhandled types and this change caused Exceptions to now be handled.

    The code previously would call the serialization function and if any
    unknown Exception occurred it would just return action_result. With
    the change to oslo.serialization an exception was no longer being
    raised and thus no Exception would be raised and caught.

    Now check if action_result is an Exception and if so return it.

    Closes-Bug: #1728368
    Change-Id: Ic374ec891a65f603433091cdae27b0d4aac8362f

Changed in glance:
status: In Progress → Fix Released
Changed in glance:
importance: Undecided → High
milestone: none → queens-rc1
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/glance 16.0.0.0rc1

This issue was fixed in the openstack/glance 16.0.0.0rc1 release candidate.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers