Incompatibility with python-webob 1.7.0

[Chuck Short]

keystone.tests.unit.test_v3_federation.WebSSOTests.test_identity_provider_specific_federated_authentication
-----------------------------------------------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):
      File "keystone/tests/unit/test_v3_federation.py", line 4067, in test_identity_provider_specific_federated_authentication
        self.PROTOCOL)
      File "keystone/federation/controllers.py", line 345, in federated_idp_specific_sso_auth
        return self.render_html_response(host, token_id)
      File "keystone/federation/controllers.py", line 357, in render_html_response
        headerlist=headers)
      File "/usr/lib/python2.7/dist-packages/webob/response.py", line 310, in __init__
        "You cannot set the body to a text value without a "
    TypeError: You cannot set the body to a text value without a charset

[Boris Bobrov (bbobrov)]

https://bugs.launchpad.net/keystonemiddleware/+bug/1653646 -- we fixed a similar issue in keystonemiddleware

[David Stanek (dstanek)]

I'm going to make some fixes to keystone so that when the upper constraints is eventually bumped up to 1.7 that we won't have an issue.

[Corey Bryant (corey.bryant)]

glance failures with webob 1.7.0:

glance.tests.unit.common.test_wsgi.JSONRequestDeserializerTest.test_has_body_valid_transfer_encoding_with_content_length
------------------------------------------------------------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):
      File "glance/tests/unit/common/test_wsgi.py", line 515, in test_has_body_valid_transfer_encoding_with_content_length
        transfer_encoding='chunked', content_length=0))
      File "/usr/lib/python2.7/dist-packages/unittest2/case.py", line 702, in assertTrue
        raise self.failureException(msg)
    AssertionError: False is not true

glance.tests.unit.v1.test_api.TestGlanceAPI.test_add_image_wrong_content_type
-----------------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):
      File "glance/tests/unit/v1/test_api.py", line 2301, in test_add_image_wrong_content_type
        self.assertEqual(http_client.BAD_REQUEST, res.status_int)
      File "/usr/lib/python2.7/dist-packages/testtools/testcase.py", line 350, in assertEqual
        self.assertThat(observed, matcher, message)
      File "/usr/lib/python2.7/dist-packages/testtools/testcase.py", line 435, in assertThat
        raise mismatch_error
    testtools.matchers._impl.MismatchError: 400 != 201

[Corey Bryant (corey.bryant)]

nova failures with webob 1.7.0:

nova.tests.unit.api.openstack.compute.test_microversions.MicroversionsTest.test_microversions_inner_function_v21
----------------------------------------------------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):
      File "nova/tests/unit/api/openstack/compute/test_microversions.py", line 302, in test_microversions_inner_function_v21
        self._test_microversions_inner_function('2.1', 'controller4_val1')
      File "/usr/lib/python2.7/dist-packages/mock/mock.py", line 1305, in patched
        return func(*args, **keywargs)
      File "nova/tests/unit/api/openstack/compute/test_microversions.py", line 291, in _test_microversions_inner_function
        self.assertEqual(200, res.status_int)
      File "/usr/lib/python2.7/dist-packages/testtools/testcase.py", line 350, in assertEqual
        self.assertThat(observed, matcher, message)
      File "/usr/lib/python2.7/dist-packages/testtools/testcase.py", line 435, in assertThat
        raise mismatch_error
    testtools.matchers._impl.MismatchError: 200 != 400

Captured pythonlogging:
~~~~~~~~~~~~~~~~~~~~~~~
    2017-01-18 08:45:09,875 INFO [nova.api.openstack] Loaded extensions: ['test-basic', 'test-microversions']

nova.tests.unit.api.openstack.compute.test_microversions.LegacyMicroversionsTest.test_microversions_inner_function_v21
----------------------------------------------------------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):
      File "nova/tests/unit/api/openstack/compute/test_microversions.py", line 302, in test_microversions_inner_function_v21
        self._test_microversions_inner_function('2.1', 'controller4_val1')
      File "/usr/lib/python2.7/dist-packages/mock/mock.py", line 1305, in patched
        return func(*args, **keywargs)
      File "nova/tests/unit/api/openstack/compute/test_microversions.py", line 291, in _test_microversions_inner_function
        self.assertEqual(200, res.status_int)
      File "/usr/lib/python2.7/dist-packages/testtools/testcase.py", line 350, in assertEqual
        self.assertThat(observed, matcher, message)
      File "/usr/lib/python2.7/dist-packages/testtools/testcase.py", line 435, in assertThat
        raise mismatch_error
    testtools.matchers._impl.MismatchError: 200 != 400

Captured pythonlogging:
~~~~~~~~~~~~~~~~~~~~~~~
    2017-01-18 08:45:13,158 INFO [nova.api.openstack] Loaded extensions: ['test-basic', 'test-microversions']

nova.tests.unit.api.openstack.compute.test_microversions.MicroversionsTest.test_microversions_inner_function_v22
----------------------------------------------------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):
      File "nova/tests/unit/api/openstack/compute/test_microversions.py", line 299, in test_microversions_inner_function_v22
        self._test_microversions_inner_function('2.2', 'controller4_val2')
      File "/usr/lib/python2.7/dist-packages/mock/mock.py", line 1305, i...

[Corey Bryant (corey.bryant)]

Marking this as high at least for ubuntu since this is blocking us from releasing B2 since ubuntu is at webob 1.7.0.

[Corey Bryant (corey.bryant)]

This appears to run deeper than just test failures. See glance bug: https://bugs.launchpad.net/ubuntu/+source/glance/+bug/1657459

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/422234

[Corey Bryant (corey.bryant)]

This appears to also be affecting python-oslo.middleware. Adding test output from dup bug report:

======================================================================
FAIL: oslo_middleware.tests.test_sizelimit.TestRequestBodySizeLimiter.test_request_too_large_no_content_length
tags: worker-7
----------------------------------------------------------------------
Empty attachments:
  stderr
  stdout

Traceback (most recent call last):
  File "oslo_middleware/tests/test_sizelimit.py", line 108, in test_request_too_large_no_content_length
    self.assertEqual(413, response.status_int)
  File "/home/chuck/work/server/openstack/packaging/git/openstack/oslo.middleware/.tox/py27/local/lib/python2.7/site-packages/testtools/testcase.py", line 411, in assertEqual
    self.assertThat(observed, matcher, message)
  File "/home/chuck/work/server/openstack/packaging/git/openstack/oslo.middleware/.tox/py27/local/lib/python2.7/site-packages/testtools/testcase.py", line 498, in assertThat
    raise mismatch_error
testtools.matchers._impl.MismatchError: 413 != 200
Ran 103 tests in 0.217s (-0.074s)
FAILED (id=3, failures=1)
error: testr failed (1)
ERROR: InvocationError:

[Corey Bryant (corey.bryant)]

Appending a few comments that sigmavirus had on bug #1657459:

"I filed https://github.com/Pylons/webob/issues/307 although I doubt anything will come of it upstream. If anything, that might help others."

"Another facet of this for Glance seems to be related to the PR in WebOb that broke other projects: http://paste.openstack.org/show/595699/ I suspect this is due to us returning `webob.exc.HTTPNoContent()`. It looks like our gzip middleware expects there to be a content-type but that's no longer auto-generated for us."

[Corey Bryant (corey.bryant)]

I've lowered the importance from critical to low for Ubuntu. Debian just uploaded python-webob 1:1.6.2-2 to replace 1.7.0-1, which I didn't realize was an option. We'll get 1:1.6.2-2 synced over to zesty and that should solve our webob problems in Ocata.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/422234
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=38bce1d869264a6f1ea0e13a0301c8cc4bfc84af
Submitter: Jenkins
Branch: master

commit 38bce1d869264a6f1ea0e13a0301c8cc4bfc84af
Author: David Stanek <email address hidden>
Date: Fri Mar 24 15:12:27 2017 +0000

    Small fixes for WebOb 1.7 compatibiltity

    WebOb 1.7 includes a few changes that are not backward
    compatible. The ones I noticed and fixed are:
     1. When supplying unicode content for the response body
        the charset is now required if headers are also provided.
        Previously a default was used.
     2. Content-Length is no longer being set when creating a
        webob.Response object. It appears to be correctly set
        when directly setting the body property.

    Upstream change that caused the issues:
      https://github.com/Pylons/webob/commit/35fd585

    Closes-bug: #1657452
    Change-Id: Iaf2dd45fc86e3eb5e56be0a3e1582a6ddf960bc1

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/423366
Committed: https://git.openstack.org/cgit/openstack/glance/commit/?id=7a843f7e1fb1bab504fea0b2c59bf7c22121da71
Submitter: Jenkins
Branch: master

commit 7a843f7e1fb1bab504fea0b2c59bf7c22121da71
Author: Ian Cordasco <email address hidden>
Date: Fri Jan 20 16:38:10 2017 +0000

    Fix incompatibilities with WebOb 1.7

    WebOb 1.7 changed [0] how request bodies are determined to be
    readable. Prior to version 1.7, the following is how WebOb
    determined if a request body is readable:
      #1 Request method is one of POST, PUT or PATCH
      #2 ``content_length`` length is set
      #3 Special flag ``webob.is_body_readable`` is set

    The special flag ``webob.is_body_readable`` was used to signal
    WebOb to consider a request body readable despite the content length
    not being set. #1 above is how ``chunked`` Transfer Encoding was
    supported implicitly in WebOb < 1.7.

    Now with WebOb 1.7, a request body is considered readable only if
    ``content_length`` is set and it's non-zero [1]. So, we are only left
    with #2 and #3 now. This drops implicit support for ``chunked``
    Transfer Encoding Glance relied on. Hence, to emulate #1, Glance must
    set the the special flag upon checking the HTTP methods that may have
    bodies. This is precisely what this patch attemps to do.

    [0] https://github.com/Pylons/webob/pull/283
    [1] https://github.com/Pylons/webob/pull/283/files#diff-706d71e82f473a3b61d95c2c0d833b60R894

    Closes-bug: #1657459
    Closes-bug: #1657452
    Co-Authored-By: Hemanth Makkapati <email address hidden>
    Change-Id: I19f15165a3d664d5f3a361f29ad7000ba2465a85

[Lance Bragstad (lbragstad)]

Here is another related fix that was committed to keystone - https://review.openstack.org/#/c/451559/

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/glance 15.0.0.0b1 development milestone.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/keystone 12.0.0.0b1 development milestone.

[James Page (james-page)]

Marking all Ubuntu tasks as fix released; Ocata UCA shipped with an earlier webob version anyway, and subsequent releases have the required fix.

[Takashi Kajinami (kajinamit)]

I think this was fixed in oslo.middleware by https://review.opendev.org/c/openstack/oslo.middleware/+/453712