http header value must be a string

[Bjørnar Ness]

https://github.com/openstack/cinder/commit/26ac08ca79bb9e69d63a4a3cead567556bb31c97#diff-bbd9cce0fac828a3b6b9996c87a2e83eR1155

webob/response.py", line 1041, in __call__
2016-02-28 16:42:04.596 50 ERROR cinder start_response(self.status, headerlist)

TypeError: http header value must be a string

Debugging headerlist gives:

[('Content-Type', 'application/json'), ('x-compute-request-id', b'req-091d3fac-ec78-4cdd-ab9e-ebc5f1ccd7fa'), ('Content-Length', "b'203'"), ('OpenStack-Volume-microversion', '2.0'), ('Vary', 'OpenStack-Volume-microversion')]

[Michal Dulko (michal-dulko-f)]

I've assigned the bug to Scott to notify him about it.

Victor Stinner (IRC: haypo) may also be interested in helping with Python 3 problems.

[Scott DAngelo (scott-dangelo)]

Bjornar, can you please give some information on how to reproduce this bug? Cinder does not fully support python3. Did you find this in Devstack?
You had mentioned in IRC that this repros with:
cinder-manage service list

Was there anything else that reproduced this?
thanks.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/286821

[Matthew Thode (prometheanfire)]

I ran into this with the python3.4 version of wsgi, 2.7 works.

2016-03-29 12:57:57.355 22885 CRITICAL cinder [req-d1efa6e4-9fd0-42c7-8122-e732b9fa71b3 eb7a9d71345241fdb524bc0590b71d94 48ddb9bf27c342e8a9640fe4e526519f - - -] TypeError: http header value must be a string
2016-03-29 12:57:57.355 22885 ERROR cinder Traceback (most recent call last):
2016-03-29 12:57:57.355 22885 ERROR cinder File "/usr/lib64/python3.4/site-packages/paste/urlmap.py", line 216, in __call__
2016-03-29 12:57:57.355 22885 ERROR cinder return app(environ, start_response)
2016-03-29 12:57:57.355 22885 ERROR cinder File "/usr/lib64/python3.4/site-packages/webob/dec.py", line 144, in __call__
2016-03-29 12:57:57.355 22885 ERROR cinder return resp(environ, start_response)
2016-03-29 12:57:57.355 22885 ERROR cinder File "/usr/lib64/python3.4/site-packages/webob/response.py", line 1021, in __call__
2016-03-29 12:57:57.355 22885 ERROR cinder start_response(self.status, headerlist)
2016-03-29 12:57:57.355 22885 ERROR cinder TypeError: http header value must be a string
2016-03-29 12:57:57.355 22885 ERROR cinder

[Matthew Thode (prometheanfire)]

I should probably also state this occurred with stable/mitaka

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/286821
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=c20f335feee62328058a428803e84319a476600c
Submitter: Jenkins
Branch: master

commit c20f335feee62328058a428803e84319a476600c
Author: scottda <email address hidden>
Date: Tue Mar 1 13:21:01 2016 -0700

    http header value must be a string

    Change the logic of how we encode the response headers to be
    a utf-8 string for pre- and post python3.

    Change-Id: I8c9484e436b6f0a2a5b7594c6f1c83441ddbd6db
    Closes-Bug: #1550951

[Scott DAngelo (scott-dangelo)]

Bjørnar or Mathhew, can you please have a look at:
https://review.openstack.org/#/c/304462

It is a slight change to the fix in this patch, and it would be great if you could test it on your Python3 system.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/313183

[OpenStack Infra (hudson-openstack)]

Change abandoned by Matthew Thode (<email address hidden>) on branch: stable/mitaka
Review: https://review.openstack.org/313183

[Scott DAngelo (scott-dangelo)]

Mathew confirmed that testing showed https://review.openstack.org/#/c/304462 worked.

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/cinder 9.0.0.0b1 development milestone.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/358102
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=23421084a515144299b09adab9c0f0aee9635b3f
Submitter: Jenkins
Branch: stable/mitaka

commit 23421084a515144299b09adab9c0f0aee9635b3f
Author: Ivan Kolodyazhny <email address hidden>
Date: Tue Apr 12 12:01:39 2016 +0300

    Use utils.convert_str to convert HTTP header values

    HTTP header values must be a string, Originally fixed with:
     Change the logic of how we encode the response headers to be
      a utf-8 string for pre- and post python3.
     https://review.openstack.org/#/c/286821
     Closes-Bug: #1550951

    That code was changed:
     Use utils.convert_str to convert HTTP header values
     This patch removes duplicate code between wsgi.py and utils.py modules.
     https://review.openstack.org/#/c/304462

    and then fixed:
     Use to_utf8() instead of safe_encode() in convert_str()
     safe_encode() convert string using locale [1]. To avoid possible confuses
     we should use to_utf8() method instead.
     [1] https://wiki.openstack.org/wiki/Python3#safe_encode
     https://review.openstack.org/#/c/313521

    Change-Id: Ic3eefc2bc28ff202324e7859102b289aac7974d4