TypeError: an integer is required

[JPascal]

2013-08-16 13:20:10 ERROR [keystone.common.wsgi] an integer is required
Traceback (most recent call last):
  File "/opt/keystone/keystone/common/wsgi.py", line 372, in __call__
    response = self.process_request(request)
  File "/opt/keystone/keystone/middleware/core.py", line 111, in process_request
    params_json = request.body
  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 677, in _body__get
    self.make_body_seekable() # we need this to have content_length
  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 922, in make_body_seekable
    self.copy_body()
  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 938, in copy_body
    self.body = self.body_file_raw.read()
  File "/opt/keystone/keystone/common/utils.py", line 261, in read
    result = self.data.read(i)
TypeError: an integer is required

[Wu Wenxiang (wu-wenxiang)]

Could you give more information about this issue?
For example:
1. Which operation trigger this issue?
2. How to reproduce?
3. Which version?

[JPascal (eshurmin)]

Version:
https://github.com/openstack/keystone/tree/2013.2.b2

Reproduce:
# request
curl -H "Content-type: application/json" -H "X-Auth-Token:$token" -d '{"tenant":{"enabled":true, "name":"admin"}}' http://localhost:35357/v2.0/tenants
# response
{"tenants_links": [], "tenants": [{"description": "", "enabled": true, "id": "abc0a6f227c740859f0fd3410ef310e9", "name": "admin"}]}

# request
curl -H "Content-type: application/json" -H "X-Auth-Token:$token" -d '{"user":{"tenant_id":"abc0a6f227c740859f0fd3410ef310e9", "password":"crowbar", "enabled":true, "name":"admin"}}' http://localhost:35357/v2.0/users
#response
{"user": {"name": "admin", "tenant_id": "abc0a6f227c740859f0fd3410ef310e9", "enabled": true, "id": "4ab39dacc7f849cebe789b640b788054"}}

#request
curl -H "Content-type: application/json" -H "X-Auth-Token:$token" -d '{"role":{"name":"admin"}}' http://localhost:35357/v2.0/OS-KSADM/roles
#response
{"role": {"id": "f10e95604b994df28bcada736afdae82", "name": "admin"}}

#request
curl -X PUT -H "Content-type: application/json" -H "X-Auth-Token:$token" http://localhost:35357/v2.0/tenants/$tenant_id/users/$user_id/roles/OS-KSADM/$role_id
#response
{"error": {"message": "an integer is required", "code": 400, "title": "Bad Request"}}

If I change https://github.com/openstack/keystone/blob/2013.2.b2/keystone/common/utils.py#L260 to
"def read(self, i=0):" this work correctly.

[Dolph Mathews (dolph)]

I'm not able to reproduce this. If this is still reproducible, please comment!

[JPascal (eshurmin)]

Hello. Long time can't look again on this bug.
Bug still exists in repo with tag 2013.2.b2

[JPascal (eshurmin)]

Also addition information.
If I run keystone as native, all work correctly, if I run using apache it bug exists.

[Dolph Mathews (dolph)]

Re-opening based on the above; I didn't test this behind apache.

[Brant Knudson (blk-u)]

Looking at current keystone/common/utils.py, https://github.com/openstack/keystone/blob/master/keystone/common/utils.py#L261, line 268 is not a read()... there's a read at line 293. Why would it be so far off?

[Dolph Mathews (dolph)]

I believe it's referring to LimitingReader.read(), here's a permalink:

  https://github.com/openstack/keystone/blob/dc48eee9d2519191c58d97fc48e3881925897275/keystone/common/utils.py#L292-L297

[xingzhou (xingzhou)]

I will be out of the office starting 2013/10/01 and will not return until
2013/10/07.

I will check mails at night and try to respond you ASAP

[JPascal (eshurmin)]

I work with git usung tag 2013.2.b2, not master branch.
Looking at current keystone/common/utils.py,
https://github.com/openstack/keystone/blob/master/keystone/common/utils.py#L261
,
line 268 is not a read()... there's a read at line 293. Why would it be
so far off?

--
You received this bug notification because you are subscribed to the bug
report.
https://bugs.launchpad.net/bugs/1213106

Title:
  TypeError: an integer is required

Status in OpenStack Identity (Keystone):
  New

Bug description:
  2013-08-16 13:20:10 ERROR [keystone.common.wsgi] an integer is required
  Traceback (most recent call last):
    File "/opt/keystone/keystone/common/wsgi.py", line 372, in __call__
      response = self.process_request(request)
    File "/opt/keystone/keystone/middleware/core.py", line 111, in
process_request
      params_json = request.body
    File "/usr/lib/python2.7/dist-packages/webob/request.py", line 677, in
_body__get
      self.make_body_seekable() # we need this to have content_length
    File "/usr/lib/python2.7/dist-packages/webob/request.py", line 922, in
make_body_seekable
      self.copy_body()
    File "/usr/lib/python2.7/dist-packages/webob/request.py", line 938, in
copy_body
      self.body = self.body_file_raw.read()
    File "/opt/keystone/keystone/common/utils.py", line 261, in read
      result = self.data.read(i)
  TypeError: an integer is required

To manage notifications about this bug go to:
https://bugs.launchpad.net/keystone/+bug/1213106/+subscriptions

[Jamie Lennox (jamielennox)]

Confirmed behind apache in master - investingating

[Jamie Lennox (jamielennox)]

Firstly this is being caused because when you issue the curl command for a PUT method without a data object it does not set a Content-Length header. If you are working with older keystones you can fix this by including a -H "Content-Length: 0" statement to your command line.

What appears to be happening is that the eventlet and mod_wsgi implementations of wsgi.Input differ. The wsgi PEP http://www.python.org/dev/peps/pep-0333 specifies you must be able to read and that you it takes an optional limit argument but it does not specify the expected default value for the limit if not provided.

None, the value we have been using works as a default for eventlet. I can't find actual documentation but -1 appears to do the same thing for mod_wsgi, but passing the wrong default value leads to this failure and something similar in eventlet.

The fix should be fairly simple, just check if a limit value was provided and only provide the optional argument to the real read if provided.

[Dolph Mathews (dolph)]

Great analysis!

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/50524

[Thierry Carrez (ttx)]

Looks like a corner case to me, so downgrading prio

[Thierry Carrez (ttx)]

That corner case is the advertised way of doing it, so back to High, and RC2-blocking

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: milestone-proposed
Review: https://review.openstack.org/50664

[Dolph Mathews (dolph)]

backport to milestone-proposed: https://review.openstack.org/#/c/50664/

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/50524
Committed: http://github.com/openstack/keystone/commit/12fd2aa50096b77c044ca8840657de5f2c16513e
Submitter: Jenkins
Branch: master

commit 12fd2aa50096b77c044ca8840657de5f2c16513e
Author: Jamie Lennox <email address hidden>
Date: Wed Oct 9 11:49:58 2013 +1000

    Don't use default value in LimitingReader

    We can't simply pass the None default on to the read operation as this
    default is handled differently between different wsgi implementations.

    Change-Id: I337e797b8dee3dfcf9299fe361cf197a176c8fe2
    Fixes: bug 1213106

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/50664
Committed: http://github.com/openstack/keystone/commit/c14ebd668f00b1d0c0f3c94fa331b2080d5ec053
Submitter: Jenkins
Branch: milestone-proposed

commit c14ebd668f00b1d0c0f3c94fa331b2080d5ec053
Author: Jamie Lennox <email address hidden>
Date: Wed Oct 9 11:49:58 2013 +1000

    Don't use default value in LimitingReader

    We can't simply pass the None default on to the read operation as this
    default is handled differently between different wsgi implementations.

    Change-Id: I337e797b8dee3dfcf9299fe361cf197a176c8fe2
    Fixes: bug 1213106

[Brant Knudson (blk-u)]

The old code without the fix is in oslo.middleware . Ran into this when trying to change Keystone to use oslo.middleware: http://logs.openstack.org/97/144697/1/check/check-tempest-dsvm-full/4a13e44/logs/apache/keystone.txt.gz#_2015-01-01_23_57_43_514 for https://review.openstack.org/#/c/144697/

I ported the fix to oslo.middleware: https://review.openstack.org/#/c/144700/

[Doug Hellmann (doug-hellmann)]

This was fixed in 7c8e3e1061c2b51be59f034e0cd5b914a4ad6f47 and released in 0.3.0