UnicodeEncodeError when Running CLI keystone tenant-delete action with non-English characters

Bug #1237892 reported by Chen Xiao on 2013-10-10
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Identity (keystone)
High
Morgan Fainberg

Bug Description

keystone tenant-delete action with non-English characters occurs UnicodeEncodeError in log file, but the tenant has been delete in fact.
input below command:
[[root@osee22-compute02 ˜]# keystone tenant-delete testソ十豹

[root@osee22-compute02 ˜]# keystone tenant-list
+----------------------------------+---------+---------+
| id | name | enabled |
+----------------------------------+---------+---------+
| e7308b169a784693b39c60e152c7d01a | Public | True |
| 9b78773f34264a428073412ef7d5485e | service | True |
+----------------------------------+---------+---------+

log:
2013-09-25 21:06:50.280 3622 TRACE keystone.common.wsgi Traceback (most recent call last):
2013-09-25 21:06:50.280 3622 TRACE keystone.common.wsgi File "/usr/lib/python2.6/site-packages/keystone/common/wsgi.py", line 238, in __call__
2013-09-25 21:06:50.280 3622 TRACE keystone.common.wsgi result = method(context, **params)
2013-09-25 21:06:50.280 3622 TRACE keystone.common.wsgi File "/usr/lib/python2.6/site-packages/keystone/identity/controllers.py", line 124, in delete_project
2013-09-25 21:06:50.280 3622 TRACE keystone.common.wsgi self.assignment_api.delete_project(tenant_id)
2013-09-25 21:06:50.280 3622 TRACE keystone.common.wsgi File "/usr/lib/python2.6/site-packages/keystone/notifications.py", line 44, in wrapper
2013-09-25 21:06:50.280 3622 TRACE keystone.common.wsgi result = f(*args, **kwargs)
2013-09-25 21:06:50.280 3622 TRACE keystone.common.wsgi File "/usr/lib/python2.6/site-packages/keystone/assignment/core.py", line 93, in delete_project
2013-09-25 21:06:50.280 3622 TRACE keystone.common.wsgi project['domain_id'])
2013-09-25 21:06:50.280 3622 TRACE keystone.common.wsgi File "/usr/lib/python2.6/site-packages/dogpile/cache/region.py", line 939, in invalidate
2013-09-25 21:06:50.280 3622 TRACE keystone.common.wsgi key = key_generator(*arg, **kw)
2013-09-25 21:06:50.280 3622 TRACE keystone.common.wsgi File "/usr/lib/python2.6/site-packages/dogpile/cache/util.py", line 86, in generate_key
2013-09-25 21:06:50.280 3622 TRACE keystone.common.wsgi return namespace + "|" + " ".join(map(to_str, args))
2013-09-25 21:06:50.280 3622 TRACE keystone.common.wsgi UnicodeEncodeError: 'ascii' codec can't encode characters in position 4-6: ordinal not in range(128)

Fix proposed to branch: master
Review: https://review.openstack.org/50938

Changed in keystone:
assignee: nobody → Chen Xiao (chenxiao)
status: New → In Progress
Dolph Mathews (dolph) on 2013-10-10
Changed in keystone:
importance: Undecided → Medium
Dolph Mathews (dolph) on 2013-10-10
tags: added: havana-rc-potential
Morgan Fainberg (mdrnstm) wrote :

I've got a lead on a bit more of the core of the issue on this bug (vs. just a simple encode at the method call). This appears to be an issue in the cache key-generator when handling unicode strings, specifically in the stringificaiton (default) mechanism. I am working on a better stringification method for cache-key generation that will be unicode friendly. Once this new mechanism is implemented, use of the to_str argument on the @cache.on_arguments decorator will resolve the issue.

Chen Xiao (chenxiao) wrote :

Hi, Morgan Fainberg,
     Your method to solve this bug is better, you are right, wish your solution, and I want to know whether you change the dogpile.cache code for solve this problem

Dolph Mathews (dolph) wrote :

The cache key generator is configurable in dogpile.cache, so we can provide our own implementation from keystone's side (even if it's just a wrapper for dogpile's implementation).

Morgan Fainberg (mdrnstm) wrote :

This is actually even easier than providing our own cache-key generator, I am working on a replacement for the stringification method that covers this error. The stringify method is passed in to the decorator itself. I have isolated the actual issue and I am working on implementing a clean mechanism to handle this without causing other encode/decode headaches. Even if we provided our own complete cache-key generator, this issue would likely still need to be addressed in a similar manner.

Dolph Mathews (dolph) wrote :

Raising the priority on this issue, as merely disabling caching is not a sufficient workaround (cache keys are still generated, regardless). Additionally, this will affect any cached driver call that expects unicode arguments.

Changed in keystone:
importance: Medium → High
milestone: none → havana-rc2

Fix proposed to branch: master
Review: https://review.openstack.org/51504

Changed in keystone:
assignee: Chen Xiao (chenxiao) → Morgan Fainberg (mdrnstm)
Changed in keystone:
assignee: Morgan Fainberg (mdrnstm) → Dolph Mathews (dolph)
Dolph Mathews (dolph) on 2013-10-14
Changed in keystone:
assignee: Dolph Mathews (dolph) → Morgan Fainberg (mdrnstm)

Reviewed: https://review.openstack.org/51504
Committed: http://github.com/openstack/keystone/commit/a4ee3ae9673d49de35409f8f5512ea4b18e4fcaa
Submitter: Jenkins
Branch: master

commit a4ee3ae9673d49de35409f8f5512ea4b18e4fcaa
Author: Morgan Fainberg <email address hidden>
Date: Sun Oct 13 18:34:24 2013 -0700

    Handle unicode at the caching layer more elegantly

    This patchset resolves an issue where in some cases unicode would
    cause the cache key generator to raise a UnicodeEncodeError due to
    the name/value being outside of the standard ascii character set.
    Included is a fix to the cache backend debug code to utilize repr
    for passing the keys/values to the logger.

    Tests in test_backend provided by chenxiao <email address hidden>

    Closes-bug: 1237892
    Change-Id: I4df1c9eb6b4a1367defdbb6bcbcd1f1f992348e5

Changed in keystone:
status: In Progress → Fix Committed

Reviewed: https://review.openstack.org/51511
Committed: http://github.com/openstack/keystone/commit/a0ae37eb3ea96a394f10d3f38e8b454d2d7a98e0
Submitter: Jenkins
Branch: milestone-proposed

commit a0ae37eb3ea96a394f10d3f38e8b454d2d7a98e0
Author: Morgan Fainberg <email address hidden>
Date: Sun Oct 13 18:34:24 2013 -0700

    Handle unicode at the caching layer more elegantly

    This patchset resolves an issue where in some cases unicode would
    cause the cache key generator to raise a UnicodeEncodeError due to
    the name/value being outside of the standard ascii character set.
    Included is a fix to the cache backend debug code to utilize repr
    for passing the keys/values to the logger.

    Tests in test_backend provided by chenxiao <email address hidden>

    Closes-bug: 1237892
    Change-Id: Ic99503987851128cd41d83ad1ea50dc4a132fbd3

Changed in keystone:
status: Fix Committed → Fix Released
Thierry Carrez (ttx) on 2013-10-17
Changed in keystone:
milestone: havana-rc2 → 2013.2
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers