Tenant does not support non-ascii characters

[Ma, Tianxiao]

[Impact]

Affected to trusty-mitaka, only has older form of user id

Assume that there is unicode user id (like John Gāo) from ldap on trusty-kilo env.
and Upgraded to trusty-mitaka.

I can't command nova list with 500 error and nova-api log show us unicode error like below

UnicodeEncodeError: 'ascii' codec can't encode character u'\xe1' in position 1: ordinal not in range(128)

[Test Case]

TBD for detail script or info

0. proper ldap setting(accounts etc..) with user id has unicode character as above
1. deploy trusty-kilo env
- without ldap config,
- if i set ldap config at the beginning, there is error because by default config blocks CRUD user on ldap)
2. setting ldap configuration
- https://pastebin.ubuntu.com/p/pfzNDPgccd/
3. remove CRUD block on keystone.conf temporarily
- https://pastebin.ubuntu.com/p/HD7mPwHDjM/
4. need to set up password for nova, cinder, etc service accounts
- openstack user set --password-prompt nova ( for all services )
- you can check password in each services configuration file
5. upgrade to trusty-mitaka
- https://pastebin.ubuntu.com/p/d6Wcp7HQVr/
6. source novarc
7. nova list

you can get 500 error

[Regression]

This patch is simple, but need to restart every api service e.g nova-api, glance-api, cinder-api etc because they are handling user id for getting information respectively. This causes temporarily service down.

[Others]

If I deployed trusty-mitaka in the beginning, keystone user id is form of uuid, so no need to patch this basically.

[Original description]
When Tenant name contains any non-acsii characters (such as Chinese), calling oslo_context/context.py#to_dict method will cause a unicodeEncodeError: 'ascii' codec can't encode characters in position 0-3: ordinal not in range(128).

[Davanum Srinivas (DIMS) (dims-v)]

Fixing in https://review.openstack.org/#/c/237767/

[ChangBo Guo(gcb) (glongwave)]

Another way to fix this issue in https://review.openstack.org/285688

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/oslo.context 2.3.0 release.

[Corey Bryant (corey.bryant)]

Hi Seoyeong,

Do you know what releases this affects?

Thanks,
Corey

[Corey Bryant (corey.bryant)]

It looks like this affects kilo and mitaka. Note the package is python-oslo-context in kilo.

[Brian Murray (brian-murray)]

Hello Ma, or anyone else affected,

Accepted python-oslo.context into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/python-oslo.context/2.2.0-2ubuntu1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Seyeong Kim (seyeongkim)]

Hello

I had different reproduce not using ldap here.

Please refer to below steps for reproducer and verification

1. deploy openstack xenial env
2. source novarc ( as admin )
3. keystone user-create --user test
4. log into keystone node and install mysql-client
5. connect to mysql with keystone user
6. do below query
- set foreign_key_checks = 0
- update local_user set user_id='Gāo' where user_id='UUID';
- update user set id='Gāo' where id='UUID';
- set foreign_key_checks = 1
7. keystone user-role-add --user test --tenant admin --role Admin
8. source novarc ( as test )
9. nova list

got unicode error on nova-cloud-controller node /var/log/syslog

10. upgraded python-oslo.context on -proposed repo

11. back to client node and nova list
12. it returns correct table without any server

Thanks

dpkg -l | grep oslo.context
ii python-oslo.context 2.2.0-2ubuntu1 all WSGI context helpers for OpenStack - Python 2.x

[Corey Bryant (corey.bryant)]

Hello Ma, or anyone else affected,

Accepted python-oslo.context into mitaka-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:mitaka-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-mitaka-needed to verification-mitaka-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-mitaka-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Seyeong Kim (seyeongkim)]

Hello

I used reproducer just above instead of using ldap.
and I added more line for resetting password

1. deploy openstack xenial env
2. source novarc ( as admin )
3. keystone user-create --name test
4. openstack user set --password-prompt test
4. log into keystone node and install mysql-client
5. connect to mysql with keystone user
6. do below query
- set foreign_key_checks = 0;
- update local_user set user_id='Gāo' where user_id='UUID';
- update user set id='Gāo' where id='UUID';
- set foreign_key_checks = 1;
7. keystone user-role-add --user test --tenant admin --role Admin
8. source novarc ( as test )
9. nova list

got unicode error on nova-cloud-controller node /var/log/upstart/nova-api-os-compute.log

10. upgraded python-oslo.context on -proposed repo

11. back to client node and nova list
12. it returns correct table without any server

Thanks

dpkg -l | grep oslo.context
ii python-oslo.context 2.2.0-2ubuntu1~cloud0 all WSGI context helpers for OpenStack - Python 2.x

[Seyeong Kim (seyeongkim)]

kilo has different DB table structure. ( no local_user )

I tried to do this issue with above reproducer.

but it just works fine.

[Seyeong Kim (seyeongkim)]

hello, any news for this SRU?

or need to do something else?

[Launchpad Janitor (janitor)]

This bug was fixed in the package python-oslo.context - 2.2.0-2ubuntu1

---------------
python-oslo.context (2.2.0-2ubuntu1) xenial; urgency=medium

  * Adding support unicode for older type of user id (LP: #1507937)
    - d/p/0001-Ensure-to_dict-supports-unicode.patch

 -- Seyeong Kim <email address hidden> Wed, 21 Mar 2018 02:23:53 -0700

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for python-oslo.context has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Corey Bryant (corey.bryant)]

The verification of the Stable Release Update for python-oslo.context has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Corey Bryant (corey.bryant)]

This bug was fixed in the package python-oslo.context - 2.2.0-2ubuntu1~cloud0
---------------

 python-oslo.context (2.2.0-2ubuntu1~cloud0) trusty-mitaka; urgency=medium
 .
   * New update for the Ubuntu Cloud Archive.
 .
 python-oslo.context (2.2.0-2ubuntu1) xenial; urgency=medium
 .
   * Adding support unicode for older type of user id (LP: #1507937)
     - d/p/0001-Ensure-to_dict-supports-unicode.patch