non-ascii usernames broken in nova

Bug #931667 reported by Andrew Bogott on 2012-02-13
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Undecided
Andrew Bogott
OpenStack Dashboard (Horizon)
Medium
Gabriel Hurley

Bug Description

Taking mtaylor's suggestion, I've been trying to create a user with the name 'måņŧÿ'. It fails in various ways. Horizon just says 'Error: Error creating user: Unhandled error.' Nova-manage gives more info:

$ nova-manage user create måņŧÿ
2012-02-09 20:16:46,518 ERROR nova.exception [req-67983579-0a3a-4004-b702-3ba4ebec5eea None None] DB exception wrapped.
(nova.exception): TRACE: Traceback (most recent call last):
(nova.exception): TRACE: File "/opt/stack/nova/nova/exception.py", line 104, in _wrap
(nova.exception): TRACE: return f(*args, **kwargs)
(nova.exception): TRACE: File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 1400, in flush
(nova.exception): TRACE: self._flush(objects)
(nova.exception): TRACE: File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 1481, in _flush
(nova.exception): TRACE: flush_context.execute()
(nova.exception): TRACE: File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 302, in execute
(nova.exception): TRACE: rec.execute(self)
(nova.exception): TRACE: File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 446, in execute
(nova.exception): TRACE: uow
(nova.exception): TRACE: File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/mapper.py", line 1887, in _save_obj
(nova.exception): TRACE: execute(statement, params)
(nova.exception): TRACE: File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1191, in execute
(nova.exception): TRACE: params)
(nova.exception): TRACE: File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1271, in _execute_clauseelement
(nova.exception): TRACE: return self.__execute_context(context)
(nova.exception): TRACE: File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1302, in __execute_context
(nova.exception): TRACE: context.parameters[0], context=context)
(nova.exception): TRACE: File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1401, in _cursor_execute
(nova.exception): TRACE: context)
(nova.exception): TRACE: File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1394, in _cursor_execute
(nova.exception): TRACE: context)
(nova.exception): TRACE: File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 299, in do_execute
(nova.exception): TRACE: cursor.execute(statement, parameters)
(nova.exception): TRACE: ProgrammingError: (ProgrammingError) You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings. u'INSERT INTO users (created_at, updated_at, deleted_at, deleted, id, name, access_key, secret_key, is_admin) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)' ('2012-02-10 02:16:46.515383', None, None, 0, 'm\xc3\xa5\xc5\x86\xc5\xa7\xc3\xbf', None, '30d6e169-64be-40ef-8f00-872939918505', 'd943602e-1875-4938-ab62-70eba6ac9ebb', 0)
(nova.exception): TRACE:
Command failed, please check log for more info
2012-02-09 20:16:46,570 CRITICAL nova [-] User %(user)s already exists.

Andrew Bogott (andrewbogott) wrote :

Probably we just need to decode some args. I'm inveestigating.

Devin Carlen (devcamcar) on 2012-02-20
Changed in horizon:
status: New → Confirmed
importance: Undecided → Medium
Andrew Bogott (andrewbogott) wrote :

It looks like we are generally fast and loose with encoding in nova. I have a patch that fixes one particular case, but we need a general solution:

1) Adopt a standard for what encoding is used for all implicitly-encoded IO. (I would propose that that standard be UTF8 rather than ASCII.)

2) At all points where strings enter Python (e.g. commandline args) immediately decode them into unicode (which can unambiguously contain all possible 8-bit encodings.)

3) At all points where 'unicodes' exit Python (being written to stdout or a log file or a database) explicitly encode them as appropriate (generally UTF8, again, especially if we're ever going to read them back in.)

Changed in nova:
assignee: nobody → Andrew Bogott (andrewbogott)
status: New → In Progress

Reviewed: https://review.openstack.org/4209
Committed: http://github.com/openstack/nova/commit/93cee6047f74e256c3695451b5dc51256208f76f
Submitter: Jenkins
Branch: master

commit 93cee6047f74e256c3695451b5dc51256208f76f
Author: Andrew Bogott <email address hidden>
Date: Wed Feb 15 13:08:27 2012 -0600

    Decode nova-manage args into unicode.

    This improves handling of non-ASCII args... it's not
    exactly a comprehensive solution, though.

    Partial (demonstration) fix for bug 931667.

    Change-Id: Icc2aabe9c0c521732003f7013438630ebc8d2238

Changed in nova:
status: In Progress → Fix Committed
Andy Chong (andycjw) wrote :

I have tested and it seems user with name 'måņŧÿ' is successfully created with the change in nova
should this bug be considered fix in horizon as well?

Changed in horizon:
status: Confirmed → Fix Committed
Andrew Bogott (andrewbogott) wrote :

Yes, I believe it is fixed.

Changed in horizon:
milestone: none → essex-rc1
assignee: nobody → Gabriel Hurley (gabriel-hurley)
Thierry Carrez (ttx) on 2012-03-20
Changed in nova:
milestone: none → essex-rc1
status: Fix Committed → Fix Released
Thierry Carrez (ttx) on 2012-03-20
Changed in horizon:
status: Fix Committed → Fix Released
Thierry Carrez (ttx) on 2012-04-05
Changed in horizon:
milestone: essex-rc1 → 2012.1
Thierry Carrez (ttx) on 2012-04-05
Changed in nova:
milestone: essex-rc1 → 2012.1
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers