UnicodeDecodeError occurs when non-ascii char in the instance name/description

[Shi Yan]

Description
===========
When using the non-ascii character in the instance description, the new instance spinning up fails due to the UnicodDecodeError in the nova-scheduler.
We are using the pike version nova in ubuntu. (16.1.4). The python env is py27

Steps to reproduce
==================
1) Add some no-ascii char into the instance description.
nova update --description "testing 测试" 2eda7ea7-94f1-4ad9-8ea5-f4f007bb4e4d

2) restart the nova-scheduler process
3) spin up a new instance. Then The new instance scheduling is failed.

The fix is remove non-ascii character and restart the nova-scheduler.

Log stack is:

2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server [req-69d3744f-8f4b-4aa4-b194-d2ec6e002bcb 8ca775f6ebab4cc6b16405b7c2fefd05 704fdb05a2f645ac8dbf4fb1222bf267 - default default] Exception during message handling: UnicodeDecodeError: 'ascii' codec can't decode byte 0xe6 in position 4326: ordinal not in range(128)
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 232, in inner
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server return func(*args, **kwargs)
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/scheduler/manager.py", line 150, in select_destinations
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server alloc_reqs_by_rp_uuid, provider_summaries)
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/scheduler/filter_scheduler.py", line 89, in select_destinations
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server alloc_reqs_by_rp_uuid, provider_summaries)
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/scheduler/filter_scheduler.py", line 158, in _schedule
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server provider_summaries)
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/scheduler/filter_scheduler.py", line 352, in _get_all_host_states
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server spec_obj)
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/scheduler/host_manager.py", line 675, in get_host_states_by_uuids
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server return self._get_host_states(context, compute_nodes, services)
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/scheduler/host_manager.py", line 720, in _get_host_states
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server self._get_instance_info(context, compute))
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/scheduler/host_manager.py", line 185, in update
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server return _locked_update(self, compute, service, aggregates, inst_dict)
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py", line 274, in inner
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server return f(*args, **kwargs)
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/scheduler/host_manager.py", line 182, in _locked_update
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server LOG.debug("Update host state with instances: %s", inst_dict)
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/logging/__init__.py", line 1440, in debug
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server self.logger.debug(msg, *args, **kwargs)
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/logging/__init__.py", line 1155, in debug
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server self._log(DEBUG, msg, args, **kwargs)
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/logging/__init__.py", line 1286, in _log
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server self.handle(record)
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/logging/__init__.py", line 1296, in handle
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server self.callHandlers(record)
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/logging/__init__.py", line 1336, in callHandlers
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server hdlr.handle(record)
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/logging/__init__.py", line 759, in handle
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server self.emit(record)
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_log/handlers.py", line 73, in emit
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server message = self.format(record)
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/logging/__init__.py", line 734, in format
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server return fmt.format(record)
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_log/formatters.py", line 442, in format
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server return logging.Formatter.format(self, record)
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/logging/__init__.py", line 476, in format
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server raise e
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server UnicodeDecodeError: 'ascii' codec can't decode byte 0xe6 in position 4326: ordinal not in range(128)
2018-11-19 16:44:45.548 12113 ERROR oslo_messaging.rpc.server

[Balazs Gibizer (balazs-gibizer)]

I could not reproduce the issue on master http://paste.openstack.org/show/743228/

[Balazs Gibizer (balazs-gibizer)]

The fix for this issue [1] is merged on master and on stable/rocky and it is proposed to stable/queens it is expected to be proposed to stable/pike as soon as the stable/queens patch merges.

[1] https://review.openstack.org/#/q/I0eda1c58a7eb54121230c880818b4b1d0fdf4893