Unicode multi-bytes character raise 500 error

[Asmita Singh]

Description:-
==============
If you try to create a host passing hostname as unicode multi-bytes characters, it gives 500 error.

Curl command:-
==============
$ curl -g -i -X POST http://172.16.114.130/reservation/v1/os-hosts -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-blazarclient" -H "X-Auth-Token: gAAAAABb0tyMHuuF5_-LbGuNPfazyGuUnjewW1n0bxHivp87v7uKTNMDHkajrsKKuiJpjUyXxvcJKpTVBHpo2Sxh8mDk3bQzGFRlooplJ0u2Uwjpn3u1-iqNf8LLKDeHtqATWLbND-_Z-K63xRvjrlEYjXfWyDyQoX9jcgzdvSbuhFsE8hmMPV4" -d '{"name": "漢字"}'

Response:-
=======================

HTTP/1.1 500 INTERNAL SERVER ERROR
Date: Fri, 26 Oct 2018 09:26:27 GMT
Server: Apache/2.4.18 (Ubuntu)
Content-Type: application/json
Content-Length: 100
Connection: close

{"error_name": "INTERNAL_SERVER_ERROR", "error_message": "Internal Server Error", "error_code": 500}

blazar-a.service logs

Logs:-
====================
Oct 26 18:27:42 open <email address hidden>[101356]: 2018-10-26 18:27:42.542 101360 ERROR blazar.api.v1.utils [-] Request aborted with status code 500 and message 'Internal Server Error'
Oct 26 18:27:42 open <email address hidden>[101356]: 2018-10-26 18:27:42.543 101360 ERROR blazar.api.v1.utils [-] Traceback (most recent call last):
Oct 26 18:27:42 open <email address hidden>[101356]: File "/opt/stack/blazar/blazar/api/v1/utils.py", line 76, in handler
Oct 26 18:27:42 open <email address hidden>[101356]: return func(**kwargs)
Oct 26 18:27:42 open <email address hidden>[101356]: File "/opt/stack/blazar/blazar/api/v1/oshosts/v1_0.py", line 37, in computehosts_create
Oct 26 18:27:42 open <email address hidden>[101356]: return api_utils.render(host=_api.create_computehost(data))
Oct 26 18:27:42 open <email address hidden>[101356]: File "/opt/stack/blazar/blazar/utils/__init__.py", line 32, in __run_method
Oct 26 18:27:42 open <email address hidden>[101356]: return getattr(self.instance, __name)(*args, **kwargs)
Oct 26 18:27:42 open <email address hidden>[101356]: File "/opt/stack/blazar/blazar/policy.py", line 112, in wrapped
Oct 26 18:27:42 open <email address hidden>[101356]: return func(self, *args, **kwargs)
Oct 26 18:27:42 open <email address hidden>[101356]: File "/opt/stack/blazar/blazar/utils/trusts.py", line 93, in wrapped
Oct 26 18:27:42 open <email address hidden>[101356]: return func(self, to_update)
Oct 26 18:27:42 open <email address hidden>[101356]: File "/opt/stack/blazar/blazar/api/v1/oshosts/service.py", line 39, in create_computehost
Oct 26 18:27:42 open <email address hidden>[101356]: return self.manager_rpcapi.create_computehost(data)
Oct 26 18:27:42 open <email address hidden>[101356]: File "/opt/stack/blazar/blazar/manager/oshosts/rpcapi.py", line 47, in create_computehost
Oct 26 18:27:42 open <email address hidden>[101356]: host_values=host_values)
Oct 26 18:27:42 open <email address hidden>[101356]: File "/opt/stack/blazar/blazar/utils/service.py", line 44, in call
Oct 26 18:27:42 open <email address hidden>[101356]: return self._client.call(ctx.to_dict(), name, **kwargs)
Oct 26 18:27:42 open <email address hidden>[101356]: File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 505, in call
Oct 26 18:27:42 open <email address hidden>[101356]: return self.prepare().call(ctxt, method, **kwargs)
Oct 26 18:27:42 open <email address hidden>[101356]: File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 179, in call
Oct 26 18:27:42 open <email address hidden>[101356]: retry=self.retry)
Oct 26 18:27:42 open <email address hidden>[101356]: File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 128, in _send
Oct 26 18:27:42 open <email address hidden>[101356]: retry=retry)
Oct 26 18:27:42 open <email address hidden>[101356]: File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 645, in send
Oct 26 18:27:42 open <email address hidden>[101356]: call_monitor_timeout, retry=retry)
Oct 26 18:27:42 open <email address hidden>[101356]: File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 636, in _send
Oct 26 18:27:42 open <email address hidden>[101356]: raise result
Oct 26 18:27:42 open <email address hidden>[101356]: KeyError: u'u\'\\u6f22\'\nTraceback (most recent call last):\n\n File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 166, in _process_incoming\n res = self.dispatcher.dispatch(message)\n\n File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 265, in dispatch\n return self._do_dispatch(endpoint, method, ctxt, args)\n\n File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 194, in _do_dispatch\n result = func(ctxt, **new_args)\n\n File "/opt/stack/blazar/blazar/utils/service.py", line 77, in run_method\n return method(**kwargs)\n\n File "/opt/stack/blazar/blazar/plugins/oshosts/host_plugin.py", line 339, in create_computehost\n servers = inventory.get_servers_per_host(host_ref)\n\n File "/opt/stack/blazar/blazar/utils/openstack/nova.py", line 500, in get_servers_per_host\n hypervisors_list = self.nova.hypervisors.search(host, servers=True)\n\n File "/usr/local/lib/python2.7/dist-packages/novaclient/v2/hypervisors.py", line 94, in search\n (parse.quote(hypervisor_match, safe=\'\'), target))\n\n File "/usr/lib/python2.7/urllib.py", line 1299, in quote\n return \'\'.join(map(quoter, s))\n\nKeyError: u\'\\u6f22\'\n'
Oct 26 18:27:42 open <email address hidden>[101356]:

[Tushar Patil (tpatil)]

In python-novaclient, if you try to list hypervisors passing --matching parameter with unicode data, then it gives KeyError.

Tested on : Ubuntu 16.04.2 LTS

Check traceback here : http://paste.openstack.org/show/744355

[Takashi Natsume (natsume-takashi)]

Python 2.7? 3.5 or 3.6?

[Tushar Patil (tpatil)]

python-novaclient: This issue is reproducible in python 2.7.

[Takashi Natsume (natsume-takashi)]

Fix proposed to branch: master (python-novaclient)
https://review.openstack.org/#/c/632942/

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/632942
Committed: https://git.openstack.org/cgit/openstack/python-novaclient/commit/?id=4450e7ba3176cd26cdfa758355f8ed4ad8bacab2
Submitter: Zuul
Branch: master

commit 4450e7ba3176cd26cdfa758355f8ed4ad8bacab2
Author: asmita singh <email address hidden>
Date: Wed Jan 2 06:59:56 2019 +0000

    Handle unicode multi-byte characters

    If you pass unicode multi-byte character to 'hypervisor-list'
    and 'instance-usage-audit-log' commands using --matching
    and --before options respectively, it returns keyerror.

    This patch fixes these issues by encoding the host name and
    date passed to the --matching and --before options
    respectively.

    Closes-Bug: #1804156

    Change-Id: I9587af7e0fdd921fcaebe3e5c6c5bb40a9393e01

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/python-novaclient 13.0.0 release.

[Pierre Riteau (priteau)]

@Asmita: does this fully fix the bug or is some work required on Blazar as well?

[Asmita Singh (asmita2018)]

This fully fixes the bug.

[Asmita Singh (asmita2018)]

@Pierre: This fully fixes the bug.