UnicodeEncodeError when snapshotting an instance or uploading an image (Essex)

[Jérôme Gallard]

Hi,
With the last Essex packages on Ubuntu, I'am not able to give a name on a snapshot with an accented character.

Command:
nova image-create c67bb8ea-7992-4254-8ab2-997826f08d92 "clihhé"
returns:
ERROR: The server has either erred or is incapable of performing the requested operation. (HTTP 500)

Nova API logs :
2012-06-05 14:19:02 DEBUG nova.image.glance [req-82eba2b0-4939-4c31-b354-e91ed9385373 93e7c6650bdd4e37b9a3a06ed6648db4 e77dafa4373f41c3aa3da4809cd31d4f] Creating image in Glance. Metadata passed in {'min_disk': '0', 'is_public': False, 'min_ram': '0', 'name': u'clihh\xe9', 'properties': {'instance_uuid': u'c67bb8ea-7992-4254-8ab2-997826f08d92', 'image_type': 'snapshot', 'user_id': '93e7c6650bdd4e37b9a3a06ed6648db4'}} from (pid=1349) create /usr/lib/python2.7/dist-packages/nova/image/glance.py:279
2012-06-05 14:19:02 DEBUG nova.image.glance [req-82eba2b0-4939-4c31-b354-e91ed9385373 93e7c6650bdd4e37b9a3a06ed6648db4 e77dafa4373f41c3aa3da4809cd31d4f] Metadata after formatting for Glance {'min_disk': '0', 'is_public': False, 'min_ram': '0', 'name': u'clihh\xe9', 'properties': {'instance_uuid': u'c67bb8ea-7992-4254-8ab2-997826f08d92', 'image_type': 'snapshot', 'user_id': '93e7c6650bdd4e37b9a3a06ed6648db4'}} from (pid=1349) create /usr/lib/python2.7/dist-packages/nova/image/glance.py:282
2012-06-05 14:19:02 ERROR nova.api.openstack [req-82eba2b0-4939-4c31-b354-e91ed9385373 93e7c6650bdd4e37b9a3a06ed6648db4 e77dafa4373f41c3aa3da4809cd31d4f] Caught error: 'ascii' codec can't encode character u'\xe9' in position 5: ordinal not in range(128)
2012-06-05 14:19:02 TRACE nova.api.openstack Traceback (most recent call last):
2012-06-05 14:19:02 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/nova/api/openstack/__init__.py", line 41, in __call__
2012-06-05 14:19:02 TRACE nova.api.openstack return req.get_response(self.application)
2012-06-05 14:19:02 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1086, in get_response
2012-06-05 14:19:02 TRACE nova.api.openstack application, catch_exc_info=False)
2012-06-05 14:19:02 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1055, in call_application
2012-06-05 14:19:02 TRACE nova.api.openstack app_iter = application(self.environ, start_response)
2012-06-05 14:19:02 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/keystone/middleware/auth_token.py", line 176, in __call__
2012-06-05 14:19:02 TRACE nova.api.openstack return self.app(env, start_response)
2012-06-05 14:19:02 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 159, in __call__
2012-06-05 14:19:02 TRACE nova.api.openstack return resp(environ, start_response)
2012-06-05 14:19:02 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 159, in __call__
2012-06-05 14:19:02 TRACE nova.api.openstack return resp(environ, start_response)
2012-06-05 14:19:02 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 159, in __call__
2012-06-05 14:19:02 TRACE nova.api.openstack return resp(environ, start_response)
2012-06-05 14:19:02 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/routes/middleware.py", line 131, in __call__
2012-06-05 14:19:02 TRACE nova.api.openstack response = self.app(environ, start_response)
2012-06-05 14:19:02 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 159, in __call__
2012-06-05 14:19:02 TRACE nova.api.openstack return resp(environ, start_response)
2012-06-05 14:19:02 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 147, in __call__
2012-06-05 14:19:02 TRACE nova.api.openstack resp = self.call_func(req, *args, **self.kwargs)
2012-06-05 14:19:02 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 210, in call_func
2012-06-05 14:19:02 TRACE nova.api.openstack return self.func(req, *args, **kwargs)
2012-06-05 14:19:02 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py", line 803, in __call__
2012-06-05 14:19:02 TRACE nova.api.openstack content_type, body, accept)
2012-06-05 14:19:02 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py", line 851, in _process_stack
2012-06-05 14:19:02 TRACE nova.api.openstack action_result = self.dispatch(meth, request, action_args)
2012-06-05 14:19:02 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py", line 926, in dispatch
2012-06-05 14:19:02 TRACE nova.api.openstack return method(req=request, **action_args)
2012-06-05 14:19:02 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/nova/api/openstack/common.py", line 421, in inner
2012-06-05 14:19:02 TRACE nova.api.openstack return f(*args, **kwargs)
2012-06-05 14:19:02 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/servers.py", line 1121, in _action_create_image
2012-06-05 14:19:02 TRACE nova.api.openstack extra_properties=props)
2012-06-05 14:19:02 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 100, in wrapped
2012-06-05 14:19:02 TRACE nova.api.openstack return func(self, context, target, *args, **kwargs)
2012-06-05 14:19:02 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 90, in inner
2012-06-05 14:19:02 TRACE nova.api.openstack return f(self, context, instance, *args, **kw)
2012-06-05 14:19:02 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 1164, in snapshot
2012-06-05 14:19:02 TRACE nova.api.openstack extra_properties=extra_properties)
2012-06-05 14:19:02 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 1214, in _create_image
2012-06-05 14:19:02 TRACE nova.api.openstack recv_meta = self.image_service.create(context, sent_meta)
2012-06-05 14:19:02 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/nova/image/glance.py", line 285, in create
2012-06-05 14:19:02 TRACE nova.api.openstack sent_service_image_meta, data)
2012-06-05 14:19:02 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/glance/client.py", line 161, in add_image
2012-06-05 14:19:02 TRACE nova.api.openstack res = self.do_request("POST", "/images", body, headers)
2012-06-05 14:19:02 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/glance/common/client.py", line 58, in wrapped
2012-06-05 14:19:02 TRACE nova.api.openstack return func(self, *args, **kwargs)
2012-06-05 14:19:02 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/glance/common/client.py", line 420, in do_request
2012-06-05 14:19:02 TRACE nova.api.openstack headers=headers)
2012-06-05 14:19:02 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/glance/common/client.py", line 75, in wrapped
2012-06-05 14:19:02 TRACE nova.api.openstack return func(self, method, url, body, headers)
2012-06-05 14:19:02 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/glance/common/client.py", line 505, in _do_request
2012-06-05 14:19:02 TRACE nova.api.openstack c.request(method, path, body, headers)
2012-06-05 14:19:02 TRACE nova.api.openstack File "/usr/lib/python2.7/httplib.py", line 958, in request
2012-06-05 14:19:02 TRACE nova.api.openstack self._send_request(method, url, body, headers)
2012-06-05 14:19:02 TRACE nova.api.openstack File "/usr/lib/python2.7/httplib.py", line 991, in _send_request
2012-06-05 14:19:02 TRACE nova.api.openstack self.putheader(hdr, value)
2012-06-05 14:19:02 TRACE nova.api.openstack File "/usr/lib/python2.7/httplib.py", line 938, in putheader
2012-06-05 14:19:02 TRACE nova.api.openstack hdr = '%s: %s' % (header, '\r\n\t'.join([str(v) for v in values]))
2012-06-05 14:19:02 TRACE nova.api.openstack UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 5: ordinal not in range(128)
2012-06-05 14:19:02 TRACE nova.api.openstack
2012-06-05 14:19:02 INFO nova.api.openstack [req-82eba2b0-4939-4c31-b354-e91ed9385373 93e7c6650bdd4e37b9a3a06ed6648db4 e77dafa4373f41c3aa3da4809cd31d4f] http://localhost:8774/v2/e77dafa4373f41c3aa3da4809cd31d4f/servers/c67bb8ea-7992-4254-8ab2-997826f08d92/action returned with HTTP 500

[Ying Chun Guo (daisy-ycguo)]

Besides snapshot creation, I'm not able to upload an image with accented character too.

For example:

glance add name="测试VM" is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img

Output:

Failed to add image. Got error:
'ascii' codec can't decode byte 0xe6 in position 0: ordinal not in range(128)
Note: Your image metadata may still be in the registry, but the image's status will likely be 'killed'.

[Thierry Carrez (ttx)]

Not sure we are supposed to support special characters, but at the very minimum that error message needs to be cleaned up.

[Eoghan Glynn (eglynn)]

In this particular case, the non-ascii character is blowing up as a side-effect of how the glance v1 API represents meta-data as HTTP headers.

This should be resolved with the v2 API where the meta-data is promoted up to the entity-body.

[Brian Waldon (bcwaldon)]

Can we safely call this Won't Fix in Nova as well? It seems like it's not possible to do this in the v1 API.

[Jérôme Gallard (jerome-gallard)]

OK :(
But, is it possible to check this and say to the user that those characters are not allowed?

The problem is, my users can ask for a snapshot and give a name with a special character. Then they get the error. But, their initial VMs stay in the "Image Snapshot" status. No way to make a new snapshot or even to reboot them.

The solution I found is to ask to the user if it's possible to terminate it's instance (but generaly, when he/she wants to make a snapshot, he/she doesn't want to terminate the instance before the snapshot is taken :), or, the second solution I found is to updade the nova database manually. In all cases those solutions are not very cleaned...

[Xavier Queralt (xqueralt-deactivatedaccount)]

I've tried to reproduce this bug against grizzly but it seems it has already been fixed.

I haven't had any problem while creating new images/snapshots using unicode characters in their names.

Probably this could be closed.