Cannot start instance when using non-ASCII chars in user data

[Vincent Untz]

(This is on essex, not sure if this occurs with folsom)

When launching an instance, and filling User data section with something like "sdsdfgrtčřýřčýčžářžáížýáí", we end up with an error:
  Error: There was an error submitting the form. Please try again.

That's actually a crash:

========================
Traceback:
File "/usr/lib64/python2.6/site-packages/django/core/handlers/base.py" in get_response
  111. response = callback(request, *callback_args, **callback_kwargs)
File "/usr/lib64/python2.6/site-packages/horizon/decorators.py" in dec
  40. return view_func(request, *args, **kwargs)
File "/usr/lib64/python2.6/site-packages/horizon/decorators.py" in dec
  55. return view_func(request, *args, **kwargs)
File "/usr/lib64/python2.6/site-packages/horizon/decorators.py" in dec
  40. return view_func(request, *args, **kwargs)
File "/usr/lib64/python2.6/site-packages/django/views/generic/base.py" in view
  48. return self.dispatch(request, *args, **kwargs)
File "/usr/lib64/python2.6/site-packages/django/views/generic/base.py" in dispatch
  69. return handler(request, *args, **kwargs)
File "/usr/lib64/python2.6/site-packages/horizon/forms/views.py" in post
  84. return self.get(self, request, *args, **kwargs)
File "/usr/lib64/python2.6/site-packages/horizon/forms/views.py" in get
  64. form, handled = self.maybe_handle()
File "/usr/lib64/python2.6/site-packages/horizon/forms/views.py" in maybe_handle
  59. self.form, self.handled = form.maybe_handle(self.request, **kwargs)
File "/usr/lib64/python2.6/site-packages/horizon/forms/base.py" in maybe_handle
  101. exceptions.handle(request)
File "/usr/lib64/python2.6/site-packages/horizon/forms/base.py" in maybe_handle
  99. return form, form.handle(request, form.cleaned_data)
File "/usr/lib64/python2.6/site-packages/horizon/dashboards/nova/images_and_snapshots/images/forms.py" in handle
  189. redirect=redirect)
File "/usr/lib64/python2.6/site-packages/horizon/dashboards/nova/images_and_snapshots/images/forms.py" in handle
  182. instance_count=data.get('count'))
File "/usr/lib64/python2.6/site-packages/horizon/api/nova.py" in server_create
  296. min_count=instance_count), request)
File "/usr/lib64/python2.6/site-packages/novaclient/v1_1/servers.py" in create
  498. **boot_kwargs)
File "/usr/lib64/python2.6/site-packages/novaclient/v1_1/base.py" in _boot
  75. body["server"]["user_data"] = base64.b64encode(userdata)
File "/usr/lib64/python2.6/base64.py" in b64encode
  53. encoded = binascii.b2a_base64(s)[:-1]

Exception Type: UnicodeEncodeError at /nova/images_and_snapshots/e6569185-6630-4594-b43e-7888565a394e/launch/
Exception Value: 'ascii' codec can't encode character u'\xe9' in position 6:
ordinal not in range(128)
Request information:
GET: No GET data

POST:
count = u'1'
flavor = u'1'
name = u'sdfs'
tenant_id = u'bd06d26d95fa4b83bb0b1520b61c1cda'
user_data = u'sdfser\xe9"\xe9\xe9\xe9'
device_name = u'vda'
volume = u''
image_id = u'e6569185-6630-4594-b43e-7888565a394e'
keypair = u''
csrfmiddlewaretoken = u'RwPUh3Dkq48zJxMP5bmUd9XwhphSPjO2'
method = u'LaunchForm'
security_groups = u'default'
========================

This is a unicode vs str-in-utf8 issue; we simply need to change the unicode string to str-in-utf8 with a .encore('utf-8') call somewhere, but I'm unsure where to do this: is novaclient expecting its input to be utf8-encoded or unicode, or should novaclient do the encoding of unicode strings itself?

I would think the former, so the fix would be in horizon. What do you think?

[Gabriel Hurley (gabriel-hurley)]

Added Nova since the fatal error is down in novaclient. Horizon can help alleviate here, but it's worth looking at on a deeper level as well.

[Mark McLoughlin (markmc)]

Moving to novaclient, seems completely client side

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/13819

[Gabriel Hurley (gabriel-hurley)]

It looks like this is being fixed upstream, and should be verified in Horizon once it's merged.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/13819
Committed: http://github.com/openstack/python-novaclient/commit/81c01e535cecc54c2e6e847000638b52fec506af
Submitter: Jenkins
Branch: master

commit 81c01e535cecc54c2e6e847000638b52fec506af
Author: Vincent Untz <email address hidden>
Date: Fri Sep 28 11:48:38 2012 +0200

    Encode user data to utf-8 when creating a server

    This is needed for data that we don't read from IO directly, since we're
    base64-encoding the user data, and this requires ther user data to not
    be of the unicode type.

    We're being tolerant, as we'll accept user data that is already in a str
    object.

    Fixes: bug 1049161
    Change-Id: I4320670de564c8029c7aef14da2492c6f8752efe

[Tihomir Trifonov (ttrifonov)]

The novaclient fixes the problem. Verified in Dashboard - works fine.

[Gabriel Hurley (gabriel-hurley)]

As Tihomir said, this is verified fixed.