If user set a non-ascii character to the key of an image property and using qpid backend, the instance provisioning will be failed.
Use case:
1. Set a non-ascii character to the key of an image property:
"glance image-update --property '中文'='中文' 5a7e6de5-15b9-49d2-977e-eda0a221c0b9"
2. Launch a instance with that image:
"nova boot --image cirros-0.3.0-x86_64 --flavor m1.tiny --nic net-id=881cbe80-3876-4f4c-b8c8-7b23cedc14f4 lzy-1"
3. Then the provisioning failed and nova-api service catch the exception and write to log:
File "/usr/lib/python2.6/site-packages/qpid/messaging/driver.py", line 1052, in process
self.send(snd, msg)
File "/usr/lib/python2.6/site-packages/qpid/messaging/driver.py", line 1263, in send
body = enc(msg.content)
File "/usr/lib/python2.6/site-packages/qpid/messaging/message.py", line 28, in encode
sc.write_primitive(type, x)
File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 73, in write_primitive
getattr(self, "write_%s" % type.NAME)(v)
File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 257, in write_map
sc.write(string.joinfields(map(self._write_map_elem, m.keys(), m.values()), ""))
File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 250, in _write_map_elem
sc.write_primitive(type, v)
File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 73, in write_primitive
getattr(self, "write_%s" % type.NAME)(v)
File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 257, in write_map
sc.write(string.joinfields(map(self._write_map_elem, m.keys(), m.values()), ""))
File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 250, in _write_map_elem
sc.write_primitive(type, v)
File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 73, in write_primitive
getattr(self, "write_%s" % type.NAME)(v)
File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 257, in write_map
sc.write(string.joinfields(map(self._write_map_elem, m.keys(), m.values()), ""))
File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 250, in _write_map_elem
sc.write_primitive(type, v)
File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 73, in write_primitive
getattr(self, "write_%s" % type.NAME)(v)
File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 257, in write_map
sc.write(string.joinfields(map(self._write_map_elem, m.keys(), m.values()), ""))
File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 250, in _write_map_elem
sc.write_primitive(type, v)
File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 73, in write_primitive
getattr(self, "write_%s" % type.NAME)(v)
File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 257, in write_map
sc.write(string.joinfields(map(self._write_map_elem, m.keys(), m.values()), ""))
File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 248, in _write_map_elem
sc.write_str8(k)
File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 185, in write_str8
self.write_vbin8(s.encode("utf8"))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe4 in position 0: ordinal not in range(128)
Analysis:
In nova-api service, the provisioning request will be handled in "nova/compute/api.py' file "_validate_and_provision_instance" method ("create_instance" method called), the image and its properties will be primitive and make a request spec and pass to nova-scheduler by rpc calls via message queue. And nova-api call glance api v1.0 ("get" method) to get image's infromation, under current glanceclient implementation, those non-ascii character in the image property part will be encoded by utf-8, so the root cause is that qpid (with qpid_python-0.18 lib) can not handle those utf-8 string.
Solution:
There are two choices to resolve this issue:
1. Resolve in glanceclient. glance-api server return utf-8 property (key/value pair) to glanceclient, so we need convert the them back to unicode from utf-8 ("_image_meta_from_headers" method in "image.py" file), this will cause client.get() method return unicode property string (key/value pair) and nova-api will get those uncode strings and pass them to nova-scheduler via rpc. this unicode encoding logic just like image property update logic ("_image_meta_to_headers" method), update method always pass unicode image property to glance-api server but not utf-8, so this change will keep the encoding consistency.
And this fix also need change 2 points in nova-api code ("nova/compute/api.py"), they are call str() on the image properties, it should be unicode() after above change.
2. Resolve in nova-api. This way probably is not good then above one since the area affected is greater the it. in this way, we need convert utf-8 string to unicode then to put it into queue, so the change/fix in "to_primitive" method of "jsonutils.py" file. for now, there are two problems:
a. it not take care key within a dict object, that caused utf-8 encoded key string (in this case, it's a key of an image property) will not be primitive.
b. need ask to_primitive take care utf-8 string, IMO, need convert it to unicode but not pass the utf-8 (basestring object) directly. So this fix will affect more area then above glanceclient fixing way, this fix has commonality, even user use rabbitmq backend, any utf-8 encoded key within a dict will be convert include glance properties and any other dict objects which want to primitive and put into queue.
IMO, I prefer use #1 way to address this issue.
Addressed here: https:/