Exception injecting key results in another error in some i18n cases

[Édouard Thuleau]

nova rev 1388 on Ubuntu 11.04

If an error occur in injecting data in the image befor the boot, the libvirt driver raises an exception an just puts a warning log (cf. nova/virt/libvirt/connection.py", line 919). But with the revision 1388, I get this error in nova-compute log and the instance fails :

2011-08-08 14:40:18,776 INFO nova.virt.libvirt_conn [-] instance instance-00000015 : injection de clef dans l'image 6
2011-08-08 14:40:18,776 DEBUG nova.utils [-] Execution de la commande (sous-processus) : sudo qemu-nbd -c /dev/nbd15 /var/lib/nova/instance
s/instance-00000015/disk from (pid=18279) execute /usr/lib/pymodules/python2.7/nova/utils.py:158
2011-08-08 14:40:19,796 DEBUG nova.utils [-] Execution de la commande (sous-processus) : sudo tune2fs -c 0 -i 0 /dev/nbd15 from (pid=18279)
 execute /usr/lib/pymodules/python2.7/nova/utils.py:158
2011-08-08 14:40:21,372 DEBUG nova.utils [-] Execution de la commande (sous-processus) : sudo mount /dev/nbd15 /tmp/tmpcz0Gtc from (pid=182
79) execute /usr/lib/pymodules/python2.7/nova/utils.py:158
2011-08-08 14:40:21,391 DEBUG nova.utils [-] Le résultat était 32 from (pid=18279) execute /usr/lib/pymodules/python2.7/nova/utils.py:176
2011-08-08 14:40:21,391 DEBUG nova.utils [-] Execution de la commande (sous-processus) : rmdir /tmp/tmpcz0Gtc from (pid=18279) execute /usr
/lib/pymodules/python2.7/nova/utils.py:158
2011-08-08 14:40:21,401 DEBUG nova.utils [-] Execution de la commande (sous-processus) : sudo qemu-nbd -d /dev/nbd15 from (pid=18279) execu
te /usr/lib/pymodules/python2.7/nova/utils.py:158
2011-08-08 14:40:21,417 ERROR nova.exception [-] Exception non prévue
(nova.exception): TRACE: Traceback (most recent call last):
(nova.exception): TRACE: File "/usr/lib/pymodules/python2.7/nova/exception.py", line 97, in wrapped
(nova.exception): TRACE: return f(*args, **kw)
(nova.exception): TRACE: File "/usr/lib/pymodules/python2.7/nova/virt/libvirt/connection.py", line 585, in spawn
(nova.exception): TRACE: block_device_mapping=block_device_mapping)
(nova.exception): TRACE: File "/usr/lib/pymodules/python2.7/nova/virt/libvirt/connection.py", line 919, in _create_image
(nova.exception): TRACE: ' data into image %(img_id)s (%(e)s)') % locals())
(nova.exception): TRACE: UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 11: ordinal not in range(128)
(nova.exception): TRACE:
2011-08-08 14:40:21,418 ERROR nova.compute.manager [-] Instance '21' failed to spawn. Is virtualization enabled in the BIOS? Details: 'asci
i' codec can't encode character u'\xe9' in position 11: ordinal not in range(128)
(nova.compute.manager): TRACE: Traceback (most recent call last):
(nova.compute.manager): TRACE: File "/usr/lib/pymodules/python2.7/nova/compute/manager.py", line 357, in _run_instance
(nova.compute.manager): TRACE: self.driver.spawn(context, instance, network_info, bd_mapping)
(nova.compute.manager): TRACE: File "/usr/lib/pymodules/python2.7/nova/exception.py", line 124, in wrapped
(nova.compute.manager): TRACE: raise Error(str(e))
(nova.compute.manager): TRACE: Error: 'ascii' codec can't encode character u'\xe9' in position 11: ordinal not in range(128)
(nova.compute.manager): TRACE:
2011-08-08 14:40:21,523 DEBUG nova.compute.manager [-] Vérification de l'état de instance-00000015 from (pid=18279) _update_state /usr/lib/
pymodules/python2.7/nova/compute/manager.py:183

[Édouard Thuleau (ethuleau)]

I found the problem. I use a system configure in french language.
In this case, the return error message contains some accentuated letters and the log message cannot decode this warning.

Is it a bug ?

[Thierry Carrez (ttx)]

Looks like a resurgence of bug 733079

[Thierry Carrez (ttx)]

There is the same type of error hiding in case a utils.execute calls fails:

in exception.py, see ProcessExecutionError

 message = _('%(description)s\nCommand: %(cmd)s\n'
                    'Exit code: %(exit_code)s\nStdout: %(stdout)r\n'
                    'Stderr: %(stderr)r') % locals()

description unfortunately may contain unicode from i18n.

[justinsb (justin-fathomdb)]

Here what I've diagnosed:

* Python 3 uses unicode strings everywhere, but Python 2 has two different string types - str and unicode

* If you call str(s) on a unicode string with extended characters, it tries to convert to ascii (the default python 2 encoding) and it fails with the exception above

* If we use unicode types everywhere, we'd be OK.

* _ returns unicode strings (because we called install with unicode=1). This is good.

* I think this bug report relates to this call:
(nova.compute.manager): TRACE: raise Error(str(e))

* That call was removed in this commit, so this particular bug has been fixed:
git show 19b3b7a0c50dd6a68238daf8

* However, this does mean that everywhere we call str(x) is potentially problematic (?). I'm going to open another bug on this issue.