Doesn't support Unicode names

[Pellaeon Lin]

Some of my project descriptions are in Chinese. When I openstack project list:

'ascii' codec can't encode characters in position 765-767: ordinal not in range(128)
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/cliff/app.py", line 374, in run_subcommand
    result = cmd.run(parsed_args)
  File "/usr/local/lib/python2.7/dist-packages/cliff/display.py", line 93, in run
    self.produce_output(parsed_args, column_names, data)
  File "/usr/local/lib/python2.7/dist-packages/cliff/lister.py", line 56, in produce_output
    parsed_args,
  File "/usr/local/lib/python2.7/dist-packages/cliff/formatters/table.py", line 60, in emit_list
    stdout.write(formatted)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 765-767: ordinal not in range(128)

[John Dennis (jdennis-a)]

The problem is likely in your terminal environment and not in openstack.

Please perform the following to help us diagnose the problem, run python interactively and print out the stdout encoding value, e.g.

% python
>>> import sys
>>> sys.stdout.encoding

If the value is "ascii" or "ANSI_X3.4-1968" or any other equivalent name for ASCII the problem is in your environment. You most likely do not have your LANG environment variable set correctly. LANG should be of the form:

lang_code [underscore region_code] dot encoding, for me as a US English my LANG looks like this:

en_US.UTF-8

The critical thing here is the .encoding value of UTF-8. Python when it starts up looks up your locale information and if it find an encoding value sets the encoding property of stdin, stdout and stderr to the value found in your environment, if it's not defined in your environment it defaults to ASCII. Also you should be aware this only occurs if stdin, stdout and stderr are attached to a TTY, if you redirect the encoding on these streams will default to sys defaultencoding which is set in the site.py of your Python distribution which defaults to ASCII.

Note, this only applies to Python 2, Python 3 has different semantics.

[John Dennis (jdennis-a)]

Is this problem only seen when output is redirected?

[Pellaeon Lin (pellaeon)]

Ah, thanks, I was not aware of Python encoding subtleties.

Setting LC_ALL=en_US.UTF-8 fixes normal output of Unicode, but still doesn't work when output is redirected.

Setting PYTHONIOENCODING=utf-8 fixes Unicode output when redirected.
(From http://stackoverflow.com/a/4027726)