Unicode output problems

[Sean McKeon]

I'm using doit to automate building of RPMs for python modules. For several modules, somewhere in the output of setup.py, there's some Unicode that's not being handled properly, generating the error:

Exception in thread Thread-4:
Traceback (most recent call last):
  File "/usr/lib/python2.6/threading.py", line 532, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.6/threading.py", line 484, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/lib/python2.6/site-packages/doit-0.11.0-py2.6.egg/doit/action.py", line 59, in _print_process_output
    realtime.write(line)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2018' in position 28: ordinal not in range(128)

To see the error in action, download http://pypi.python.org/packages/source/p/pymongo/pymongo-1.10.1.tar.gz#md5=9e94096991285c384adfa8cafaf39c88 and then place the following into your dodo.py :
----
# dodo.py
def task_show_bug():
    return { "actions": [ "tar --strip-component 1 -xzf pymongo*gz",
                          "python setup.py build"
           ]
           }
----
...and run the task.

System Information:
============
(asdf)[mckeons@virtual-matt bug]$ doit --version
0.11.0
bin @ /usr/lib/python2.6/site-packages/doit-0.11.0-py2.6.egg/doit/doit_cmd.pyc
lib @ /usr/lib/python2.6/site-packages/doit-0.11.0-py2.6.egg/doit
(asdf)[mckeons@virtual-matt bug]$ head -1 `which doit`
#!/home/mckeons/asdf/bin/python2.6
(asdf)[mckeons@virtual-matt bug]$ /home/mckeons/asdf/bin/python2.6
Python 2.6.5 (r265:79063, Feb 28 2011, 21:55:56)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-50)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
(asdf)[mckeons@virtual-matt bug]$ uname -a
Linux virtual-matt 2.6.18-238.9.1.el5 #1 SMP Tue Apr 12 18:10:56 EDT 2011 i686 i686 i386 GNU/Linux
(asdf)[mckeons@virtual-matt bug]$ cat /etc/redhat-release
CentOS release 5.6 (Final)
(asdf)[mckeons@virtual-matt bug]$

[schettino72 (schettino72)]

I have to confess that I get very confused with this unicode stuff... can you try this patch?

=== modified file 'doit/action.py'
--- doit/action.py 2011-04-16 17:26:58 +0000
+++ doit/action.py 2011-04-30 02:23:00 +0000
@@ -56,7 +56,7 @@
             if line:
                 capture.write(line)
                 if realtime:
- realtime.write(line)
+ realtime.write(line.encode('utf-8'))
             if not line and process.poll() != None:
                 break

[Sean McKeon (sean-mckeon)]

Tried the patch, works great. Thanks for the quick response!