[packaging] UnicodeDecodeError when LANG not set

[Jason Gerard DeRose]

Note, this is NOT an upstream PySkein bug!

A combination of an upstream Python bug (http://bugs.python.org/issue10419) and the fact that the Launchpad build servers run without any locale set (LANG environment variable is not set) is causing PySkein builds to fail with:

  UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 71: ordinal not in range(128)

Full build log:

  http://launchpadlibrarian.net/63011800/buildlog_ubuntu-natty-i386.pyskein_0.7.0-0ubuntu3_FAILEDTOBUILD.txt.gz

You can easily reproduce it in the PySkein tree like this:

  LANG=C python3 setup.py build

For now the best solution (I think) is probably to set LANG=en_US.UTF-8 in debian/rules, but I'm not debhelper or make savy enough to know how to do this. Help greatly appreciated, otherwise I'll keep plodding along with this.

[Jason Gerard DeRose (jderose)]

Hmm, I thought I fixed this as of r15 in lp:~pyskein/pyskein/packaging but I'm not sure now.

Even with an `export LANG=C` before running `bzr builddeb` I'm having a hard time reproducing this locally.

[Jason Gerard DeRose (jderose)]

Hmm, no r15 didn't fix it:

http://launchpadlibrarian.net/63014569/buildlog_ubuntu-natty-i386.pyskein_0.7.0-0ubuntu4_FAILEDTOBUILD.txt.gz

[Colin Watson (cjwatson)]

If you need a UTF-8 locale in a build context, you need to generate it dynamically, as you cannot assume that it is already available. Something like this:

debian/tmp-locale:
        mkdir -p $@
        if ! localedef -i en_US -c -f UTF-8 -A /usr/share/locale/locale.alias --quiet $@/en_US.UTF-8; then \
                rm -rf $@; \
                exit 1; \
        fi

Then make your other targets depend on debian/tmp-locale, run setup.py with LOCPATH=$(CURDIR)/debian/tmp-locale, and rm -rf debian/tmp-locale in an override_dh_clean target.

[Jason Gerard DeRose (jderose)]

Just pasting in some advice Colin gave me on IRC so I don't loose track of it:

(08:46:09 AM) jderose: cjwatson: quick question: how do I "make your other targets depend on debian/tmp-locale"? sorry, not very make savvy
(08:49:30 AM) cjwatson: jderose: like this: override_dh_install: debian/tmp-locale
(08:49:56 AM) jderose: cjwatson: okay, thanks again!
(08:50:20 AM) cjwatson: (sorry, override_dh_auto_install in your case, but you get the idea)

[Jason Gerard DeRose (jderose)]

Oops, just realized I linked to the wrong upstream Python bug. It's this one - http://bugs.python.org/issue9561

[Jason Gerard DeRose (jderose)]

And this might also be related - http://bugs.python.org/issue8409

[Jason Gerard DeRose (jderose)]

Thanks to some patient help from kklimonda (and others) in #ubuntu-motu, this build issue is fixed as of revno 20 in lp:~pyskein/pyskein/packaging

Thanks for the help, everyone!