unicode decoding problem in staticweb

[Eohyung Lee]

When prefix or subdir is unicode, it fails.

reproduce:
    # before test, enable staticweb middleware
 $ swift post test -r '.r:*'
 $ swift post test -m 'web-listings: true'

 $ mkdir ☃
 $ mkdir ☃/☃
 $ touch ☃/☃/file
 $ swift upload test ☃

 $ curl -v 'http://127.0.0.1/v1/AUTH_test/test/☃/'
 $ curl -v 'http://127.0.0.1/v1/AUTH_test/test/☃/☃/'

# test subdir
root@saio:~# curl -v 'http://127.0.0.1/v1/AUTH_test/test/☃/'
* About to connect() to 127.0.0.1 port 80 (#0)
* Trying 127.0.0.1... connected
* Connected to 127.0.0.1 (127.0.0.1) port 80 (#0)
> GET /v1/AUTH_test/test/☃/ HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-pc-linux-gnu) libcurl/7.19.7 OpenSSL/0.9.8k zlib/1.2.3.3 libidn/1.15
> Host: 127.0.0.1
> Accept: */*
>
< HTTP/1.1 500 Internal Error
< Content-Length: 17
< Content-Type: text/plain
< X-Trans-Id: txcba5fc6184d84c48ba04efc2cabe1c4c
< Date: Thu, 21 Mar 2013 01:49:16 GMT
<
* Connection #0 to host 127.0.0.1 left intact
* Closing connection #0
An error occurred

Mar 21 10:49:16 saio catch_errors Error: 'ascii' codec can't decode byte 0xe2 in position 157: ordinal not in range(128): Traceback (most recent call last): File "/usr/lib/python2.6/dist-packages/swift-1.7.6-py2.6.egg/swift/common/middleware/catch_errors.py", line 36, in handle_request resp = self._app_call(env) File "/usr/lib/python2.6/dist-packages/swift-1.7.6-py2.6.egg/swift/common/wsgi.py", line 270, in _app_call resp = self.app(env, self._start_response) File "/usr/lib/python2.6/dist-packages/swift-1.7.6-py2.6.egg/swift/common/middleware/healthcheck.py", line 57, in __call__ return self.app(env, start_response) File "/usr/lib/python2.6/dist-packages/swift-1.7.6-py2.6.egg/swift/common/middleware/memcache.py", line 65, in __call__ return self.app(env, start_response) File "/usr/local/lib/python2.6/dist-packages/swauth-1.0.7.dev-py2.6.egg/swauth/middleware.py", line 229, in __call__ return self.app(env, start_response) File "/usr/lib/python2.6/dist-packages/swift-1.7.6-py2.6.egg/swift/common/middleware/staticweb.py", line 559, in __call__ return context.handle_object(env, start_response) File "/usr/lib/python2.6/dist-packages/swift-1.7.6-py2.6.egg/swift/common/middleware/staticweb.py", line 437, in handle_object return self._listing(env, start_response, self.obj) File "/usr/lib/python2.6/dist-packages/swift-1.7.6-py2.6.egg/swift/common/middleware/staticweb.py", line 308, in _listing (quote(subdir), cgi.escape(subdir)) UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 157: ordinal not in range(128) (txn: txcba5fc6184d84c48ba04efc2cabe1c4c)

# test prefix
root@saio:~# curl -v 'http://127.0.0.1/v1/AUTH_test/test/☃/☃/'
* About to connect() to 127.0.0.1 port 80 (#0)
* Trying 127.0.0.1... connected
* Connected to 127.0.0.1 (127.0.0.1) port 80 (#0)
> GET /v1/AUTH_test/test/☃/☃/ HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-pc-linux-gnu) libcurl/7.19.7 OpenSSL/0.9.8k zlib/1.2.3.3 libidn/1.15
> Host: 127.0.0.1
> Accept: */*
>
< HTTP/1.1 500 Internal Error
< Content-Length: 17
< Content-Type: text/plain
< X-Trans-Id: tx8865b8247bf24941a1bd54a44bf8afba
< Date: Thu, 21 Mar 2013 01:49:57 GMT
<
* Connection #0 to host 127.0.0.1 left intact
* Closing connection #0
An error occurred

Mar 21 10:49:57 saio catch_errors Error: 'ascii' codec can't decode byte 0xe2 in position 157: ordinal not in range(128): Traceback (most recent call last): File "/usr/lib/python2.6/dist-packages/swift-1.7.6-py2.6.egg/swift/common/middleware/catch_errors.py", line 36, in handle_request resp = self._app_call(env) File "/usr/lib/python2.6/dist-packages/swift-1.7.6-py2.6.egg/swift/common/wsgi.py", line 270, in _app_call resp = self.app(env, self._start_response) File "/usr/lib/python2.6/dist-packages/swift-1.7.6-py2.6.egg/swift/common/middleware/healthcheck.py", line 57, in __call__ return self.app(env, start_response) File "/usr/lib/python2.6/dist-packages/swift-1.7.6-py2.6.egg/swift/common/middleware/memcache.py", line 65, in __call__ return self.app(env, start_response) File "/usr/local/lib/python2.6/dist-packages/swauth-1.0.7.dev-py2.6.egg/swauth/middleware.py", line 229, in __call__ return self.app(env, start_response) File "/usr/lib/python2.6/dist-packages/swift-1.7.6-py2.6.egg/swift/common/middleware/staticweb.py", line 559, in __call__ return context.handle_object(env, start_response) File "/usr/lib/python2.6/dist-packages/swift-1.7.6-py2.6.egg/swift/common/middleware/staticweb.py", line 437, in handle_object return self._listing(env, start_response, self.obj) File "/usr/lib/python2.6/dist-packages/swift-1.7.6-py2.6.egg/swift/common/middleware/staticweb.py", line 324, in _listing replace('T', ' ')) UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 157: ordinal not in range(128) (txn: tx8865b8247bf24941a1bd54a44bf8afba)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/24960

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/24960
Committed: http://github.com/openstack/swift/commit/05c8d61aebb39d8695c5b295ff399e87e4749540
Submitter: Jenkins
Branch: master

commit 05c8d61aebb39d8695c5b295ff399e87e4749540
Author: Eohyung Lee <email address hidden>
Date: Thu Mar 21 11:35:11 2013 +0900

    Fix for unicode issues in staticweb

    When prefix or subdir is unicode, it fails.
    Fixes: bug #1158092

    Change-Id: I966a7a198a5ca5ea152872c94240e8c2ebe57d36

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: milestone-proposed
Review: https://review.openstack.org/25682

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/25682
Committed: http://github.com/openstack/swift/commit/cd180b9022354bcb15e88e52a07eda178e12f5dc
Submitter: Jenkins
Branch: milestone-proposed

commit cd180b9022354bcb15e88e52a07eda178e12f5dc
Author: Eohyung Lee <email address hidden>
Date: Thu Mar 21 11:35:11 2013 +0900

    Fix for unicode issues in staticweb

    When prefix or subdir is unicode, it fails.
    Fixes: bug #1158092

    Change-Id: I966a7a198a5ca5ea152872c94240e8c2ebe57d36
    (cherry picked from commit 05c8d61aebb39d8695c5b295ff399e87e4749540)