swift staticweb: traceback when listing files with accents

[Christophe Le Guern]

Swift version: 1.4.4

I have a container configured with staticweb as:
- swift ... post -r '.r:*' web
- swift ... post -m 'web-listings: true' web
- swift ... post -m 'web-index: index.html' web

The container contains no index.html, so it tries to do web listing.

If the container contains a file with accents such as "thé.txt", it gives me a traceback:
---
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.6/eventlet/wsgi.py", line 336, in handle_one_response
    result = self.application(self.environ, start_response)
  File "/usr/lib/pymodules/python2.6/swift/common/middleware/healthcheck.py", line 38, in __call__
    return self.app(env, start_response)
  File "/usr/lib/pymodules/python2.6/swift/common/middleware/memcache.py", line 32, in __call__
    return self.app(env, start_response)
  File "/usr/lib/pymodules/python2.6/swauth/middleware.py", line 221, in __call__
    return self.app(env, start_response)
  File "/usr/lib/pymodules/python2.6/swift/common/middleware/staticweb.py", line 479, in __call__
    return self._handle_container(env, start_response)
  File "/usr/lib/pymodules/python2.6/swift/common/middleware/staticweb.py", line 384, in _handle_container
    return self._listing(env, start_response)
  File "/usr/lib/pymodules/python2.6/swift/common/middleware/staticweb.py", line 349, in _listing
    quote(name), cgi.escape(name),
  File "/usr/lib/python2.6/urllib.py", line 1222, in quote
    res = map(safe_map.__getitem__, s)
KeyError: u'\xe9'
---

[Christophe Le Guern (c35sys)]

- Configuration: Nginx in frontend with proxy_pass
- Headers: ISO-8859-1 sent by staticweb

[Christophe Le Guern (c35sys)]

nginx now configured to provide utf-8, however I still have 500 error with traceback.

[Christophe Le Guern (c35sys)]

I added "catch_errors" to proxy pipeline, and I now have: "an error occured".
The syslog still contains the previous error.

[Christophe Le Guern (c35sys)]

I created a patch which is "working" for us.

The good thing is I can now display container listing without any error.

Test case:
- file with accent
- file containing accents

By working, it means it is only working when nginx (configured with utf8) is in front to display the file content (with accents).

The header served by staticweb for the test case (without nginx) is still ISO-8859-1, I don't know how to change this.

[gholt (gholt)]

Based on what you found and fixed, here's the patch I submitted for merging into Swift:

https://review.openstack.org/#change,3143

I'm not sure why your patch wasn't returning Content-Type: text/html; charset=UTF-8 from staticweb itself. It definitely is in my testing. Maybe give this new patch a try and see if something was just a little off?

[gholt (gholt)]

This does bring up the larger issue of other uses of quote within Swift. I think I'll pop a bug for looking into that.

[Christophe Le Guern (c35sys)]

I think I didn't explain very well the remaining problem when using the patch.

When using my patch, the listing is correct, displaying Content-type utf8, however, when I click the file (which is containing accents), it doesn't work, and the content type is ISO-8859-1 again.

Hope it's a bit more clear.

[gholt (gholt)]

Hmm, I think I'm still confused what the remaining problem is. Here's a log of a set of test commands I ran, after uploading a snowman-named file:

$ curl -i http://127.0.0.1:8080/v1/AUTH_test/testutf8/
HTTP/1.1 200 OK
Content-Type: text/html; charset=UTF-8
...
    <td class="colname"><a href="%E2%98%83">☃</a></td>
...

$ $ curl -i http://127.0.0.1:8080/v1/AUTH_test/testutf8/%E2%98%83
HTTP/1.1 200 OK
Last-Modified: Wed, 18 Jan 2012 19:01:22 GMT
Etag: 098f6bcd4621d373cade4e832627b4f6
X-Object-Meta-Mtime: 1326913085
Accept-Ranges: bytes
Content-Length: 4
Content-Type: application/octet-stream
Date: Wed, 18 Jan 2012 19:02:48 GMT

test

[Christophe Le Guern (c35sys)]

As an example, try to add "test éèà €" inside your file , then :

curl -i http://127.0.0.1:8080/v1/AUTH_test/testutf8/%E2%98%83

[gholt (gholt)]

Ah, in that case you're now talking about the content-type on the file itself, which has little to do with staticweb.

$ curl -XPUT -Hx-auth-token:AUTH_tk9dcb85ee914f4de4b81895be369cdabf http://127.0.0.1:8080/v1/AUTH_test/testutf8/%E2%98%83 --data-binary @testfile
... 201 Created ...

$ curl -i http://127.0.0.1:8080/v1/AUTH_test/testutf8/%E2%98%83
...
Content-Type: application/x-www-form-urlencoded
...

$ curl -XPUT -H 'content-type: text/html; charset=UTF-8' -Hx-auth-token:AUTH_tk9dcb85ee914f4de4b81895be369cdabf http://127.0.0.1:8080/v1/AUTH_test/testutf8/%E2%98%83 --data-binary @testfile
... 201 Created ...

$ curl -i http://127.0.0.1:8080/v1/AUTH_test/testutf8/%E2%98%83
...
Content-Type: text/html; charset=UTF-8
...

[Christophe Le Guern (c35sys)]

Thanks for the new patch, I deployed it and everthing is working as expected !

About file upload, I was a bit confused as I tried many options to upload (swift tool, php, ftp-cloudfs),
so thanks a lot for the curl tips !!!