Upload failed if filename is unicode string in formpost (python3)
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenStack Object Storage (swift) |
Fix Released
|
High
|
Unassigned |
Bug Description
If the filename in form-data is unicode string, the proxy-server occure exception.
I try to reproduce this in test code to change the filename with unicode. But the all test case passed.
This is my workaround solution to fix this problem.
I changed this line.
https:/
```
filename = attributes[
subenv['PATH_INFO'] += filename or 'filename'
```
I have referenced how the eventlet's wsgi process the path_info.
https:/
=======
Here is a error log
```
2019-12-
Traceback (most recent call last):
File "/usr/lib/
yield chunk
GeneratorExit
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/
resp = self._app_call(env)
File "/usr/lib/
resp = self.app(env, self._start_
File "/usr/lib/
return self.app(env, gatekeeper_
File "/usr/lib/
iterable = self.app(env, my_start_response)
File "/usr/lib/
return self.app(env, start_response)
File "/usr/lib/
return self.app(env, start_response)
File "/usr/lib/
return self.app(env, start_response)
File "/usr/lib/
return func(self, Request(env))(env, start_response)
File "/usr/lib/
return func(self, Request(env))(env, start_response)
File "/usr/lib/
return self.app(env, start_response)
File "/usr/lib/
return self.app(env, start_response)
File "/usr/lib/
resp = self.call_func(req, *args, **kw)
File "/usr/lib/
return self.func(req, *args, **kwargs)
File "/usr/lib/
response = req.get_
File "/usr/lib/
application, catch_exc_
File "/usr/lib/
app_iter = application(
File "/usr/lib/
return resp(env, start_response)
File "/usr/lib/
return self.app(env, start_response)
File "/usr/lib/
return self._app(environ, start_response)
File "/usr/lib/
return self.app(environ, keystone_
File "/usr/lib/
env, attrs['boundary'])
File "/usr/lib/
self.
File "/usr/lib/
close_
File "/usr/lib/
return close_method()
File "/usr/lib/
close_method()
File "/usr/lib/
start_time, time.time(), resp_headers=
File "/usr/lib/
if req.path.
File "/usr/lib/
self.
File "/usr/lib/
raise TypeError('Expected a WSGI string; got %r' % wsgi_str)
TypeError: Expected a WSGI string; got '/v1/AUTH_
```
Funny -- I tried uploading a file named ☃ (snowman) and my browser seems to have turned that into ... and HTML entity?? "☃"
But putting it in the upload prefix got me a traceback! A little different from yours, though:
Jan 7 23:45:34 saio proxy-server: get_keys(): from callback: 'latin-1' codec can't encode character '\u2603' in position 24: ordinal not in range(256): swift/swift/ common/ middleware/ crypto/ crypto_ utils.py" , line 170, in get_keys keys(key_ id=key_ id) swift/swift/ common/ middleware/ crypto/ keymaster. py", line 130, in fetch_crypto_keys id=secret_ id) swift/swift/ common/ middleware/ crypto/ keymaster. py", line 299, in create_key bytes(path) , swift/swift/ common/ swob.py" , line 279, in wsgi_to_bytes encode( 'latin1' )
Traceback (most recent call last):
File "/vagrant/
keys = fetch_crypto_
File "/vagrant/
path, secret_
File "/vagrant/
return hmac.new(key, wsgi_to_
File "/vagrant/
return wsgi_str.
UnicodeEncodeError: 'latin-1' codec can't encode character '\u2603' in position 24: ordinal not in range(256)
I can get a similar error using curl to get a snowman in the filename:
$ curl -F redirect= /v1/AUTH_ test/c/ thanks. html -F maxmax_file_count=3 -F expires=1578445456 -F signature= b60d8d455deb53f 8a2a50a794ab155 c8a9925b5c -F file0=" @/home/ vagrant/ .s3cfg; filename= ☃" https:/ /saio/v1/ AUTH_test/ c/upload_ -v
So I tried disabling encryption -- which caused a pretty fun blow-up, this time coming out on STDERR:
Exception ignored in: dleware. __call_ _.<locals> .iter_response at 0x7f8910b5dc50> swift/swift/ common/ middleware/ proxy_logging. py", line 409, in iter_response resp_headers) swift/swift/ common/ middleware/ proxy_logging. py", line 224, in log_request startswith( '/v1/') : swift/swift/ common/ swob.py" , line 1070, in path 'PATH_INFO' ]) swift/swift/ common/ swob.py" , line 309, in wsgi_quote test/c/ upload_ ☃'
<generator object ProxyLoggingMid
Traceback (most recent call last):
File "/vagrant/
start_time, time.time(), resp_headers=
File "/vagrant/
if req.path.
File "/vagrant/
self.environ[
File "/vagrant/
raise TypeError('Expected a WSGI string; got %r' % wsgi_str)
TypeError
:
Expected a WSGI string; got '/v1/AUTH_
Which is looking more familiar. Differences might be explained by changes on master since train to quote more paths when logging? At any rate, that's certainly close enough for me to call this confirmed!
Probably off-topic: My first try with curl led to... something strange:
$ curl -F max_file_ size=102400 -F max_file_count=3 -F expires=1578444242 -F signature= df2b9b797125c7f f2dbe320206538f 6190f17a78 -F file0=" @~/.s3cfg; filename= ☃" https:/ /saio/v1/ AUTH_test/ c/let_it_☃_ -v test/c/ let_it_ ☃_ HTTP/1.1 form-data; boundary= ------- ------- ------- ---fe289f46b0bc 0e60
Warning: setting file ~/.s3cfg failed!
* Trying 127.0.1.1...
... <snip> ...
> POST /v1/AUTH_
> Host: saio
> User-Agent: curl/7.58.0
> Accept: */*
> Transfer-Encoding: chunked
> Content-Type: multipart/
> Expect: 100-continue
>
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS Unknown, Unknown (23...