Percent signs in object names cause trouble for versioned_writes
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenStack Object Storage (swift) |
Fix Released
|
High
|
Kota Tsuyuzaki |
Bug Description
This is likely related to what goes wrong in https:/
In a normal container, you can create objects named '%ff' just fine by encoding the percent sign as %25 in the request:
$ curl http://
$ curl -i http://
HTTP/1.1 201 Created
Last-Modified: Tue, 13 Mar 2018 17:25:27 GMT
Content-Length: 0
Etag: 81dc9bdb52d04dc
Content-Type: text/html; charset=UTF-8
X-Trans-Id: tx6718c67d32364
X-Openstack-
Date: Tue, 13 Mar 2018 17:25:26 GMT
But, if you try to do that with a versioned container, it fails with a 412:
$ curl http://
$ curl http://
$ curl http://
HTTP/1.1 412 Precondition Failed
Content-Length: 92
Content-Type: text/html; charset=UTF-8
X-Trans-Id: tx0e46490a8a504
X-Openstack-
Date: Tue, 13 Mar 2018 17:29:56 GMT
<html>
Logs say
Mar 13 17:29:56 saio proxy-server: - - 13/Mar/
Mar 13 17:29:56 saio proxy-server: 127.0.0.1 127.0.0.1 13/Mar/
Mar 13 17:29:56 saio proxy-server: STDERR: 127.0.0.1 - - [13/Mar/2018 17:29:56] "PUT /v1/AUTH_
...which is particular strange in that there are *three* distinct object names recorded: %25FF, %2525ff, and %25ff
Adding another layer of encoding lets you PUT:
$ curl http://
HTTP/1.1 201 Created
Last-Modified: Tue, 13 Mar 2018 17:34:16 GMT
Content-Length: 0
Etag: 81dc9bdb52d04dc
Content-Type: text/html; charset=UTF-8
X-Trans-Id: tx06e6e8fe0b7f4
X-Openstack-
Date: Tue, 13 Mar 2018 17:34:15 GMT
...but it doesn't have the same name:
$ curl http://
%25ff
$ curl http://
%ff
Worse, versioning doesn't work for the object!
$ curl http://
HTTP/1.1 201 Created
Last-Modified: Tue, 13 Mar 2018 17:35:07 GMT
Content-Length: 0
Etag: 674f3c2c1a8a6f9
Content-Type: text/html; charset=UTF-8
X-Trans-Id: txbe9856f0a8954
X-Openstack-
Date: Tue, 13 Mar 2018 17:35:06 GMT
$ curl http://
HTTP/1.1 204 No Content
X-Container-
Accept-Ranges: bytes
X-Storage-Policy: default
Last-Modified: Tue, 13 Mar 2018 17:29:11 GMT
X-Container-
X-Timestamp: 1520961447.09430
Content-Type: text/plain; charset=utf-8
Content-Length: 0
X-Trans-Id: tx258340276e704
X-Openstack-
Date: Tue, 13 Mar 2018 17:35:09 GMT
Changed in swift: | |
importance: | Undecided → High |
Changed in swift: | |
assignee: | nobody → Kota Tsuyuzaki (tsuyuzaki-kota) |
OK, I confirmed this behavior exists with the current master. The one more odd thing is the successful PUT request with %%25ff was also logged as 409 response in the proxy-server but no any errors exist under the transaction id.
Mar 22 06:46:08 saio proxy-server: User: test uses token AUTH_tk156bef95 123d40fe800845b 539418eaa (trans_id txf6382b861f044 63083921- 005ab35130) 2018/06/ 46/08 HEAD /v1/AUTH_ test%3Fformat% 3Djson HTTP/1.0 204 - Swift - - - - txf6382b861f044 63083921- 005ab35130 - 0.0106 GET_ACCOUNT_INFO - 1521701168. 470243931 1521701168. 480846882 - 2018/06/ 46/08 HEAD /v1/AUTH_ test/c2% 3Fformat% 3Djson HTTP/1.0 204 - Swift - - - - txf6382b861f044 63083921- 005ab35130 - 0.0198 GET_CONTAINER_INFO - 1521701168. 481514931 1521701168. 501288891 0 63083921- 005ab35130) 2018/06/ 46/08 GET /v1/AUTH_ test/c2/ %2525ff HTTP/1.0 499 - Swift - - 70 - txf6382b861f044 63083921- 005ab35130 - 0.0245 VW - 1521701168. 505810976 1521701168. 530276060 0 63083921- 005ab35130) (client_ip: 127.0.0.1) 2018/06/ 46/08 PUT /v1/AUTH_ test/c2/ %252525ff HTTP/1.0 201 - curl/7.47.0 AUTH_tk156bef951... 4 - - txf6382b861f044 63083921- 005ab35130 - 0.0935 - - 1521701168. 469007015 1521701168. 562541962 0 test/c2/ %2525ff HTTP/1.1" 201 314 0.096156 (txn: txf6382b861f044 63083921- 005ab35130)
Mar 22 06:46:08 saio proxy-server: - - 22/Mar/
Mar 22 06:46:08 saio proxy-server: - - 22/Mar/
Mar 22 06:46:08 saio proxy-server: User test:tester has admin authorizing. (txn: txf6382b861f044
Mar 22 06:46:08 saio proxy-server: - - 22/Mar/
Mar 22 06:46:08 saio proxy-server: User test:tester has admin authorizing. (txn: txf6382b861f044
Mar 22 06:46:08 saio proxy-server: 127.0.0.1 127.0.0.1 22/Mar/
Mar 22 06:46:08 saio proxy-server: STDERR: 127.0.0.1 - - [22/Mar/2018 06:46:08] "PUT /v1/AUTH_
I'll dive into deeper what pass was taken in the versions_writes middleware...