Percent signs in object names cause trouble for versioned_writes

Bug #1755554 reported by Tim Burke on 2018-03-13
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Object Storage (swift)
High
Kota Tsuyuzaki

Bug Description

This is likely related to what goes wrong in https://bugs.launchpad.net/swift/+bug/1598093 and https://bugs.launchpad.net/swift/+bug/1678022

In a normal container, you can create objects named '%ff' just fine by encoding the percent sign as %25 in the request:

 $ curl http://saio:8090/v1/AUTH_test/c1 -X PUT

 $ curl -i http://saio:8090/v1/AUTH_test/c1/%25ff -X PUT -d 1234
 HTTP/1.1 201 Created
 Last-Modified: Tue, 13 Mar 2018 17:25:27 GMT
 Content-Length: 0
 Etag: 81dc9bdb52d04dc20036dbd8313ed055
 Content-Type: text/html; charset=UTF-8
 X-Trans-Id: tx6718c67d32364c96b1d59-005aa80986
 X-Openstack-Request-Id: tx6718c67d32364c96b1d59-005aa80986
 Date: Tue, 13 Mar 2018 17:25:26 GMT

But, if you try to do that with a versioned container, it fails with a 412:

 $ curl http://saio:8090/v1/AUTH_test/c2 -X PUT -H x-history-location:v

 $ curl http://saio:8090/v1/AUTH_test/v -X PUT

 $ curl http://saio:8090/v1/AUTH_test/c2/%25ff -X PUT -d 1234 -i
 HTTP/1.1 412 Precondition Failed
 Content-Length: 92
 Content-Type: text/html; charset=UTF-8
 X-Trans-Id: tx0e46490a8a50457a8cdba-005aa80a94
 X-Openstack-Request-Id: tx0e46490a8a50457a8cdba-005aa80a94
 Date: Tue, 13 Mar 2018 17:29:56 GMT

 <html><h1>Precondition Failed</h1><p>A precondition for this request was not met.</p></html>

Logs say

 Mar 13 17:29:56 saio proxy-server: - - 13/Mar/2018/17/29/56 GET /v1/AUTH_test/c2/%25FF HTTP/1.0 499 - Swift - - 29 - tx0e46490a8a50457a8cdba-005aa80a94 - 0.0003 VW - 1520962196.922559977 1520962196.922908068 -
 Mar 13 17:29:56 saio proxy-server: 127.0.0.1 127.0.0.1 13/Mar/2018/17/29/56 PUT /v1/AUTH_test/c2/%2525ff HTTP/1.0 412 - curl/7.35.0 - - 92 - tx0e46490a8a50457a8cdba-005aa80a94 - 0.0018 - - 1520962196.921812057 1520962196.923567057 -
 Mar 13 17:29:56 saio proxy-server: STDERR: 127.0.0.1 - - [13/Mar/2018 17:29:56] "PUT /v1/AUTH_test/c2/%25ff HTTP/1.1" 412 333 0.003989 (txn: tx0e46490a8a50457a8cdba-005aa80a94)

...which is particular strange in that there are *three* distinct object names recorded: %25FF, %2525ff, and %25ff

Adding another layer of encoding lets you PUT:

 $ curl http://saio:8090/v1/AUTH_test/c2/%2525ff -X PUT -d 1234 -i
 HTTP/1.1 201 Created
 Last-Modified: Tue, 13 Mar 2018 17:34:16 GMT
 Content-Length: 0
 Etag: 81dc9bdb52d04dc20036dbd8313ed055
 Content-Type: text/html; charset=UTF-8
 X-Trans-Id: tx06e6e8fe0b7f4fa0ada9b-005aa80b97
 X-Openstack-Request-Id: tx06e6e8fe0b7f4fa0ada9b-005aa80b97
 Date: Tue, 13 Mar 2018 17:34:15 GMT

...but it doesn't have the same name:

 $ curl http://saio:8090/v1/AUTH_test/c2
 %25ff

 $ curl http://saio:8090/v1/AUTH_test/c1
 %ff

Worse, versioning doesn't work for the object!

 $ curl http://saio:8090/v1/AUTH_test/c2/%2525ff -X PUT -d 5678 -i
 HTTP/1.1 201 Created
 Last-Modified: Tue, 13 Mar 2018 17:35:07 GMT
 Content-Length: 0
 Etag: 674f3c2c1a8a6f90461e8a66fb5550ba
 Content-Type: text/html; charset=UTF-8
 X-Trans-Id: txbe9856f0a8954beabc757-005aa80bca
 X-Openstack-Request-Id: txbe9856f0a8954beabc757-005aa80bca
 Date: Tue, 13 Mar 2018 17:35:06 GMT

 $ curl http://saio:8090/v1/AUTH_test/v -i
 HTTP/1.1 204 No Content
 X-Container-Object-Count: 0
 Accept-Ranges: bytes
 X-Storage-Policy: default
 Last-Modified: Tue, 13 Mar 2018 17:29:11 GMT
 X-Container-Bytes-Used: 0
 X-Timestamp: 1520961447.09430
 Content-Type: text/plain; charset=utf-8
 Content-Length: 0
 X-Trans-Id: tx258340276e704cea9442f-005aa80bcd
 X-Openstack-Request-Id: tx258340276e704cea9442f-005aa80bcd
 Date: Tue, 13 Mar 2018 17:35:09 GMT

Changed in swift:
importance: Undecided → High
Kota Tsuyuzaki (tsuyuzaki-kota) wrote :

OK, I confirmed this behavior exists with the current master. The one more odd thing is the successful PUT request with %%25ff was also logged as 409 response in the proxy-server but no any errors exist under the transaction id.

Mar 22 06:46:08 saio proxy-server: User: test uses token AUTH_tk156bef95123d40fe800845b539418eaa (trans_id txf6382b861f04463083921-005ab35130)
Mar 22 06:46:08 saio proxy-server: - - 22/Mar/2018/06/46/08 HEAD /v1/AUTH_test%3Fformat%3Djson HTTP/1.0 204 - Swift - - - - txf6382b861f04463083921-005ab35130 - 0.0106 GET_ACCOUNT_INFO - 1521701168.470243931 1521701168.480846882 -
Mar 22 06:46:08 saio proxy-server: - - 22/Mar/2018/06/46/08 HEAD /v1/AUTH_test/c2%3Fformat%3Djson HTTP/1.0 204 - Swift - - - - txf6382b861f04463083921-005ab35130 - 0.0198 GET_CONTAINER_INFO - 1521701168.481514931 1521701168.501288891 0
Mar 22 06:46:08 saio proxy-server: User test:tester has admin authorizing. (txn: txf6382b861f04463083921-005ab35130)
Mar 22 06:46:08 saio proxy-server: - - 22/Mar/2018/06/46/08 GET /v1/AUTH_test/c2/%2525ff HTTP/1.0 499 - Swift - - 70 - txf6382b861f04463083921-005ab35130 - 0.0245 VW - 1521701168.505810976 1521701168.530276060 0
Mar 22 06:46:08 saio proxy-server: User test:tester has admin authorizing. (txn: txf6382b861f04463083921-005ab35130) (client_ip: 127.0.0.1)
Mar 22 06:46:08 saio proxy-server: 127.0.0.1 127.0.0.1 22/Mar/2018/06/46/08 PUT /v1/AUTH_test/c2/%252525ff HTTP/1.0 201 - curl/7.47.0 AUTH_tk156bef951... 4 - - txf6382b861f04463083921-005ab35130 - 0.0935 - - 1521701168.469007015 1521701168.562541962 0
Mar 22 06:46:08 saio proxy-server: STDERR: 127.0.0.1 - - [22/Mar/2018 06:46:08] "PUT /v1/AUTH_test/c2/%2525ff HTTP/1.1" 201 314 0.096156 (txn: txf6382b861f04463083921-005ab35130)

I'll dive into deeper what pass was taken in the versions_writes middleware...

Changed in swift:
assignee: nobody → Kota Tsuyuzaki (tsuyuzaki-kota)

Fix proposed to branch: master
Review: https://review.openstack.org/555245

Changed in swift:
status: Confirmed → In Progress

Reviewed: https://review.openstack.org/555245
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=0e3e7b9b0953baaf7a0686881fd2348fde7d1e59
Submitter: Zuul
Branch: master

commit 0e3e7b9b0953baaf7a0686881fd2348fde7d1e59
Author: Kota Tsuyuzaki <email address hidden>
Date: Thu Mar 22 19:26:24 2018 +0900

    Fix versioned writes error with url-encoded object name

    With url encoded object name like '%25ff' that can be url-encoded
    value after decoded can cause 412 Precondition Failed. And more,
    that can do nothing (no versioned object creation) even it returns
    a successful response.

    The root causes are in versioned_writes middleware as follows:

    A. unnecessary unquote in object_request method
    B. incorrect use of make_pre_authed_request that takes 'quoted'
       path in the args. That is described at [1] explicitely.

    This patch resolved those 2 bugs at once, and then, now we can create
    %25ff versioned object reported in the launchpad with this patch.

    Perhaps, more tests would be nice to have. This patch added a few
    test cases on that.

    1: https://github.com/openstack/swift/blob/master/swift/common/wsgi.py#L1174

    Note that make_subrequest and its caller should have *quoted* path but
    make_env should *NOT*. That might make us confused.

    Closes-Bug: #1755554

    Change-Id: Ibcd90cc633c68973929ee5249c6598c22b342e3e

Changed in swift:
status: In Progress → Fix Released

Reviewed: https://review.openstack.org/570436
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=fbb8d7ebb52038fe445e2ca7e5c0459e8ff5ac5e
Submitter: Zuul
Branch: master

commit fbb8d7ebb52038fe445e2ca7e5c0459e8ff5ac5e
Author: Tim Burke <email address hidden>
Date: Thu May 24 13:00:58 2018 -0700

    Clarify that archive location headers should be URL-encoded

    Fix up function tests to actually *do* that quoting, and fix
    _listing_pages_iter to respect that.

    Change-Id: I1554042510819ea878b4c70417721944115e17f4
    Related-Bug: 1229142
    Related-Change: I425440f76b8328f8e119d390bfa4c7022181e89e
    Related-Bug: 1755554
    Related-Change: Ibcd90cc633c68973929ee5249c6598c22b342e3e

This issue was fixed in the openstack/swift 2.18.0 release.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers