Object Versioning for Large Object does not work

Bug #1365862 reported by Daisuke Morita
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Object Storage (swift)
Fix Released
Undecided
Alistair Coles

Bug Description

When SLO/DLO manifest is contained in the container with X-Versions-Location header, the manifest is copied to the versions-location container for overriding PUT request. However, when the manifest is restored by deleting overriding one, the manifest does no longer work as expected.

For example in SLO:

$ curl -v -X PUT -H "X-Auth-Token: <token>" -H "X-Versions-Location: versioned" http://localhost:8080/v1/AUTH_test/slotest
$ curl -v -X PUT -d @part1 -H "X-Auth-Token: <token>" http://localhost:8080/v1/AUTH_test/slotest/part1
$ curl -v -X PUT -d @part2 -H "X-Auth-Token: <token>" http://localhost:8080/v1/AUTH_test/slotest/part2
$ curl -v -X PUT -d @manifest -H "X-Auth-Token: <token>" http://localhost:8080/v1/AUTH_test/slotest/slo?multipart-manifest=put
$ curl -v -X HEAD -H "X-Auth-Token: <token>" http://localhost:8080/v1/AUTH_test/slotest/slo
< HTTP/1.1 200 OK
< Content-Length: 2097152
< Accept-Ranges: bytes
< Last-Modified: Fri, 05 Sep 2014 06:57:41 GMT
< Etag: "6538fda7d1a764bbc1587b91f9d8d903"
< X-Timestamp: 1409900260.68222
< X-Static-Large-Object: True
< Content-Type: application/x-www-form-urlencoded
< X-Trans-Id: txaa55b130048a410fa339d-0054095ee7
< Date: Fri, 05 Sep 2014 06:57:43 GMT

# Suppose parts of large object are updated
$ curl -v -X PUT -d @part1 -H "X-Auth-Token: <token>" http://localhost:8080/v1/AUTH_test/slotest/part1
$ curl -v -X PUT -d @part2 -H "X-Auth-Token: <token>" http://localhost:8080/v1/AUTH_test/slotest/part2
$ curl -v -X PUT -d @manifest -H "X-Auth-Token: <token>" http://localhost:8080/v1/AUTH_test/slotest/slo?multipart-manifest=put
$ curl -v -X HEAD -H "X-Auth-Token: <token>" http://localhost:8080/v1/AUTH_test/slotest/slo
< HTTP/1.1 200 OK
< Content-Length: 2097152
< Accept-Ranges: bytes
< Last-Modified: Fri, 05 Sep 2014 06:59:51 GMT
< Etag: "a025ac8e76cd51e8baa3b38f3ab21d7d"
< X-Timestamp: 1409900390.15489
< X-Static-Large-Object: True
< Content-Type: application/x-www-form-urlencoded
< X-Trans-Id: tx0d7041a5203648e4b1e37-0054095f74
< Date: Fri, 05 Sep 2014 07:00:04 GMT

# Suppose you find some mistakes in uploaded file, then you want to revert old large object
$ curl -v -X DELETE -d @manifest -H "X-Auth-Token: <token>" http://localhost:8080/v1/AUTH_test/slotest/slo?multipart-manifest=delete
< HTTP/1.1 200 OK
< Content-Type: text/plain
< X-Trans-Id: tx3c8db72d65ff4789a4f6a-00540960b6
< Date: Fri, 05 Sep 2014 07:05:26 GMT
< Transfer-Encoding: chunked
<
Number Deleted: 3
Number Not Found: 0
Response Body:
Response Status: 200 OK
Errors:
$ curl -v -X GET -H "X-Auth-Token: <token>" http://localhost:8080/v1/AUTH_test/slotest/slo
< HTTP/1.1 200 OK
< Content-Length: 380
< Accept-Ranges: bytes
< Last-Modified: Fri, 05 Sep 2014 07:05:27 GMT
< Etag: fb72065d4ac25373fb45977fce3094db
< X-Timestamp: 1409900726.67266
< Content-Type: application/x-www-form-urlencoded
< X-Trans-Id: tx4e9cdd11641045ee912a0-00540960e1
< Date: Fri, 05 Sep 2014 07:06:09 GMT
<
[{"hash": "068a2aa5d4c38253ffca32aaf3bf6c54", "last_modified": "2014-09-05T05:32:08.000000", "bytes": 1048576, "name": "/slotest/part1", "content_type": "application/x-www-form-urlencoded"}, {"hash": "a055f22623eddc772b8974846ab3c9fe", "last_modified": "2014-09-05T05:33:53.000000", "bytes": 1048576, "name": "/slotest/part2", "content_type": "application/x-www-form-urlencoded"}]

The root cause of this behavior seems that X-Static-Large-Object or X-Object-Manifest header is not copied when old version object is copied to versions-location container.

Zhou Yuan (yuan-zhou)
Changed in swift:
status: New → Confirmed
assignee: nobody → Zhou Yuan (yuan-zhou)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to swift (master)

Fix proposed to branch: master
Review: https://review.openstack.org/123765

Changed in swift:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on swift (master)

Change abandoned by John Dickinson (<email address hidden>) on branch: master
Review: https://review.openstack.org/123765
Reason: Abandoning due to lack of activity since the last negative review. You can restore the change if you want to keep working on it.

Revision history for this message
Alistair Coles (alistair-coles) wrote :

I can't reproduce this bug on 2.5.0, but it seems to be replaced by another bug: https://bugs.launchpad.net/swift/+bug/1537042

Changed in swift:
status: In Progress → Won't Fix
Revision history for this message
Alistair Coles (alistair-coles) wrote :

Actually I did reproduce this bug once I had corrected the position of versioned_writes in the proxy pipeline, where the auto-insert logic was misplacing it to the left of SLO. See bug https://bugs.launchpad.net/swift/+bug/1537042

Changed in swift:
status: Won't Fix → Confirmed
Changed in swift:
assignee: Zhou Yuan (yuan-zhou) → Alistair Coles (alistair-coles)
Changed in swift:
status: Confirmed → In Progress
Revision history for this message
Alistair Coles (alistair-coles) wrote :
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to swift (master)

Reviewed: https://review.openstack.org/260179
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=b13a85367ef7f5fd64e285174859025d34b4e451
Submitter: Jenkins
Branch: master

commit b13a85367ef7f5fd64e285174859025d34b4e451
Author: Thiago da Silva <email address hidden>
Date: Mon Dec 21 16:25:45 2015 -0200

    decouple versioned writes from COPY

    This change removes the use of the COPY request in the versioned
    writes middleware. It changes the COPY verb for GETs and PUTs
    requests. The main reasoning for this change is to remove any
    dependency that versioning had on copy, which will allow for the COPY
    functionality to be moved to middleware and to be to the left of the
    versioned writes middleware in the proxy pipeline. In this way,
    no COPY request will ever arrive at the versioned writes middleware.

    A side benefit of this change is that it removes a HEAD request from
    the PUT path. Instead of checking if a current version exists, a
    GET request is sent, in case of success, a PUT is sent to the
    versions container.

    A unit test was removed that tested non-default storage policies.
    This test is no longer necessary, since it was used to test
    specific policy handling code in the COPY method in the proxy
    object controller.

    Closes-Bug: #1365862

    Change-Id: Idf34fa8d04ff292df7134b6d4aa94ff40887b3a4
    Co-Authored-By: Alistair Coles <email address hidden>
    Co-Authored-By: Janie Richling <email address hidden>
    Co-Authored-By: Kota Tsuyuzaki <email address hidden>
    Signed-off-by: Thiago da Silva <email address hidden>

Changed in swift:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to swift (feature/crypto)

Fix proposed to branch: feature/crypto
Review: https://review.openstack.org/308874

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to swift (feature/crypto)
Download full text (8.2 KiB)

Reviewed: https://review.openstack.org/308874
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=bc138e74cc60d94710486365636a5153422e00a3
Submitter: Jenkins
Branch: feature/crypto

commit 7d7eaab5afa4c36f0ac8467784138fc423f6ac4f
Author: OpenStack Proposal Bot <email address hidden>
Date: Mon Apr 18 06:31:34 2016 +0000

    Imported Translations from Zanata

    For more information about this automatic import see:
    https://wiki.openstack.org/wiki/Translations/Infrastructure

    Change-Id: I88be5c9dbb1fcc3a15592d42af7160478308b1f4

commit ca304cd08e9f8d37e4027f2f71dd77ebba3a30f9
Author: Samuel Merritt <email address hidden>
Date: Fri Apr 15 17:22:44 2016 -0700

    Ignore negative suffix-byte-range requests.

    If the client asked for "Range: bytes=--123", Swift would respond with
    a 206 and a Content-Length of -123. Now that Range header is ignored
    just like all kinds of other invalid Range headers.

    Change-Id: I30d4522d223076ce342d20c52f57ff0eb2aea1f4
    Closes-Bug: 1571106

commit 746d928a875281a7154dcd438f46a58bbf656db9
Author: Dmitriy Ukhlov <email address hidden>
Date: Fri Apr 8 16:00:16 2016 +0300

    Adds eventlet monkey patching of select module if thread is pathed

    Oslo.messaging pika driver requires patching of select module if thread
    is patched.
    Pika driver uses select call and if it is not patched onsuming messages
    blocks whole eventlet loop

    Closes-Bug: #1570242
    Change-Id: I9756737309f401ebddb7475eb84725f65bca01bf

commit c96d5c671db9c96f65067d93c0a307cf4b7d91b4
Author: oshritf <email address hidden>
Date: Thu Feb 18 14:50:08 2016 +0200

    Per container stat. report

    In addition to the container sync stat. report, keeping per container
    statistics allows administrator with more control over bytes
    transfered over a specific time per user account: The per container stats
    are crucial for billing purposes and provides the operator a 'progress
    bar' equivalent on the container's replication status.

    Change-Id: Ia8abcdaf53e466e8d60a957c76e32c2b2c5dc3fa

commit be84b03a07892f4972dd59309ad261fc72bd7ede
Author: dharmendra <email address hidden>
Date: Thu Apr 14 16:41:09 2016 +0530

    Removing unused clause

    Removing unused clause from common/middleware/dlo.py

    Change-Id: I7de9aaefd203c4f1be00ee74b89b5184fd419469

commit fb3692c9bb662d9cadc4238920f86676857a7f1f
Author: Tim Burke <email address hidden>
Date: Wed Apr 13 11:07:44 2016 -0700

    Don't include conditional headers in SLO HEAD requests

    Previously, attempting to PUT a SLO manifest with `If-None-Match: *`
    would include the header when validating the segments, causing the
    upload to fail.

    Now when SLO validates segments, no conditional headers will be
    included in the HEAD request.

    Change-Id: I03ad454092d3caa73d29e6d30d8033b45bc96136
    Closes-Bug: #1569253

commit 0e7fca576cee81dd6ca8774760cb880c3fff9c1c
Author: Tin Lam <email address hidden>
Date: Fri Apr 8 18:17:44 2016 -0500

    Convert README.md to README.rst

    Change-Id: I223890bd4ffe469becc2127f9362243cdb52b...

Read more...

tags: added: in-feature-crypto
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to swift (feature/hummingbird)

Fix proposed to branch: feature/hummingbird
Review: https://review.openstack.org/323599

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to swift (feature/hummingbird)
Download full text (84.7 KiB)

Reviewed: https://review.openstack.org/323599
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=0330478b70d0a699a0f9c21ef87c7e639d92564b
Submitter: Jenkins
Branch: feature/hummingbird

commit 5fe392b562de3baed080704df433fb392cb4fb31
Author: Ondřej Nový <email address hidden>
Date: Tue May 31 16:25:50 2016 +0200

    Fixed typo

    Change-Id: I7a35c0076360c7a23cf405189828d3c252ec6708

commit b52eccb3b1ea0591f0040587228d3705b5d3f68d
Author: Clay Gerrard <email address hidden>
Date: Wed May 25 11:21:25 2016 -0700

    Clarify overload best practices in admin guide

    Change-Id: Ib7c08bdeab6374771bb8e2b05053e7e16973524d

commit f1fd50723bb84c4941e949895576733f6eb67793
Author: Christian Schwede <email address hidden>
Date: Wed May 25 09:53:31 2016 +0200

    Add dispersion --verbose example to admin guide

    Change-Id: I5f9cacedde2a329332ccf744800b6f2453e8b28e

commit b3ab715c055283ccfea9a504d6da20741d82e7ad
Author: Matthew Oliver <email address hidden>
Date: Wed May 25 14:35:54 2016 +1000

    Add ring-builder dispersion command to admin guide

    This change updates the admin guide to point out the dispersion command
    in swift-ring-builder and mentions the dispersion verbose table to make
    it more obvious to operators.

    Change-Id: I72b4c8b2d718e6063de0fdabbaf4f2b73694e0a4

commit fb7a8e9ab7596a36a6992a3a8f8c6d005a2c2829
Author: Tim Burke <email address hidden>
Date: Tue May 24 13:37:58 2016 -0700

    Add links to mitaka install guides

    Change-Id: I62331923751c521daded4468b5cc5f03655226bc

commit e09c4ee7800e82aa09ca2f6ae375420b766182a4
Author: Tim Burke <email address hidden>
Date: Fri Apr 29 12:12:00 2016 -0500

    Allow concurrent bulk deletes

    Before, server-side deletes of static large objects could take a long
    time to complete since the proxy would wait for a response to each
    segment DELETE before starting the next DELETE request.

    Now, operators can configure a concurrency factor for the slo and bulk
    middlewares to allow up to N concurrent DELETE requests. By default, two
    DELETE requests will be allowed at a time.

    Note that objects and containers are now deleted in separate passes, to
    reduce the likelihood of 409 Conflict responses when deleting
    containers.

    Upgrade Consideration
    =====================
    If operators have enabled the bulk or slo middlewares and would like to
    preserve the prior (single-threaded) DELETE behavior, they must add the
    following line to their [filter:slo] and [filter:bulk] proxy config
    sections:

       delete_concurrency = 1

    This may be done prior to upgrading Swift.

    UpgradeImpact
    Closes-Bug: 1524454
    Change-Id: I128374d74a4cef7a479b221fd15eec785cc4694a

commit 226557afc42c245e050d84162497f46341407ef7
Author: Tim Burke <email address hidden>
Date: Thu May 19 18:55:40 2016 -0700

    Turn on H703, so our translators don't punch us

    Change-Id: I4ce3068f79563e4d4296c6e1078bc12f0cf84c96
    Related-Bug: 1559431

commit 7b706926a8ed5bbcec3a678e868e301c9a6ed8f1
Author: Alistair Coles <email address hidden>
Date: Mon May ...

tags: added: in-feature-hummingbird
Revision history for this message
Doug Hellmann (doug-hellmann) wrote : Fix included in openstack/swift 2.8.0

This issue was fixed in the openstack/swift 2.8.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.