'Content-Length' header does not show up for DLO GET with 10k+ segments

Bug #1680219 reported by Andrew Welleck
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Object Storage (swift)
Fix Released
Undecided
Kazuhiro MIYAHARA

Bug Description

We have noticed that when doing a GET on a DLO that is segmented 10k+ times the 'Content-Length' header is not included in the response. I was able to recreate this in an SAIO using python-swiftclient to upload a 10k byte object. Then using curl to create the object manifest:
curl -v -XPUT -H "X-Auth-Token:$TOKEN" -H "X-Object-Manifest:container1_segment/10kb.bin" http://localhost/auth/v1.0/
And finally curl to GET the object:
curl -XGET -i $STORAGE_URL/container1/10kb.bin -H X-Auth-Token:$TOKEN
When testing an object with slightly less than 10k segments the response header includes the 'Content-Length'.

Revision history for this message
Tim Burke (1-tim-z) wrote :

Correct -- the Content-Length header is only included if all DLO segments fit within a single listing page. This is in part because by the time that we get to the second (or third, or fourth, ...) listing page, different objects may appear in the listing. If you want Content-Length returned on all GETs/HEADs, you should use SLOs (which offer more guarantees about the composition and integrity of the large object).

We should call this out in the documentation at https://docs.openstack.org/swift/latest/api/large_objects.html#dynamic-large-objects

tags: added: documentation
Changed in swift:
status: New → Confirmed
tags: added: low-hanging-fruit
Revision history for this message
Samuel Merritt (torgomatic) wrote :

It's also meant to protect the cluster. Swift could, in principle, get all the pages of the container listing and compute across them, but that leaves us vulnerable to someone uploading hundreds of millions of tiny objects to a single container, making a DLO that references them all, and then making a whole lot of HEAD requests for that DLO, causing huge amounts of load on that particular container.

The optimal maximum number of container-listing pages to retrieve is debatable; it's at least one, but less than a million. In any case, it's finite, so users will always be able to make DLOs with more segments than we are willing to retrieve.

Changed in swift:
assignee: nobody → Kazuhiro MIYAHARA (miyahara-kazuhiro)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to swift (master)

Fix proposed to branch: master
Review: https://review.openstack.org/501103

Changed in swift:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to swift (master)

Reviewed: https://review.openstack.org/501103
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=a2f557e92103eb1b105db1099e10f38034a0609f
Submitter: Jenkins
Branch: master

commit a2f557e92103eb1b105db1099e10f38034a0609f
Author: Kazuhiro MIYAHARA <email address hidden>
Date: Wed Sep 6 04:03:10 2017 +0000

    Add explanation about Content-Length header limit with DLO to docs

    If a number of DLO segments is larger than container listing limit,
    Content-Length header will not be included in GET or HEAD response.
    However, this fact is not explained in document of large objects.

    This patch add explanation about this fact to the document.

    Change-Id: Ia45fad05797f38fa8b6b0ed917b4f9d7fb337149
    Closes-Bug: 1680219

Changed in swift:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/swift 2.16.0

This issue was fixed in the openstack/swift 2.16.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.