'Content-Length' header does not show up for DLO GET with 10k+ segments

[Andrew Welleck]

We have noticed that when doing a GET on a DLO that is segmented 10k+ times the 'Content-Length' header is not included in the response. I was able to recreate this in an SAIO using python-swiftclient to upload a 10k byte object. Then using curl to create the object manifest:
curl -v -XPUT -H "X-Auth-Token:$TOKEN" -H "X-Object-Manifest:container1_segment/10kb.bin" http://localhost/auth/v1.0/
And finally curl to GET the object:
curl -XGET -i $STORAGE_URL/container1/10kb.bin -H X-Auth-Token:$TOKEN
When testing an object with slightly less than 10k segments the response header includes the 'Content-Length'.

[Tim Burke (1-tim-z)]

Correct -- the Content-Length header is only included if all DLO segments fit within a single listing page. This is in part because by the time that we get to the second (or third, or fourth, ...) listing page, different objects may appear in the listing. If you want Content-Length returned on all GETs/HEADs, you should use SLOs (which offer more guarantees about the composition and integrity of the large object).

We should call this out in the documentation at https://docs.openstack.org/swift/latest/api/large_objects.html#dynamic-large-objects

[Samuel Merritt (torgomatic)]

It's also meant to protect the cluster. Swift could, in principle, get all the pages of the container listing and compute across them, but that leaves us vulnerable to someone uploading hundreds of millions of tiny objects to a single container, making a DLO that references them all, and then making a whole lot of HEAD requests for that DLO, causing huge amounts of load on that particular container.

The optimal maximum number of container-listing pages to retrieve is debatable; it's at least one, but less than a million. In any case, it's finite, so users will always be able to make DLOs with more segments than we are willing to retrieve.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/501103

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/501103
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=a2f557e92103eb1b105db1099e10f38034a0609f
Submitter: Jenkins
Branch: master

commit a2f557e92103eb1b105db1099e10f38034a0609f
Author: Kazuhiro MIYAHARA <email address hidden>
Date: Wed Sep 6 04:03:10 2017 +0000

    Add explanation about Content-Length header limit with DLO to docs

    If a number of DLO segments is larger than container listing limit,
    Content-Length header will not be included in GET or HEAD response.
    However, this fact is not explained in document of large objects.

    This patch add explanation about this fact to the document.

    Change-Id: Ia45fad05797f38fa8b6b0ed917b4f9d7fb337149
    Closes-Bug: 1680219

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/swift 2.16.0 release.