Swift HEAD request does not response with right content length

[Tong Li]

Swift HEAD request against a segmented object will return content length of zero. The real content length should be returned.

a patch which contains a query into container object table will return the correct content length.

SELECT sum(size) as content_length
                        FROM object
                        WHERE name like '<name_prefix>/%'

this will not generate a lot of traffic and it is only a database query which will return right length for segmented objects.

[Samuel Merritt (torgomatic)]

I do not think this is a good idea. Currently the content length in the HEAD response is, in fact, the length of the manifest object.

Consider a 1024-byte object named /a/c/o. A response to "HEAD /v1/a/c/o" will tell me that the Content-Length is 1024.

If I change the value of Content-Type, then the Content-Length remains 1024.
If I change the value of X-Delete-At, then the Content-Length remains 1024.
If I change the value of X-Object-Meta-Owners-Dogs-Favorite-Kibble, then the Content-Length remains 1024.
If I change the value of X-Object-Manifest, the Content-Length changes to some other number? That seems crazy IMHO.

Updating an object's metadata doesn't affect the actual data, so headers describing the actual data, like Content-Length and Etag, shouldn't be affected by a metadata change.

[John Dickinson (notmyname)]

Except that the RFC says that the HEAD and GET must return the same values

http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.4

I'd vote for updating both the content-length and the etag if the x-object-manifest header value changes. Seems like an expensive operation, though.

[Samuel Merritt (torgomatic)]

Hm, that's true. I hadn't considered that part.

The very next sentence in the RFC says "The metainformation contained in the HTTP headers in response to a HEAD request SHOULD be identical to the information sent in response to a GET request."

So, I guess this is something that Swift SHOULD (in the RFC 2119 sense) do, but if the implementation is too costly, Swift can skip out on it. Also, in addition to Content-Length, the Etag probably ought to change when X-Object-Manifest is changed.

[Tong Li (litong01)]

Sam, If we really want to make sure we are not telling a lie to the client, as John suggested, the HEAD operation should really
tell the client the real content length. I am not sure about the etag though. For content-length, we can simply do the sql against
the db (I already have the code working for that), John and Sam, do you think that is an expensive operation? or you think create
ETag is an expensive operation? Etag is generated by join each hash of the segment, then hexdigest, this only happens if the
number of segments is less than the container_listing_limit. For objects have more segments than the listing limit, ETag is
not even given.

[Tong Li (litong01)]

To get the content length for segmented objects, we only need one extra http request from proxy server to container server.
Once container server receives that call, it runs the DB query (sum(size)) as I suggested in the bug description. That operation
should be very fast since the object table has the index on (deleted, name), so the query can be modified a little bit like this

SELECT sum(size) as content_length
                        FROM object
                        WHERE deleted = 0 and name like '<name_prefix>/%'

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/25914

[Samuel Merritt (torgomatic)]

As pointed out in the Gerrit comments, the correct thing to do here is to not send a Content-Length header at all.

RFC 2616 section 9.4 says "The HEAD method is identical to GET except that the server MUST NOT return a message-body in the response. The metainformation contained in the HTTP headers in response to a HEAD request SHOULD be identical to the information sent in response to a GET request."

Given a large object manifest /a/c/manifest with >= 10K segments, the response to "GET /v1/a/c/manifest" does not include a Content-Length header.

Therefore, the response to "HEAD /v1/a/c/manifest" should not include a Content-Length header.

[John Dickinson (notmyname)]

I cannot verify this with the current state of master (commit 0cf95b055549fd76aba38ea682ad4cc0a74de7fa). Both HEAD and GET return the proper content-length for DLO and SLO requests. Tested with a 1MB object split into 10240-byte pieces.

[Samuel Merritt (torgomatic)]

This happens when a DLO has at least 10,000 segments, or whatever the value of container_listing_limit in swift.conf is.

[Launchpad Janitor (janitor)]

[Expired for OpenStack Object Storage (swift) because there has been no activity for 60 days.]