HEAD requests on DLOs break when segment container is missing
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenStack Object Storage (swift) |
Fix Released
|
Undecided
|
Unassigned |
Bug Description
I'm trying to use container-sync inside a single Liberty Swift cluster to move data from a Docker Registry to a different container in a different project. The sync itself seems to be progressing well, but I'm running into bizarre errors when trying to configure the Registry application against the new container.
The problem seems to be with DLOs. On the receiving side, the DLO shows up in the container listing, but 404 is returned when trying to GET it.
I'll grab an example. Here's what I can see in the source container:
1. GET on the container finds the object. (It's listed with zero size, which is wrong, but whatever.)
$ curl -i -H 'X-Auth-Token: <redacted>' 'https:/
HTTP/1.1 200 OK
X-Container
Content-Length: 271
X-Container
X-Storage-
Accept-Ranges: bytes
X-Container
X-Container
X-Timestamp: 1447781107.80138
Content-Type: application/json; charset=utf-8
X-Trans-Id: tx387d34644f584
Date: Mon, 15 Aug 2016 14:24:20 GMT
[{"hash": "d41d8cd98f00b2
2. When I get the DLO, the content is returned.
$ curl --head -H 'X-Auth-Token: <redacted>' https:/
HTTP/1.1 200 OK
Content-Length: 1905
Etag: "d3120bba48f047
Accept-Ranges: bytes
Last-Modified: Tue, 19 Jul 2016 11:31:31 GMT
X-Object-
X-Timestamp: 1468927890.43711
Content-Type: application/
X-Trans-Id: tx1ee45a0383e44
Date: Mon, 15 Aug 2016 14:02:35 GMT
3. And of course, since this is a DLO, I can GET on the container to list the segments (just one in this case).
$ curl -i -H 'X-Auth-Token: <redacted>' 'https:/
HTTP/1.1 200 OK
X-Container
Content-Length: 499
X-Container
X-Storage-
Accept-Ranges: bytes
X-Container
X-Container
X-Timestamp: 1447781107.80138
Content-Type: application/json; charset=utf-8
X-Trans-Id: txb1369af939e54
Date: Mon, 15 Aug 2016 14:05:31 GMT
[{"hash": "293fa5ac84ff3c
Nothing fancy until here. Let's see how things look in the target container:
1. GET on the container finds the object (again, with zero size).
$ curl -i -H 'X-Auth-Token: <redacted>' 'https:/
HTTP/1.1 200 OK
X-Container
Content-Length: 271
X-Container
Accept-Ranges: bytes
X-Storage-
X-Container
X-Container
X-Timestamp: 1469003815.67509
Content-Type: application/json; charset=utf-8
X-Trans-Id: tx85710b95e5124
Date: Mon, 15 Aug 2016 13:49:11 GMT
[{"hash": "d41d8cd98f00b2
2. Which means that I can GET the object, too. Except I can't. Wait, what?
$ curl -i -H 'X-Auth-Token: <redacted>' https:/
HTTP/1.1 404 Not Found
Content-Length: 70
Content-Type: text/html; charset=UTF-8
X-Trans-Id: tx01b439f8583b4
Date: Mon, 15 Aug 2016 13:49:13 GMT
<html><h1>Not Found</h1><p>The resource could not be found.</p></html>
3. The segment has been synced correctly, though.
$ curl -i -H 'X-Auth-Token: <redacted>' 'https:/
HTTP/1.1 200 OK
X-Container
Content-Length: 499
X-Container
Accept-Ranges: bytes
X-Storage-
X-Container
X-Container
X-Timestamp: 1469003815.67509
Content-Type: application/json; charset=utf-8
X-Trans-Id: tx348d8516cb0e4
Date: Mon, 15 Aug 2016 14:38:18 GMT
[{"hash": "293fa5ac84ff3c
description: | updated |
After further inspection, I now see what is going on. The DLO is copied correctly, but the manifest is not adjusted for the new container name. When the DLO is queried (whether with HEAD or GET), Swift tries to find the segments, but croaks because the container mentioned in the manifest does not exist.
So the actual bug here is that a DLO referring to segments in a non-existing container cannot be interacted with in a meaningful way. (At least HEAD should work IMO.)