swift delete container will orphan segments

Bug #2061007 reported by clayg
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Object Storage (swift)
Invalid
Undecided
Unassigned
python-swiftclient
New
Undecided
Unassigned

Bug Description

by default "swift delete container object" will remove the segments if object is a SLO

(nvidia) clayg@banana:~/Workspace/nvidia$ swift delete test test.big --debug
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pbss.s8k.io:443
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "GET /auth/v1.0 HTTP/1.1" 200 0
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/auth/v1.0 -X GET
DEBUG:swiftclient:RESP STATUS: 200 OK
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'text/plain; charset=UTF-8', 'X-Auth-Token': 'AUTH_tkXXX', 'X-Storage-Token': 'AUTH_tk068d458ce6a74b5e89c1a7d6b8b31b4d', 'X-Storage-Url': 'https://pbss.s8k.io/v1/AUTH_clayg', 'Content-Length': '0', 'X-Trans-Id': 'txedadc71bd53c45a5a1e37-006618259c-s8k-sjc3-c07-pxy-01', 'X-Openstack-Request-Id': 'txedadc71bd53c45a5a1e37-006618259c-s8k-sjc3-c07-pxy-01', 'Date': 'Thu, 11 Apr 2024 18:02:04 GMT'}
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pbss.s8k.io:443
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "HEAD /v1/AUTH_clayg/test/test.big?symlink=get HTTP/1.1" 200 0
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/v1/AUTH_clayg/test/test.big?symlink=get -I -H "X-Auth-Token: AUTH_tkXXX"
DEBUG:swiftclient:RESP STATUS: 200 OK
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'application/octet-stream', 'Etag': '"71e938d37c1d06dc634dd24660255a88"', 'X-Object-Meta-Mtime': '1690982695.983535', 'X-Static-Large-Object': 'True', 'Last-Modified': 'Thu, 11 Apr 2024 18:01:56 GMT', 'X-Timestamp': '1712858515.24231', 'Accept-Ranges': 'bytes', 'Content-Length': '10485760', 'X-Manifest-Etag': '65e88f63acacd29def554295241102c3', 'X-Trans-Id': 'tx523d01cb562743d4bfcf4-006618259d-s8k-sjc3-c07-pxy-01', 'X-Openstack-Request-Id': 'tx523d01cb562743d4bfcf4-006618259d-s8k-sjc3-c07-pxy-01', 'Date': 'Thu, 11 Apr 2024 18:02:05 GMT'}
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "DELETE /v1/AUTH_clayg/test/test.big?multipart-manifest=delete HTTP/1.1" 200 None
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/v1/AUTH_clayg/test/test.big?multipart-manifest=delete DELETE -H "X-Auth-Token: AUTH_tkXXX"
DEBUG:swiftclient:RESP STATUS: 200 OK
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'text/plain', 'X-Trans-Id': 'tx542c45aaf89b462e93fd7-006618259d-s8k-sjc3-c07-pxy-01', 'X-Openstack-Request-Id': 'tx542c45aaf89b462e93fd7-006618259d-s8k-sjc3-c07-pxy-01', 'Date': 'Thu, 11 Apr 2024 18:02:07 GMT', 'Transfer-Encoding': 'chunked'}
DEBUG:swiftclient:RESP BODY: b'Number Deleted: 12\nNumber Not Found: 0\nResponse Body: \nResponse Status: 200 OK\nErrors:\n'
test.big

but "swift delete container" will orphan the segments - when I delete a container with >10K objects I noticed the segments were left behind and confirmed via logs all the DELETE requests were made with w/o any query params.

Given that behavior I find this help output misleading:

(nvidia) clayg@banana:~/Workspace/nvidia$ swift delete test --help
Usage: swift delete [--all] [--leave-segments]
                    [--object-threads <threads>]
                    [--container-threads <threads>]
                    [--header <header:value>]
                    [--prefix <prefix>]
                    [--versions]
                    [<container> [<object>] [--version-id <version_id>] [...]]

Delete a container or objects within a container.

Positional arguments:
  [<container>] Name of container to delete from.
  [<object>] Name of object to delete. Specify multiple times
                        for multiple objects.

Optional arguments:
  -a, --all Delete all containers and objects. Implies --versions.
  --versions Delete all versions.
  --leave-segments Do not delete segments of manifest objects.
  -H, --header <header:value>
                        Adds a custom request header to use for deleting
                        objects or an entire container .
  --object-threads <threads>
                        Number of threads to use for deleting objects.
                        Default is 10.
  --container-threads <threads>
                        Number of threads to use for deleting containers.
                        Default is 10.
  --prefix <prefix> Only delete objects beginning with <prefix>.
  --version-id <version-id>
                        Delete specific version of a versioned object.

I think we should fix the help output or change the behavior. Tested on python-swiftclient=4.5.0

N.B. in my simple dev test with only one object in the container it seems to work as you expect. Upon closer inspection I noticed my DELETE logs had source=BD

clayg (clay-gerrard)
description: updated
Changed in swift:
status: New → Invalid
clayg (clay-gerrard)
description: updated
description: updated
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to python-swiftclient (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/python-swiftclient/+/915507

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.