python-swiftclient

Bug #2061007
Comment #0

Comment 0 for bug 2061007

Revision history for this message

clayg (clay-gerrard) wrote on 2024-04-11:

by default "swift delete container object" will remove the segments if object is a SLO

(nvidia) clayg@banana:~/Workspace/nvidia$ swift delete test test.big --debug
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pbss.s8k.io:443
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "GET /auth/v1.0 HTTP/1.1" 200 0
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/auth/v1.0 -X GET
DEBUG:swiftclient:RESP STATUS: 200 OK
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'text/plain; charset=UTF-8', 'X-Auth-Token': 'AUTH_tk068d458ce6a74b5e89c1a7d6b8b31b4d', 'X-Storage-Token': 'AUTH_tk068d458ce6a74b5e89c1a7d6b8b31b4d', 'X-Storage-Url': 'https://pbss.s8k.io/v1/AUTH_clayg', 'Content-Length': '0', 'X-Trans-Id': 'txedadc71bd53c45a5a1e37-006618259c-s8k-sjc3-c07-pxy-01', 'X-Openstack-Request-Id': 'txedadc71bd53c45a5a1e37-006618259c-s8k-sjc3-c07-pxy-01', 'Date': 'Thu, 11 Apr 2024 18:02:04 GMT'}
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pbss.s8k.io:443
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "HEAD /v1/AUTH_clayg/test/test.big?symlink=get HTTP/1.1" 200 0
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/v1/AUTH_clayg/test/test.big?symlink=get -I -H "X-Auth-Token: AUTH_tk068d458ce6a74b5e89c1a7d6b8b31b4d"
DEBUG:swiftclient:RESP STATUS: 200 OK
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'application/octet-stream', 'Etag': '"71e938d37c1d06dc634dd24660255a88"', 'X-Object-Meta-Mtime': '1690982695.983535', 'X-Static-Large-Object': 'True', 'Last-Modified': 'Thu, 11 Apr 2024 18:01:56 GMT', 'X-Timestamp': '1712858515.24231', 'Accept-Ranges': 'bytes', 'Content-Length': '10485760', 'X-Manifest-Etag': '65e88f63acacd29def554295241102c3', 'X-Trans-Id': 'tx523d01cb562743d4bfcf4-006618259d-s8k-sjc3-c07-pxy-01', 'X-Openstack-Request-Id': 'tx523d01cb562743d4bfcf4-006618259d-s8k-sjc3-c07-pxy-01', 'Date': 'Thu, 11 Apr 2024 18:02:05 GMT'}
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "DELETE /v1/AUTH_clayg/test/test.big?multipart-manifest=delete HTTP/1.1" 200 None
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/v1/AUTH_clayg/test/test.big?multipart-manifest=delete DELETE -H "X-Auth-Token: AUTH_tk068d458ce6a74b5e89c1a7d6b8b31b4d"
DEBUG:swiftclient:RESP STATUS: 200 OK
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'text/plain', 'X-Trans-Id': 'tx542c45aaf89b462e93fd7-006618259d-s8k-sjc3-c07-pxy-01', 'X-Openstack-Request-Id': 'tx542c45aaf89b462e93fd7-006618259d-s8k-sjc3-c07-pxy-01', 'Date': 'Thu, 11 Apr 2024 18:02:07 GMT', 'Transfer-Encoding': 'chunked'}
DEBUG:swiftclient:RESP BODY: b'Number Deleted: 12\nNumber Not Found: 0\nResponse Body: \nResponse Status: 200 OK\nErrors:\n'
test.big

but "swift delete container" will orphan the segments

(nvidia) clayg@banana:~/Workspace/nvidia$ swift delete test --debug
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pbss.s8k.io:443
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "GET /auth/v1.0 HTTP/1.1" 200 0
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/auth/v1.0 -X GET
DEBUG:swiftclient:RESP STATUS: 200 OK
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'text/plain; charset=UTF-8', 'X-Auth-Token': 'AUTH_tk068d458ce6a74b5e89c1a7d6b8b31b4d', 'X-Storage-Token': 'AUTH_tk068d458ce6a74b5e89c1a7d6b8b31b4d', 'X-Storage-Url': 'https://pbss.s8k.io/v1/AUTH_clayg', 'Content-Length': '0', 'X-Trans-Id': 'tx71ab7f39c24b45c08ac4a-00661825c0-s8k-sjc11-d06-pxy-02', 'X-Openstack-Request-Id': 'tx71ab7f39c24b45c08ac4a-00661825c0-s8k-sjc11-d06-pxy-02', 'Date': 'Thu, 11 Apr 2024 18:02:41 GMT'}
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pbss.s8k.io:443
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "GET /v1/AUTH_clayg/test?format=json HTTP/1.1" 200 2
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/v1/AUTH_clayg/test?format=json -X GET -H "X-Auth-Token: AUTH_tk068d458ce6a74b5e89c1a7d6b8b31b4d" -H "Accept-Encoding: gzip"
DEBUG:swiftclient:RESP STATUS: 200 OK
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'application/json; charset=utf-8', 'Content-Length': '2', 'X-Container-Object-Count': '0', 'X-Container-Bytes-Used': '0', 'X-Timestamp': '1712858506.78033', 'Last-Modified': 'Thu, 11 Apr 2024 18:01:47 GMT', 'Accept-Ranges': 'bytes', 'X-Storage-Policy': 'EC-84-Policy', 'X-Container-Sharding': 'False', 'X-Trans-Id': 'tx196c62ddbe3e471cb29cd-00661825c1-s8k-sjc11-c02-pxy-02', 'X-Openstack-Request-Id': 'tx196c62ddbe3e471cb29cd-00661825c1-s8k-sjc11-c02-pxy-02', 'Date': 'Thu, 11 Apr 2024 18:02:41 GMT'}
DEBUG:swiftclient:RESP BODY: b'[]'
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "DELETE /v1/AUTH_clayg/test HTTP/1.1" 204 0
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/v1/AUTH_clayg/test DELETE -H "X-Auth-Token: AUTH_tk068d458ce6a74b5e89c1a7d6b8b31b4d"
DEBUG:swiftclient:RESP STATUS: 204 No Content
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'text/html; charset=UTF-8', 'Content-Length': '0', 'X-Trans-Id': 'tx94af6ed4f2144954be20a-00661825c1-s8k-sjc11-c02-pxy-02', 'X-Openstack-Request-Id': 'tx94af6ed4f2144954be20a-00661825c1-s8k-sjc11-c02-pxy-02', 'Date': 'Thu, 11 Apr 2024 18:02:41 GMT'}
test

Given that behavior I find this help output misleading:

(nvidia) clayg@banana:~/Workspace/nvidia$ swift delete test --help
Usage: swift delete [--all] [--leave-segments]
                    [--object-threads <threads>]
                    [--container-threads <threads>]
                    [--header <header:value>]
                    [--prefix <prefix>]
                    [--versions]
                    [<container> [<object>] [--version-id <version_id>] [...]]

Delete a container or objects within a container.

Positional arguments:
  [<container>] Name of container to delete from.
  [<object>] Name of object to delete. Specify multiple times
                        for multiple objects.

Optional arguments:
  -a, --all Delete all containers and objects. Implies --versions.
  --versions Delete all versions.
  --leave-segments Do not delete segments of manifest objects.
  -H, --header <header:value>
                        Adds a custom request header to use for deleting
                        objects or an entire container .
  --object-threads <threads>
                        Number of threads to use for deleting objects.
                        Default is 10.
  --container-threads <threads>
                        Number of threads to use for deleting containers.
                        Default is 10.
  --prefix <prefix> Only delete objects beginning with <prefix>.
  --version-id <version-id>
                        Delete specific version of a versioned object.

I think we fix the help output or change the behavior. Tested on python-swiftclient=4.5.0

by default "swift delete container object" will remove the segments if object is a SLO

but "swift delete container" will orphan the segments

Given that behavior I find this help output misleading:

Delete a container or objects within a container.

Positional arguments:
  [<container>]         Name of container to delete from.
  [<object>]            Name of object to delete. Specify multiple times
                        for multiple objects.

Optional arguments:
  -a, --all             Delete all containers and objects. Implies --versions.
  --versions            Delete all versions.
  --leave-segments      Do not delete segments of manifest objects.
  -H, --header <header:value>
                        Adds a custom request header to use for deleting
                        objects or an entire container .
  --object-threads <threads>
                        Number of threads to use for deleting objects.
                        Default is 10.
  --container-threads <threads>
                        Number of threads to use for deleting containers.
                        Default is 10.
  --prefix <prefix>     Only delete objects beginning with <prefix>.
  --version-id <version-id>
                        Delete specific version of a versioned object.

I think we fix the help output or change the behavior.  Tested on python-swiftclient=4.5.0