| 2024-04-11 18:43:08 |
clayg |
bug |
|
|
added bug |
| 2024-04-11 18:44:52 |
clayg |
description |
by default "swift delete container object" will remove the segments if object is a SLO
(nvidia) clayg@banana:~/Workspace/nvidia$ swift delete test test.big --debug
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pbss.s8k.io:443
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "GET /auth/v1.0 HTTP/1.1" 200 0
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/auth/v1.0 -X GET
DEBUG:swiftclient:RESP STATUS: 200 OK
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'text/plain; charset=UTF-8', 'X-Auth-Token': 'AUTH_tk068d458ce6a74b5e89c1a7d6b8b31b4d', 'X-Storage-Token': 'AUTH_tk068d458ce6a74b5e89c1a7d6b8b31b4d', 'X-Storage-Url': 'https://pbss.s8k.io/v1/AUTH_clayg', 'Content-Length': '0', 'X-Trans-Id': 'txedadc71bd53c45a5a1e37-006618259c-s8k-sjc3-c07-pxy-01', 'X-Openstack-Request-Id': 'txedadc71bd53c45a5a1e37-006618259c-s8k-sjc3-c07-pxy-01', 'Date': 'Thu, 11 Apr 2024 18:02:04 GMT'}
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pbss.s8k.io:443
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "HEAD /v1/AUTH_clayg/test/test.big?symlink=get HTTP/1.1" 200 0
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/v1/AUTH_clayg/test/test.big?symlink=get -I -H "X-Auth-Token: AUTH_tk068d458ce6a74b5e89c1a7d6b8b31b4d"
DEBUG:swiftclient:RESP STATUS: 200 OK
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'application/octet-stream', 'Etag': '"71e938d37c1d06dc634dd24660255a88"', 'X-Object-Meta-Mtime': '1690982695.983535', 'X-Static-Large-Object': 'True', 'Last-Modified': 'Thu, 11 Apr 2024 18:01:56 GMT', 'X-Timestamp': '1712858515.24231', 'Accept-Ranges': 'bytes', 'Content-Length': '10485760', 'X-Manifest-Etag': '65e88f63acacd29def554295241102c3', 'X-Trans-Id': 'tx523d01cb562743d4bfcf4-006618259d-s8k-sjc3-c07-pxy-01', 'X-Openstack-Request-Id': 'tx523d01cb562743d4bfcf4-006618259d-s8k-sjc3-c07-pxy-01', 'Date': 'Thu, 11 Apr 2024 18:02:05 GMT'}
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "DELETE /v1/AUTH_clayg/test/test.big?multipart-manifest=delete HTTP/1.1" 200 None
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/v1/AUTH_clayg/test/test.big?multipart-manifest=delete DELETE -H "X-Auth-Token: AUTH_tk068d458ce6a74b5e89c1a7d6b8b31b4d"
DEBUG:swiftclient:RESP STATUS: 200 OK
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'text/plain', 'X-Trans-Id': 'tx542c45aaf89b462e93fd7-006618259d-s8k-sjc3-c07-pxy-01', 'X-Openstack-Request-Id': 'tx542c45aaf89b462e93fd7-006618259d-s8k-sjc3-c07-pxy-01', 'Date': 'Thu, 11 Apr 2024 18:02:07 GMT', 'Transfer-Encoding': 'chunked'}
DEBUG:swiftclient:RESP BODY: b'Number Deleted: 12\nNumber Not Found: 0\nResponse Body: \nResponse Status: 200 OK\nErrors:\n'
test.big
but "swift delete container" will orphan the segments
(nvidia) clayg@banana:~/Workspace/nvidia$ swift delete test --debug
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pbss.s8k.io:443
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "GET /auth/v1.0 HTTP/1.1" 200 0
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/auth/v1.0 -X GET
DEBUG:swiftclient:RESP STATUS: 200 OK
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'text/plain; charset=UTF-8', 'X-Auth-Token': 'AUTH_tk068d458ce6a74b5e89c1a7d6b8b31b4d', 'X-Storage-Token': 'AUTH_tk068d458ce6a74b5e89c1a7d6b8b31b4d', 'X-Storage-Url': 'https://pbss.s8k.io/v1/AUTH_clayg', 'Content-Length': '0', 'X-Trans-Id': 'tx71ab7f39c24b45c08ac4a-00661825c0-s8k-sjc11-d06-pxy-02', 'X-Openstack-Request-Id': 'tx71ab7f39c24b45c08ac4a-00661825c0-s8k-sjc11-d06-pxy-02', 'Date': 'Thu, 11 Apr 2024 18:02:41 GMT'}
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pbss.s8k.io:443
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "GET /v1/AUTH_clayg/test?format=json HTTP/1.1" 200 2
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/v1/AUTH_clayg/test?format=json -X GET -H "X-Auth-Token: AUTH_tk068d458ce6a74b5e89c1a7d6b8b31b4d" -H "Accept-Encoding: gzip"
DEBUG:swiftclient:RESP STATUS: 200 OK
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'application/json; charset=utf-8', 'Content-Length': '2', 'X-Container-Object-Count': '0', 'X-Container-Bytes-Used': '0', 'X-Timestamp': '1712858506.78033', 'Last-Modified': 'Thu, 11 Apr 2024 18:01:47 GMT', 'Accept-Ranges': 'bytes', 'X-Storage-Policy': 'EC-84-Policy', 'X-Container-Sharding': 'False', 'X-Trans-Id': 'tx196c62ddbe3e471cb29cd-00661825c1-s8k-sjc11-c02-pxy-02', 'X-Openstack-Request-Id': 'tx196c62ddbe3e471cb29cd-00661825c1-s8k-sjc11-c02-pxy-02', 'Date': 'Thu, 11 Apr 2024 18:02:41 GMT'}
DEBUG:swiftclient:RESP BODY: b'[]'
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "DELETE /v1/AUTH_clayg/test HTTP/1.1" 204 0
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/v1/AUTH_clayg/test DELETE -H "X-Auth-Token: AUTH_tk068d458ce6a74b5e89c1a7d6b8b31b4d"
DEBUG:swiftclient:RESP STATUS: 204 No Content
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'text/html; charset=UTF-8', 'Content-Length': '0', 'X-Trans-Id': 'tx94af6ed4f2144954be20a-00661825c1-s8k-sjc11-c02-pxy-02', 'X-Openstack-Request-Id': 'tx94af6ed4f2144954be20a-00661825c1-s8k-sjc11-c02-pxy-02', 'Date': 'Thu, 11 Apr 2024 18:02:41 GMT'}
test
Given that behavior I find this help output misleading:
(nvidia) clayg@banana:~/Workspace/nvidia$ swift delete test --help
Usage: swift delete [--all] [--leave-segments]
[--object-threads <threads>]
[--container-threads <threads>]
[--header <header:value>]
[--prefix <prefix>]
[--versions]
[<container> [<object>] [--version-id <version_id>] [...]]
Delete a container or objects within a container.
Positional arguments:
[<container>] Name of container to delete from.
[<object>] Name of object to delete. Specify multiple times
for multiple objects.
Optional arguments:
-a, --all Delete all containers and objects. Implies --versions.
--versions Delete all versions.
--leave-segments Do not delete segments of manifest objects.
-H, --header <header:value>
Adds a custom request header to use for deleting
objects or an entire container .
--object-threads <threads>
Number of threads to use for deleting objects.
Default is 10.
--container-threads <threads>
Number of threads to use for deleting containers.
Default is 10.
--prefix <prefix> Only delete objects beginning with <prefix>.
--version-id <version-id>
Delete specific version of a versioned object.
I think we fix the help output or change the behavior. Tested on python-swiftclient=4.5.0 |
by default "swift delete container object" will remove the segments if object is a SLO
(nvidia) clayg@banana:~/Workspace/nvidia$ swift delete test test.big --debug
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pbss.s8k.io:443
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "GET /auth/v1.0 HTTP/1.1" 200 0
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/auth/v1.0 -X GET
DEBUG:swiftclient:RESP STATUS: 200 OK
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'text/plain; charset=UTF-8', 'X-Auth-Token': 'AUTH_tkXXX', 'X-Storage-Token': 'AUTH_tk068d458ce6a74b5e89c1a7d6b8b31b4d', 'X-Storage-Url': 'https://pbss.s8k.io/v1/AUTH_clayg', 'Content-Length': '0', 'X-Trans-Id': 'txedadc71bd53c45a5a1e37-006618259c-s8k-sjc3-c07-pxy-01', 'X-Openstack-Request-Id': 'txedadc71bd53c45a5a1e37-006618259c-s8k-sjc3-c07-pxy-01', 'Date': 'Thu, 11 Apr 2024 18:02:04 GMT'}
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pbss.s8k.io:443
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "HEAD /v1/AUTH_clayg/test/test.big?symlink=get HTTP/1.1" 200 0
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/v1/AUTH_clayg/test/test.big?symlink=get -I -H "X-Auth-Token: AUTH_tkXXX"
DEBUG:swiftclient:RESP STATUS: 200 OK
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'application/octet-stream', 'Etag': '"71e938d37c1d06dc634dd24660255a88"', 'X-Object-Meta-Mtime': '1690982695.983535', 'X-Static-Large-Object': 'True', 'Last-Modified': 'Thu, 11 Apr 2024 18:01:56 GMT', 'X-Timestamp': '1712858515.24231', 'Accept-Ranges': 'bytes', 'Content-Length': '10485760', 'X-Manifest-Etag': '65e88f63acacd29def554295241102c3', 'X-Trans-Id': 'tx523d01cb562743d4bfcf4-006618259d-s8k-sjc3-c07-pxy-01', 'X-Openstack-Request-Id': 'tx523d01cb562743d4bfcf4-006618259d-s8k-sjc3-c07-pxy-01', 'Date': 'Thu, 11 Apr 2024 18:02:05 GMT'}
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "DELETE /v1/AUTH_clayg/test/test.big?multipart-manifest=delete HTTP/1.1" 200 None
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/v1/AUTH_clayg/test/test.big?multipart-manifest=delete DELETE -H "X-Auth-Token: AUTH_tkXXX"
DEBUG:swiftclient:RESP STATUS: 200 OK
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'text/plain', 'X-Trans-Id': 'tx542c45aaf89b462e93fd7-006618259d-s8k-sjc3-c07-pxy-01', 'X-Openstack-Request-Id': 'tx542c45aaf89b462e93fd7-006618259d-s8k-sjc3-c07-pxy-01', 'Date': 'Thu, 11 Apr 2024 18:02:07 GMT', 'Transfer-Encoding': 'chunked'}
DEBUG:swiftclient:RESP BODY: b'Number Deleted: 12\nNumber Not Found: 0\nResponse Body: \nResponse Status: 200 OK\nErrors:\n'
test.big
but "swift delete container" will orphan the segments
(nvidia) clayg@banana:~/Workspace/nvidia$ swift delete test --debug
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pbss.s8k.io:443
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "GET /auth/v1.0 HTTP/1.1" 200 0
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/auth/v1.0 -X GET
DEBUG:swiftclient:RESP STATUS: 200 OK
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'text/plain; charset=UTF-8', 'X-Auth-Token': 'AUTH_tkXXX', 'X-Storage-Token': 'AUTH_tk068d458ce6a74b5e89c1a7d6b8b31b4d', 'X-Storage-Url': 'https://pbss.s8k.io/v1/AUTH_clayg', 'Content-Length': '0', 'X-Trans-Id': 'tx71ab7f39c24b45c08ac4a-00661825c0-s8k-sjc11-d06-pxy-02', 'X-Openstack-Request-Id': 'tx71ab7f39c24b45c08ac4a-00661825c0-s8k-sjc11-d06-pxy-02', 'Date': 'Thu, 11 Apr 2024 18:02:41 GMT'}
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pbss.s8k.io:443
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "GET /v1/AUTH_clayg/test?format=json HTTP/1.1" 200 2
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/v1/AUTH_clayg/test?format=json -X GET -H "X-Auth-Token: AUTH_tkXXX" -H "Accept-Encoding: gzip"
DEBUG:swiftclient:RESP STATUS: 200 OK
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'application/json; charset=utf-8', 'Content-Length': '2', 'X-Container-Object-Count': '0', 'X-Container-Bytes-Used': '0', 'X-Timestamp': '1712858506.78033', 'Last-Modified': 'Thu, 11 Apr 2024 18:01:47 GMT', 'Accept-Ranges': 'bytes', 'X-Storage-Policy': 'EC-84-Policy', 'X-Container-Sharding': 'False', 'X-Trans-Id': 'tx196c62ddbe3e471cb29cd-00661825c1-s8k-sjc11-c02-pxy-02', 'X-Openstack-Request-Id': 'tx196c62ddbe3e471cb29cd-00661825c1-s8k-sjc11-c02-pxy-02', 'Date': 'Thu, 11 Apr 2024 18:02:41 GMT'}
DEBUG:swiftclient:RESP BODY: b'[]'
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "DELETE /v1/AUTH_clayg/test HTTP/1.1" 204 0
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/v1/AUTH_clayg/test DELETE -H "X-Auth-Token: AUTH_tkXXX"
DEBUG:swiftclient:RESP STATUS: 204 No Content
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'text/html; charset=UTF-8', 'Content-Length': '0', 'X-Trans-Id': 'tx94af6ed4f2144954be20a-00661825c1-s8k-sjc11-c02-pxy-02', 'X-Openstack-Request-Id': 'tx94af6ed4f2144954be20a-00661825c1-s8k-sjc11-c02-pxy-02', 'Date': 'Thu, 11 Apr 2024 18:02:41 GMT'}
test
Given that behavior I find this help output misleading:
(nvidia) clayg@banana:~/Workspace/nvidia$ swift delete test --help
Usage: swift delete [--all] [--leave-segments]
[--object-threads <threads>]
[--container-threads <threads>]
[--header <header:value>]
[--prefix <prefix>]
[--versions]
[<container> [<object>] [--version-id <version_id>] [...]]
Delete a container or objects within a container.
Positional arguments:
[<container>] Name of container to delete from.
[<object>] Name of object to delete. Specify multiple times
for multiple objects.
Optional arguments:
-a, --all Delete all containers and objects. Implies --versions.
--versions Delete all versions.
--leave-segments Do not delete segments of manifest objects.
-H, --header <header:value>
Adds a custom request header to use for deleting
objects or an entire container .
--object-threads <threads>
Number of threads to use for deleting objects.
Default is 10.
--container-threads <threads>
Number of threads to use for deleting containers.
Default is 10.
--prefix <prefix> Only delete objects beginning with <prefix>.
--version-id <version-id>
Delete specific version of a versioned object.
I think we fix the help output or change the behavior. Tested on python-swiftclient=4.5.0 |
|
| 2024-04-11 18:45:53 |
clayg |
bug task added |
|
python-swiftclient |
|
| 2024-04-11 18:46:09 |
clayg |
swift: status |
New |
Invalid |
|
| 2024-04-11 20:40:43 |
clayg |
description |
by default "swift delete container object" will remove the segments if object is a SLO
(nvidia) clayg@banana:~/Workspace/nvidia$ swift delete test test.big --debug
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pbss.s8k.io:443
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "GET /auth/v1.0 HTTP/1.1" 200 0
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/auth/v1.0 -X GET
DEBUG:swiftclient:RESP STATUS: 200 OK
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'text/plain; charset=UTF-8', 'X-Auth-Token': 'AUTH_tkXXX', 'X-Storage-Token': 'AUTH_tk068d458ce6a74b5e89c1a7d6b8b31b4d', 'X-Storage-Url': 'https://pbss.s8k.io/v1/AUTH_clayg', 'Content-Length': '0', 'X-Trans-Id': 'txedadc71bd53c45a5a1e37-006618259c-s8k-sjc3-c07-pxy-01', 'X-Openstack-Request-Id': 'txedadc71bd53c45a5a1e37-006618259c-s8k-sjc3-c07-pxy-01', 'Date': 'Thu, 11 Apr 2024 18:02:04 GMT'}
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pbss.s8k.io:443
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "HEAD /v1/AUTH_clayg/test/test.big?symlink=get HTTP/1.1" 200 0
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/v1/AUTH_clayg/test/test.big?symlink=get -I -H "X-Auth-Token: AUTH_tkXXX"
DEBUG:swiftclient:RESP STATUS: 200 OK
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'application/octet-stream', 'Etag': '"71e938d37c1d06dc634dd24660255a88"', 'X-Object-Meta-Mtime': '1690982695.983535', 'X-Static-Large-Object': 'True', 'Last-Modified': 'Thu, 11 Apr 2024 18:01:56 GMT', 'X-Timestamp': '1712858515.24231', 'Accept-Ranges': 'bytes', 'Content-Length': '10485760', 'X-Manifest-Etag': '65e88f63acacd29def554295241102c3', 'X-Trans-Id': 'tx523d01cb562743d4bfcf4-006618259d-s8k-sjc3-c07-pxy-01', 'X-Openstack-Request-Id': 'tx523d01cb562743d4bfcf4-006618259d-s8k-sjc3-c07-pxy-01', 'Date': 'Thu, 11 Apr 2024 18:02:05 GMT'}
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "DELETE /v1/AUTH_clayg/test/test.big?multipart-manifest=delete HTTP/1.1" 200 None
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/v1/AUTH_clayg/test/test.big?multipart-manifest=delete DELETE -H "X-Auth-Token: AUTH_tkXXX"
DEBUG:swiftclient:RESP STATUS: 200 OK
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'text/plain', 'X-Trans-Id': 'tx542c45aaf89b462e93fd7-006618259d-s8k-sjc3-c07-pxy-01', 'X-Openstack-Request-Id': 'tx542c45aaf89b462e93fd7-006618259d-s8k-sjc3-c07-pxy-01', 'Date': 'Thu, 11 Apr 2024 18:02:07 GMT', 'Transfer-Encoding': 'chunked'}
DEBUG:swiftclient:RESP BODY: b'Number Deleted: 12\nNumber Not Found: 0\nResponse Body: \nResponse Status: 200 OK\nErrors:\n'
test.big
but "swift delete container" will orphan the segments
(nvidia) clayg@banana:~/Workspace/nvidia$ swift delete test --debug
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pbss.s8k.io:443
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "GET /auth/v1.0 HTTP/1.1" 200 0
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/auth/v1.0 -X GET
DEBUG:swiftclient:RESP STATUS: 200 OK
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'text/plain; charset=UTF-8', 'X-Auth-Token': 'AUTH_tkXXX', 'X-Storage-Token': 'AUTH_tk068d458ce6a74b5e89c1a7d6b8b31b4d', 'X-Storage-Url': 'https://pbss.s8k.io/v1/AUTH_clayg', 'Content-Length': '0', 'X-Trans-Id': 'tx71ab7f39c24b45c08ac4a-00661825c0-s8k-sjc11-d06-pxy-02', 'X-Openstack-Request-Id': 'tx71ab7f39c24b45c08ac4a-00661825c0-s8k-sjc11-d06-pxy-02', 'Date': 'Thu, 11 Apr 2024 18:02:41 GMT'}
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pbss.s8k.io:443
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "GET /v1/AUTH_clayg/test?format=json HTTP/1.1" 200 2
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/v1/AUTH_clayg/test?format=json -X GET -H "X-Auth-Token: AUTH_tkXXX" -H "Accept-Encoding: gzip"
DEBUG:swiftclient:RESP STATUS: 200 OK
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'application/json; charset=utf-8', 'Content-Length': '2', 'X-Container-Object-Count': '0', 'X-Container-Bytes-Used': '0', 'X-Timestamp': '1712858506.78033', 'Last-Modified': 'Thu, 11 Apr 2024 18:01:47 GMT', 'Accept-Ranges': 'bytes', 'X-Storage-Policy': 'EC-84-Policy', 'X-Container-Sharding': 'False', 'X-Trans-Id': 'tx196c62ddbe3e471cb29cd-00661825c1-s8k-sjc11-c02-pxy-02', 'X-Openstack-Request-Id': 'tx196c62ddbe3e471cb29cd-00661825c1-s8k-sjc11-c02-pxy-02', 'Date': 'Thu, 11 Apr 2024 18:02:41 GMT'}
DEBUG:swiftclient:RESP BODY: b'[]'
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "DELETE /v1/AUTH_clayg/test HTTP/1.1" 204 0
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/v1/AUTH_clayg/test DELETE -H "X-Auth-Token: AUTH_tkXXX"
DEBUG:swiftclient:RESP STATUS: 204 No Content
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'text/html; charset=UTF-8', 'Content-Length': '0', 'X-Trans-Id': 'tx94af6ed4f2144954be20a-00661825c1-s8k-sjc11-c02-pxy-02', 'X-Openstack-Request-Id': 'tx94af6ed4f2144954be20a-00661825c1-s8k-sjc11-c02-pxy-02', 'Date': 'Thu, 11 Apr 2024 18:02:41 GMT'}
test
Given that behavior I find this help output misleading:
(nvidia) clayg@banana:~/Workspace/nvidia$ swift delete test --help
Usage: swift delete [--all] [--leave-segments]
[--object-threads <threads>]
[--container-threads <threads>]
[--header <header:value>]
[--prefix <prefix>]
[--versions]
[<container> [<object>] [--version-id <version_id>] [...]]
Delete a container or objects within a container.
Positional arguments:
[<container>] Name of container to delete from.
[<object>] Name of object to delete. Specify multiple times
for multiple objects.
Optional arguments:
-a, --all Delete all containers and objects. Implies --versions.
--versions Delete all versions.
--leave-segments Do not delete segments of manifest objects.
-H, --header <header:value>
Adds a custom request header to use for deleting
objects or an entire container .
--object-threads <threads>
Number of threads to use for deleting objects.
Default is 10.
--container-threads <threads>
Number of threads to use for deleting containers.
Default is 10.
--prefix <prefix> Only delete objects beginning with <prefix>.
--version-id <version-id>
Delete specific version of a versioned object.
I think we fix the help output or change the behavior. Tested on python-swiftclient=4.5.0 |
by default "swift delete container object" will remove the segments if object is a SLO
(nvidia) clayg@banana:~/Workspace/nvidia$ swift delete test test.big --debug
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pbss.s8k.io:443
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "GET /auth/v1.0 HTTP/1.1" 200 0
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/auth/v1.0 -X GET
DEBUG:swiftclient:RESP STATUS: 200 OK
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'text/plain; charset=UTF-8', 'X-Auth-Token': 'AUTH_tkXXX', 'X-Storage-Token': 'AUTH_tk068d458ce6a74b5e89c1a7d6b8b31b4d', 'X-Storage-Url': 'https://pbss.s8k.io/v1/AUTH_clayg', 'Content-Length': '0', 'X-Trans-Id': 'txedadc71bd53c45a5a1e37-006618259c-s8k-sjc3-c07-pxy-01', 'X-Openstack-Request-Id': 'txedadc71bd53c45a5a1e37-006618259c-s8k-sjc3-c07-pxy-01', 'Date': 'Thu, 11 Apr 2024 18:02:04 GMT'}
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pbss.s8k.io:443
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "HEAD /v1/AUTH_clayg/test/test.big?symlink=get HTTP/1.1" 200 0
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/v1/AUTH_clayg/test/test.big?symlink=get -I -H "X-Auth-Token: AUTH_tkXXX"
DEBUG:swiftclient:RESP STATUS: 200 OK
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'application/octet-stream', 'Etag': '"71e938d37c1d06dc634dd24660255a88"', 'X-Object-Meta-Mtime': '1690982695.983535', 'X-Static-Large-Object': 'True', 'Last-Modified': 'Thu, 11 Apr 2024 18:01:56 GMT', 'X-Timestamp': '1712858515.24231', 'Accept-Ranges': 'bytes', 'Content-Length': '10485760', 'X-Manifest-Etag': '65e88f63acacd29def554295241102c3', 'X-Trans-Id': 'tx523d01cb562743d4bfcf4-006618259d-s8k-sjc3-c07-pxy-01', 'X-Openstack-Request-Id': 'tx523d01cb562743d4bfcf4-006618259d-s8k-sjc3-c07-pxy-01', 'Date': 'Thu, 11 Apr 2024 18:02:05 GMT'}
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "DELETE /v1/AUTH_clayg/test/test.big?multipart-manifest=delete HTTP/1.1" 200 None
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/v1/AUTH_clayg/test/test.big?multipart-manifest=delete DELETE -H "X-Auth-Token: AUTH_tkXXX"
DEBUG:swiftclient:RESP STATUS: 200 OK
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'text/plain', 'X-Trans-Id': 'tx542c45aaf89b462e93fd7-006618259d-s8k-sjc3-c07-pxy-01', 'X-Openstack-Request-Id': 'tx542c45aaf89b462e93fd7-006618259d-s8k-sjc3-c07-pxy-01', 'Date': 'Thu, 11 Apr 2024 18:02:07 GMT', 'Transfer-Encoding': 'chunked'}
DEBUG:swiftclient:RESP BODY: b'Number Deleted: 12\nNumber Not Found: 0\nResponse Body: \nResponse Status: 200 OK\nErrors:\n'
test.big
but "swift delete container" will orphan the segments
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pbss.s8k.io:443
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "GET /auth/v1.0 HTTP/1.1" 200 0
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/auth/v1.0 -X GET
DEBUG:swiftclient:RESP STATUS: 200 OK
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'text/plain; charset=UTF-8', 'X-Auth-Token': 'AUTH_tkXXX', 'X-Storage-Token': 'AUTH_tk48bef83e838c4aa8a06cd9c92465d7a2', 'X-Storage-Url': 'https://pbss.s8k.io/v1/AUTH_clayg', 'Content-Length': '0', 'X-Trans-Id': 'tx33367a83996a4341b93e0-0066184a58-s8k-sjc3-d06-obj-01', 'X-Openstack-Request-Id': 'tx33367a83996a4341b93e0-0066184a58-s8k-sjc3-d06-obj-01', 'Date': 'Thu, 11 Apr 2024 20:38:48 GMT'}
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pbss.s8k.io:443
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "GET /v1/AUTH_clayg/new-est?format=json HTTP/1.1" 200 228
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/v1/AUTH_clayg/new-est?format=json -X GET -H "X-Auth-Token: AUTH_tkXXX" -H "Accept-Encoding: gzip"
DEBUG:swiftclient:RESP STATUS: 200 OK
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'application/json; charset=utf-8', 'Content-Length': '228', 'X-Container-Object-Count': '1', 'X-Container-Bytes-Used': '2727', 'X-Timestamp': '1712858546.35960', 'Last-Modified': 'Thu, 11 Apr 2024 18:02:27 GMT', 'Accept-Ranges': 'bytes', 'X-Storage-Policy': 'EC-84-Policy', 'X-Container-Sharding': 'False', 'X-Trans-Id': 'txac9df35f178d4bc68d83a-0066184a58-s8k-sjc11-d06-pxy-01', 'X-Openstack-Request-Id': 'txac9df35f178d4bc68d83a-0066184a58-s8k-sjc11-d06-pxy-01', 'Date': 'Thu, 11 Apr 2024 20:38:48 GMT'}
DEBUG:swiftclient:RESP BODY: b'[{"bytes": 10485760, "hash": "8261e54c8897faf876339362627c1ac5", "name": "test.big", "content_type": "application/octet-stream", "last_modified": "2024-04-11T18:02:34.307400", "slo_etag": "\\"71e938d37c1d06dc634dd24660255a88\\""}]'
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pbss.s8k.io:443
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "GET /v1/AUTH_clayg/new-est?format=json&marker=test.big HTTP/1.1" 200 2
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/v1/AUTH_clayg/new-est?format=json&marker=test.big -X GET -H "X-Auth-Token: AUTH_tkXXX" -H "Accept-Encoding: gzip"
DEBUG:swiftclient:RESP STATUS: 200 OK
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'application/json; charset=utf-8', 'Content-Length': '2', 'X-Container-Object-Count': '1', 'X-Container-Bytes-Used': '2727', 'X-Timestamp': '1712858546.35960', 'Last-Modified': 'Thu, 11 Apr 2024 18:02:27 GMT', 'Accept-Ranges': 'bytes', 'X-Storage-Policy': 'EC-84-Policy', 'X-Container-Sharding': 'False', 'X-Trans-Id': 'tx5afdcab620244dd2a39bc-0066184a59-s8k-sjc11-d06-pxy-01', 'X-Openstack-Request-Id': 'tx5afdcab620244dd2a39bc-0066184a59-s8k-sjc11-d06-pxy-01', 'Date': 'Thu, 11 Apr 2024 20:38:49 GMT'}
DEBUG:swiftclient:RESP BODY: b'[]'
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "GET /auth/v1.0 HTTP/1.1" 200 0
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/auth/v1.0 -X GET
DEBUG:swiftclient:RESP STATUS: 200 OK
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'text/plain; charset=UTF-8', 'X-Auth-Token': 'AUTH_tkXXX', 'X-Storage-Token': 'AUTH_tk48bef83e838c4aa8a06cd9c92465d7a2', 'X-Storage-Url': 'https://pbss.s8k.io/v1/AUTH_clayg', 'Content-Length': '0', 'X-Trans-Id': 'txf71e7636cce44ba7b0fb3-0066184a59-s8k-sjc11-c21-pxy-01', 'X-Openstack-Request-Id': 'txf71e7636cce44ba7b0fb3-0066184a59-s8k-sjc11-c21-pxy-01', 'Date': 'Thu, 11 Apr 2024 20:38:49 GMT'}
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pbss.s8k.io:443
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "HEAD /v1/AUTH_clayg/new-est/test.big?symlink=get HTTP/1.1" 200 0
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/v1/AUTH_clayg/new-est/test.big?symlink=get -I -H "X-Auth-Token: AUTH_tkXXX"
DEBUG:swiftclient:RESP STATUS: 200 OK
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'application/octet-stream', 'Etag': '"71e938d37c1d06dc634dd24660255a88"', 'X-Object-Meta-Mtime': '1690982695.983535', 'X-Static-Large-Object': 'True', 'Last-Modified': 'Thu, 11 Apr 2024 18:02:35 GMT', 'X-Timestamp': '1712858554.30740', 'Accept-Ranges': 'bytes', 'Content-Length': '10485760', 'X-Manifest-Etag': '8261e54c8897faf876339362627c1ac5', 'X-Trans-Id': 'txd0d40df0b72b40bca0ebf-0066184a59-s8k-sjc11-a10-pxy-02', 'X-Openstack-Request-Id': 'txd0d40df0b72b40bca0ebf-0066184a59-s8k-sjc11-a10-pxy-02', 'Date': 'Thu, 11 Apr 2024 20:38:49 GMT'}
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "DELETE /v1/AUTH_clayg/new-est/test.big?multipart-manifest=delete HTTP/1.1" 200 None
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/v1/AUTH_clayg/new-est/test.big?multipart-manifest=delete DELETE -H "X-Auth-Token: AUTH_tkXXX"
DEBUG:swiftclient:RESP STATUS: 200 OK
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'text/plain', 'X-Trans-Id': 'tx85e13d9bf54a416486f26-0066184a59-s8k-sjc11-a10-pxy-02', 'X-Openstack-Request-Id': 'tx85e13d9bf54a416486f26-0066184a59-s8k-sjc11-a10-pxy-02', 'Date': 'Thu, 11 Apr 2024 20:38:50 GMT', 'Transfer-Encoding': 'chunked'}
DEBUG:swiftclient:RESP BODY: b'Number Deleted: 12\nNumber Not Found: 0\nResponse Body: \nResponse Status: 200 OK\nErrors:\n'
test.big
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "DELETE /v1/AUTH_clayg/new-est HTTP/1.1" 204 0
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/v1/AUTH_clayg/new-est DELETE -H "X-Auth-Token: AUTH_tkXXX"
DEBUG:swiftclient:RESP STATUS: 204 No Content
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'text/html; charset=UTF-8', 'Content-Length': '0', 'X-Trans-Id': 'tx7058e8f62f7a4b1c99a85-0066184a5a-s8k-sjc11-d06-pxy-01', 'X-Openstack-Request-Id': 'tx7058e8f62f7a4b1c99a85-0066184a5a-s8k-sjc11-d06-pxy-01', 'Date': 'Thu, 11 Apr 2024 20:38:50 GMT'}
new-est
Given that behavior I find this help output misleading:
(nvidia) clayg@banana:~/Workspace/nvidia$ swift delete test --help
Usage: swift delete [--all] [--leave-segments]
[--object-threads <threads>]
[--container-threads <threads>]
[--header <header:value>]
[--prefix <prefix>]
[--versions]
[<container> [<object>] [--version-id <version_id>] [...]]
Delete a container or objects within a container.
Positional arguments:
[<container>] Name of container to delete from.
[<object>] Name of object to delete. Specify multiple times
for multiple objects.
Optional arguments:
-a, --all Delete all containers and objects. Implies --versions.
--versions Delete all versions.
--leave-segments Do not delete segments of manifest objects.
-H, --header <header:value>
Adds a custom request header to use for deleting
objects or an entire container .
--object-threads <threads>
Number of threads to use for deleting objects.
Default is 10.
--container-threads <threads>
Number of threads to use for deleting containers.
Default is 10.
--prefix <prefix> Only delete objects beginning with <prefix>.
--version-id <version-id>
Delete specific version of a versioned object.
I think we fix the help output or change the behavior. Tested on python-swiftclient=4.5.0 |
|
| 2024-04-11 20:44:28 |
clayg |
description |
by default "swift delete container object" will remove the segments if object is a SLO
(nvidia) clayg@banana:~/Workspace/nvidia$ swift delete test test.big --debug
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pbss.s8k.io:443
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "GET /auth/v1.0 HTTP/1.1" 200 0
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/auth/v1.0 -X GET
DEBUG:swiftclient:RESP STATUS: 200 OK
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'text/plain; charset=UTF-8', 'X-Auth-Token': 'AUTH_tkXXX', 'X-Storage-Token': 'AUTH_tk068d458ce6a74b5e89c1a7d6b8b31b4d', 'X-Storage-Url': 'https://pbss.s8k.io/v1/AUTH_clayg', 'Content-Length': '0', 'X-Trans-Id': 'txedadc71bd53c45a5a1e37-006618259c-s8k-sjc3-c07-pxy-01', 'X-Openstack-Request-Id': 'txedadc71bd53c45a5a1e37-006618259c-s8k-sjc3-c07-pxy-01', 'Date': 'Thu, 11 Apr 2024 18:02:04 GMT'}
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pbss.s8k.io:443
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "HEAD /v1/AUTH_clayg/test/test.big?symlink=get HTTP/1.1" 200 0
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/v1/AUTH_clayg/test/test.big?symlink=get -I -H "X-Auth-Token: AUTH_tkXXX"
DEBUG:swiftclient:RESP STATUS: 200 OK
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'application/octet-stream', 'Etag': '"71e938d37c1d06dc634dd24660255a88"', 'X-Object-Meta-Mtime': '1690982695.983535', 'X-Static-Large-Object': 'True', 'Last-Modified': 'Thu, 11 Apr 2024 18:01:56 GMT', 'X-Timestamp': '1712858515.24231', 'Accept-Ranges': 'bytes', 'Content-Length': '10485760', 'X-Manifest-Etag': '65e88f63acacd29def554295241102c3', 'X-Trans-Id': 'tx523d01cb562743d4bfcf4-006618259d-s8k-sjc3-c07-pxy-01', 'X-Openstack-Request-Id': 'tx523d01cb562743d4bfcf4-006618259d-s8k-sjc3-c07-pxy-01', 'Date': 'Thu, 11 Apr 2024 18:02:05 GMT'}
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "DELETE /v1/AUTH_clayg/test/test.big?multipart-manifest=delete HTTP/1.1" 200 None
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/v1/AUTH_clayg/test/test.big?multipart-manifest=delete DELETE -H "X-Auth-Token: AUTH_tkXXX"
DEBUG:swiftclient:RESP STATUS: 200 OK
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'text/plain', 'X-Trans-Id': 'tx542c45aaf89b462e93fd7-006618259d-s8k-sjc3-c07-pxy-01', 'X-Openstack-Request-Id': 'tx542c45aaf89b462e93fd7-006618259d-s8k-sjc3-c07-pxy-01', 'Date': 'Thu, 11 Apr 2024 18:02:07 GMT', 'Transfer-Encoding': 'chunked'}
DEBUG:swiftclient:RESP BODY: b'Number Deleted: 12\nNumber Not Found: 0\nResponse Body: \nResponse Status: 200 OK\nErrors:\n'
test.big
but "swift delete container" will orphan the segments
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pbss.s8k.io:443
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "GET /auth/v1.0 HTTP/1.1" 200 0
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/auth/v1.0 -X GET
DEBUG:swiftclient:RESP STATUS: 200 OK
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'text/plain; charset=UTF-8', 'X-Auth-Token': 'AUTH_tkXXX', 'X-Storage-Token': 'AUTH_tk48bef83e838c4aa8a06cd9c92465d7a2', 'X-Storage-Url': 'https://pbss.s8k.io/v1/AUTH_clayg', 'Content-Length': '0', 'X-Trans-Id': 'tx33367a83996a4341b93e0-0066184a58-s8k-sjc3-d06-obj-01', 'X-Openstack-Request-Id': 'tx33367a83996a4341b93e0-0066184a58-s8k-sjc3-d06-obj-01', 'Date': 'Thu, 11 Apr 2024 20:38:48 GMT'}
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pbss.s8k.io:443
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "GET /v1/AUTH_clayg/new-est?format=json HTTP/1.1" 200 228
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/v1/AUTH_clayg/new-est?format=json -X GET -H "X-Auth-Token: AUTH_tkXXX" -H "Accept-Encoding: gzip"
DEBUG:swiftclient:RESP STATUS: 200 OK
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'application/json; charset=utf-8', 'Content-Length': '228', 'X-Container-Object-Count': '1', 'X-Container-Bytes-Used': '2727', 'X-Timestamp': '1712858546.35960', 'Last-Modified': 'Thu, 11 Apr 2024 18:02:27 GMT', 'Accept-Ranges': 'bytes', 'X-Storage-Policy': 'EC-84-Policy', 'X-Container-Sharding': 'False', 'X-Trans-Id': 'txac9df35f178d4bc68d83a-0066184a58-s8k-sjc11-d06-pxy-01', 'X-Openstack-Request-Id': 'txac9df35f178d4bc68d83a-0066184a58-s8k-sjc11-d06-pxy-01', 'Date': 'Thu, 11 Apr 2024 20:38:48 GMT'}
DEBUG:swiftclient:RESP BODY: b'[{"bytes": 10485760, "hash": "8261e54c8897faf876339362627c1ac5", "name": "test.big", "content_type": "application/octet-stream", "last_modified": "2024-04-11T18:02:34.307400", "slo_etag": "\\"71e938d37c1d06dc634dd24660255a88\\""}]'
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pbss.s8k.io:443
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "GET /v1/AUTH_clayg/new-est?format=json&marker=test.big HTTP/1.1" 200 2
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/v1/AUTH_clayg/new-est?format=json&marker=test.big -X GET -H "X-Auth-Token: AUTH_tkXXX" -H "Accept-Encoding: gzip"
DEBUG:swiftclient:RESP STATUS: 200 OK
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'application/json; charset=utf-8', 'Content-Length': '2', 'X-Container-Object-Count': '1', 'X-Container-Bytes-Used': '2727', 'X-Timestamp': '1712858546.35960', 'Last-Modified': 'Thu, 11 Apr 2024 18:02:27 GMT', 'Accept-Ranges': 'bytes', 'X-Storage-Policy': 'EC-84-Policy', 'X-Container-Sharding': 'False', 'X-Trans-Id': 'tx5afdcab620244dd2a39bc-0066184a59-s8k-sjc11-d06-pxy-01', 'X-Openstack-Request-Id': 'tx5afdcab620244dd2a39bc-0066184a59-s8k-sjc11-d06-pxy-01', 'Date': 'Thu, 11 Apr 2024 20:38:49 GMT'}
DEBUG:swiftclient:RESP BODY: b'[]'
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "GET /auth/v1.0 HTTP/1.1" 200 0
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/auth/v1.0 -X GET
DEBUG:swiftclient:RESP STATUS: 200 OK
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'text/plain; charset=UTF-8', 'X-Auth-Token': 'AUTH_tkXXX', 'X-Storage-Token': 'AUTH_tk48bef83e838c4aa8a06cd9c92465d7a2', 'X-Storage-Url': 'https://pbss.s8k.io/v1/AUTH_clayg', 'Content-Length': '0', 'X-Trans-Id': 'txf71e7636cce44ba7b0fb3-0066184a59-s8k-sjc11-c21-pxy-01', 'X-Openstack-Request-Id': 'txf71e7636cce44ba7b0fb3-0066184a59-s8k-sjc11-c21-pxy-01', 'Date': 'Thu, 11 Apr 2024 20:38:49 GMT'}
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pbss.s8k.io:443
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "HEAD /v1/AUTH_clayg/new-est/test.big?symlink=get HTTP/1.1" 200 0
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/v1/AUTH_clayg/new-est/test.big?symlink=get -I -H "X-Auth-Token: AUTH_tkXXX"
DEBUG:swiftclient:RESP STATUS: 200 OK
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'application/octet-stream', 'Etag': '"71e938d37c1d06dc634dd24660255a88"', 'X-Object-Meta-Mtime': '1690982695.983535', 'X-Static-Large-Object': 'True', 'Last-Modified': 'Thu, 11 Apr 2024 18:02:35 GMT', 'X-Timestamp': '1712858554.30740', 'Accept-Ranges': 'bytes', 'Content-Length': '10485760', 'X-Manifest-Etag': '8261e54c8897faf876339362627c1ac5', 'X-Trans-Id': 'txd0d40df0b72b40bca0ebf-0066184a59-s8k-sjc11-a10-pxy-02', 'X-Openstack-Request-Id': 'txd0d40df0b72b40bca0ebf-0066184a59-s8k-sjc11-a10-pxy-02', 'Date': 'Thu, 11 Apr 2024 20:38:49 GMT'}
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "DELETE /v1/AUTH_clayg/new-est/test.big?multipart-manifest=delete HTTP/1.1" 200 None
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/v1/AUTH_clayg/new-est/test.big?multipart-manifest=delete DELETE -H "X-Auth-Token: AUTH_tkXXX"
DEBUG:swiftclient:RESP STATUS: 200 OK
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'text/plain', 'X-Trans-Id': 'tx85e13d9bf54a416486f26-0066184a59-s8k-sjc11-a10-pxy-02', 'X-Openstack-Request-Id': 'tx85e13d9bf54a416486f26-0066184a59-s8k-sjc11-a10-pxy-02', 'Date': 'Thu, 11 Apr 2024 20:38:50 GMT', 'Transfer-Encoding': 'chunked'}
DEBUG:swiftclient:RESP BODY: b'Number Deleted: 12\nNumber Not Found: 0\nResponse Body: \nResponse Status: 200 OK\nErrors:\n'
test.big
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "DELETE /v1/AUTH_clayg/new-est HTTP/1.1" 204 0
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/v1/AUTH_clayg/new-est DELETE -H "X-Auth-Token: AUTH_tkXXX"
DEBUG:swiftclient:RESP STATUS: 204 No Content
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'text/html; charset=UTF-8', 'Content-Length': '0', 'X-Trans-Id': 'tx7058e8f62f7a4b1c99a85-0066184a5a-s8k-sjc11-d06-pxy-01', 'X-Openstack-Request-Id': 'tx7058e8f62f7a4b1c99a85-0066184a5a-s8k-sjc11-d06-pxy-01', 'Date': 'Thu, 11 Apr 2024 20:38:50 GMT'}
new-est
Given that behavior I find this help output misleading:
(nvidia) clayg@banana:~/Workspace/nvidia$ swift delete test --help
Usage: swift delete [--all] [--leave-segments]
[--object-threads <threads>]
[--container-threads <threads>]
[--header <header:value>]
[--prefix <prefix>]
[--versions]
[<container> [<object>] [--version-id <version_id>] [...]]
Delete a container or objects within a container.
Positional arguments:
[<container>] Name of container to delete from.
[<object>] Name of object to delete. Specify multiple times
for multiple objects.
Optional arguments:
-a, --all Delete all containers and objects. Implies --versions.
--versions Delete all versions.
--leave-segments Do not delete segments of manifest objects.
-H, --header <header:value>
Adds a custom request header to use for deleting
objects or an entire container .
--object-threads <threads>
Number of threads to use for deleting objects.
Default is 10.
--container-threads <threads>
Number of threads to use for deleting containers.
Default is 10.
--prefix <prefix> Only delete objects beginning with <prefix>.
--version-id <version-id>
Delete specific version of a versioned object.
I think we fix the help output or change the behavior. Tested on python-swiftclient=4.5.0 |
by default "swift delete container object" will remove the segments if object is a SLO
(nvidia) clayg@banana:~/Workspace/nvidia$ swift delete test test.big --debug
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pbss.s8k.io:443
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "GET /auth/v1.0 HTTP/1.1" 200 0
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/auth/v1.0 -X GET
DEBUG:swiftclient:RESP STATUS: 200 OK
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'text/plain; charset=UTF-8', 'X-Auth-Token': 'AUTH_tkXXX', 'X-Storage-Token': 'AUTH_tk068d458ce6a74b5e89c1a7d6b8b31b4d', 'X-Storage-Url': 'https://pbss.s8k.io/v1/AUTH_clayg', 'Content-Length': '0', 'X-Trans-Id': 'txedadc71bd53c45a5a1e37-006618259c-s8k-sjc3-c07-pxy-01', 'X-Openstack-Request-Id': 'txedadc71bd53c45a5a1e37-006618259c-s8k-sjc3-c07-pxy-01', 'Date': 'Thu, 11 Apr 2024 18:02:04 GMT'}
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pbss.s8k.io:443
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "HEAD /v1/AUTH_clayg/test/test.big?symlink=get HTTP/1.1" 200 0
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/v1/AUTH_clayg/test/test.big?symlink=get -I -H "X-Auth-Token: AUTH_tkXXX"
DEBUG:swiftclient:RESP STATUS: 200 OK
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'application/octet-stream', 'Etag': '"71e938d37c1d06dc634dd24660255a88"', 'X-Object-Meta-Mtime': '1690982695.983535', 'X-Static-Large-Object': 'True', 'Last-Modified': 'Thu, 11 Apr 2024 18:01:56 GMT', 'X-Timestamp': '1712858515.24231', 'Accept-Ranges': 'bytes', 'Content-Length': '10485760', 'X-Manifest-Etag': '65e88f63acacd29def554295241102c3', 'X-Trans-Id': 'tx523d01cb562743d4bfcf4-006618259d-s8k-sjc3-c07-pxy-01', 'X-Openstack-Request-Id': 'tx523d01cb562743d4bfcf4-006618259d-s8k-sjc3-c07-pxy-01', 'Date': 'Thu, 11 Apr 2024 18:02:05 GMT'}
DEBUG:urllib3.connectionpool:https://pbss.s8k.io:443 "DELETE /v1/AUTH_clayg/test/test.big?multipart-manifest=delete HTTP/1.1" 200 None
DEBUG:swiftclient:REQ: curl -i https://pbss.s8k.io/v1/AUTH_clayg/test/test.big?multipart-manifest=delete DELETE -H "X-Auth-Token: AUTH_tkXXX"
DEBUG:swiftclient:RESP STATUS: 200 OK
DEBUG:swiftclient:RESP HEADERS: {'Content-Type': 'text/plain', 'X-Trans-Id': 'tx542c45aaf89b462e93fd7-006618259d-s8k-sjc3-c07-pxy-01', 'X-Openstack-Request-Id': 'tx542c45aaf89b462e93fd7-006618259d-s8k-sjc3-c07-pxy-01', 'Date': 'Thu, 11 Apr 2024 18:02:07 GMT', 'Transfer-Encoding': 'chunked'}
DEBUG:swiftclient:RESP BODY: b'Number Deleted: 12\nNumber Not Found: 0\nResponse Body: \nResponse Status: 200 OK\nErrors:\n'
test.big
but "swift delete container" will orphan the segments - when I delete a container with >10K objects I noticed the segments were left behind and confirmed via logs all the DELETE requests were made with w/o any query params.
Given that behavior I find this help output misleading:
(nvidia) clayg@banana:~/Workspace/nvidia$ swift delete test --help
Usage: swift delete [--all] [--leave-segments]
[--object-threads <threads>]
[--container-threads <threads>]
[--header <header:value>]
[--prefix <prefix>]
[--versions]
[<container> [<object>] [--version-id <version_id>] [...]]
Delete a container or objects within a container.
Positional arguments:
[<container>] Name of container to delete from.
[<object>] Name of object to delete. Specify multiple times
for multiple objects.
Optional arguments:
-a, --all Delete all containers and objects. Implies --versions.
--versions Delete all versions.
--leave-segments Do not delete segments of manifest objects.
-H, --header <header:value>
Adds a custom request header to use for deleting
objects or an entire container .
--object-threads <threads>
Number of threads to use for deleting objects.
Default is 10.
--container-threads <threads>
Number of threads to use for deleting containers.
Default is 10.
--prefix <prefix> Only delete objects beginning with <prefix>.
--version-id <version-id>
Delete specific version of a versioned object.
I think we should fix the help output or change the behavior. Tested on python-swiftclient=4.5.0
N.B. in my simple dev test with only one object in the container it seems to work as you expect. Upon closer inspection I noticed my DELETE logs had source=BD |
|
