Swift client hangs during download -all with large number of objects

[Eric Miller]

CentOS 7.6 server and client machines
Kolla-Ansible 7.0.2 installed Swift Proxy and Swift
Latest pip and latest Swift client

I'm trying to download all objects from an account using the following command:
swift download --all

The download is being made to an XFS-formatted device.

The account has a single container that has 426,787 objects in it. I have also replicated this issue with another account with a single container that has over 600,000 objects.

Everything appears to be working fine, and then the swift client just stops. No further downloads occur and there is no I/O nor CPU utilization of the swift client process. The last attempt got through 320,869 downloads before it stopped.

I have also tried reducing the number of object and container threads to 1, and the same issue occurs. Every attempt stops at a different place, so it does not appear to be related to a specific object or path.

It does not appear to be a crashed Swift Proxy or Swift node - other clients can access files just fine through the same nodes.

Is there a limit somewhere that I'm hitting with the client? Or is there a better approach to downloading an entire account? The machine running the swift client has 16GiB of RAM and 8 vCPUs and is not running anything else.

I'm not quite sure how to debug this issue. Any suggestions?

Thanks!

Eric

[Eric Miller (erickmiller)]

Just a short note that when using rclone, no problems occur - I was able to sync 1.7TiB of data without issue, which included over 1 million files.

Eric