s3api ListParts does not return all parts

[Florent Vennetier]

When trying to list the parts of a multipart upload object with a small page size, some parts are missing from the result. This is due to the fact that part "obj/X/10" is sorted before "obj/X/2".

Steps to reproduce (I did this with SAIO virtual machine):
1. create a bucket
   aws --endpoint-url http://localhost:8080 s3 mb s3://bucket0
2. create a multipart object with at least 11 parts
   aws --endpoint-url http://localhost:8080 s3 cp ./400MB.dat s3://bucket0/400MB.dat
3. get the upload ID from the logs
   journalctl -o short-precise -t proxy-server | grep uploadId | sed -En -e 's,([^?]+)\?uploadId=([^&]+)&(.*),\2,p' | tail -n 1
4. list the parts
   aws --endpoint-url http://localhost:8080 s3api list-parts --bucket bucket0 --key 400MB.dat --upload-id ZTBkNDk0ZjUtN2RiNC00YTI1LWI4M2EtNWRlYTI5OWZkYzJm --page-size 5

This is what I get:

{
    "Parts": [
        {
            "PartNumber": 1,
            "LastModified": "2020-12-04T16:12:00.303Z",
            "ETag": "\"5f363e0e58a95f06cbe9bbc662c5dfb6\"",
            "Size": 5242880
        },
        {
            "PartNumber": 10,
            "LastModified": "2020-12-04T16:12:00.265Z",
            "ETag": "\"5f363e0e58a95f06cbe9bbc662c5dfb6\"",
            "Size": 5242880
        },
        {
            "PartNumber": 11,
            "LastModified": "2020-12-04T16:12:01.360Z",
            "ETag": "\"5f363e0e58a95f06cbe9bbc662c5dfb6\"",
            "Size": 5242880
        },
        {
            "PartNumber": 12,
            "LastModified": "2020-12-04T16:12:01.525Z",
            "ETag": "\"5f363e0e58a95f06cbe9bbc662c5dfb6\"",
            "Size": 5242880
        },
        {
            "PartNumber": 13,
            "LastModified": "2020-12-04T16:12:01.762Z",
            "ETag": "\"5f363e0e58a95f06cbe9bbc662c5dfb6\"",
            "Size": 5242880
        },
        {
            "PartNumber": 14,
            "LastModified": "2020-12-04T16:12:01.527Z",
            "ETag": "\"5f363e0e58a95f06cbe9bbc662c5dfb6\"",
            "Size": 5242880
        }
    ],
    "Initiator": {
        "ID": "test:tester",
        "DisplayName": "test:tester"
    },
    "Owner": {
        "DisplayName": "test:tester",
        "ID": "test:tester"
    },
    "StorageClass": "STANDARD"
}

Without specifying a page size in the 4th step, I get the 80 parts of my object.

In attachment is a unit test showing a bad "NextPartNumberMarker" in the first response from s3api.

[Tim Burke (1-tim-z)]

Thanks for the great test! I pushed up https://review.opendev.org/c/openstack/swift/+/765626 to address. Hope you don't mind but it has you as author, since the vast majority of the changes were yours!

[Florent Vennetier (fvennetier)]

Thanks for the submission. I amended the commit to fix pep8 issues.

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Tim Burke <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/swift/+/765626
Reason: Superseded by https://review.opendev.org/c/openstack/swift/+/811247

[Tim Burke (1-tim-z)]

Fixed by https://review.opendev.org/c/openstack/swift/+/811247