s3api doesn't delete SLOs during multideletes

[Tim Burke]

This sounds kind of like https://bugs.launchpad.net/swift/+bug/1805227, but isn't.

So we have some code that's supposed to handle multipart uploads [1] (inherited from swift3 [2]), but since we're using the bulk middleware machinery [3], we need to read the response (and check for errors!) if we're going to actually issue any deletes.

[1] https://github.com/openstack/swift/blob/2.20.0/swift/common/middleware/s3api/controllers/multi_delete.py#L117-L118
[2] https://github.com/openstack/swift3/blob/1.12/swift3/controllers/multi_delete.py#L110-L111
[3] https://github.com/openstack/swift/blob/2.20.0/swift/common/middleware/slo.py#L1416-L1419

[Bhaskar Singhal (bhaskarsinghal)]

What happens when I upload the same large object twice? As far as I have noticed, both uploads will get different uploadIds and we'll end up having one manifest (with the latest uploadIds) and the segments for both uploads (twice the space).

How will these older segments (without manifest file) get deleted?

[Tim Burke (1-tim-z)]

You're very good at hunting down s3api bugs, thank you! :-)

Wrote up https://bugs.launchpad.net/swift/+bug/1813202 as a separate issue.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/628701
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=b35fc41184edf8b6d88ccbf5a4e34a7924e9c46a
Submitter: Zuul
Branch: master

commit b35fc41184edf8b6d88ccbf5a4e34a7924e9c46a
Author: Tim Burke <email address hidden>
Date: Fri Jan 4 23:06:06 2019 +0000

    s3api: Delete multipart uploads via multi-delete

    We have code that's *supposed* to do it, but we weren't reading the
    result of the bulk-delete, so we never actually deleted anything!

    Change-Id: I5c972749cadf903161456f34371a6f83ebc05eb9
    Closes-Bug: 1810567

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: feature/losf
Review: https://review.openstack.org/640955

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/640955
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=d90ab533ba421602429c89f63e306f4e62ac9c78
Submitter: Zuul
Branch: feature/losf

commit 860ae1e001d95df3096cfc81c42a9fe6feda067f
Author: Pete Zaitcev <email address hidden>
Date: Mon Feb 18 19:03:49 2019 -0600

    py3: port proxy account controller

    Change-Id: I5a0d2108d01395aedb4ac18464ed2e38ad13ce84

commit 8b519d1abcb4d7b34c8bbdfe535f472fe105231c
Author: Tim Burke <email address hidden>
Date: Fri Mar 1 12:13:27 2019 -0800

    Clean up func tests ahead of py3

    - ConfigParser.set() requires that the value be a string
    - The stdlib HTTP client responses don't have a body property
    - We might raise a ResponseError with response=None
    - Bodies should be bytes
    - Headers should be strings
    - Make containers()/files() return native strings
    - file() isn't a thing on py3
    - format should be a parm, not a header
    - Switch sorted() to use key instead of cmp
    - Use integer division explicitly

    Change-Id: I99d3eebc9d7ec4e8b295352294b831492135c568

commit 45994e4d3187decdfa2513c0fc0fa337ec888948
Author: Tim Burke <email address hidden>
Date: Mon Feb 25 14:00:45 2019 -0800

    py3: port object updater

    Change-Id: Ib1cd0659d5085ff9f8555bc08bd17a8f02f2efc0

commit 349c54998461accd25b84e32781a10938d4167a4
Author: Tim Burke <email address hidden>
Date: Mon Feb 25 13:42:37 2019 -0800

    py3: port object auditor

    Change-Id: Ic00273e5d3120f7daba1d3cbf084159f1e7d8bbb

commit 04a0ff2e969cde4192d54a47ef11e9c0505a9555
Author: John Dickinson <email address hidden>
Date: Mon Feb 25 16:38:07 2019 -0800

    added changelog entries from stable releases

    Change-Id: Ia2ba8f7a998bc55ba3fba3ce37f1d2974646494f

commit 5409c4f347ee0f600249794844d61635136fb075
Author: Tim Burke <email address hidden>
Date: Fri Feb 22 14:23:15 2019 -0800

    Make test_multiphase_put_drains_extra_commit_junk_disconnect less flakey

    Change-Id: I82503b13e4541196ad056e861221e9429c7f2c1c

commit be3e21582967724f23db6e671f2c29128d39472e
Author: Tim Burke <email address hidden>
Date: Thu Feb 21 10:13:48 2019 -0800

    py3: display help if no subcommand provided to manage-shard-ranges

    That way we don't traceback when you just provide a DB file.

    At some point, py3 switched to having optional subparsers. In py37,
    they added a kwarg to say whether a subparser is optional or required,
    but for the sake of earlier versions, we have to check whether it was
    set manually and cannot rely on argparse doing it for us. See also:

    - https://bugs.python.org/issue9253
    - https://github.com/python/cpython/commit/aaf6fc0
    - https://bugs.python.org/issue33109
    - https://github.com/python/cpython/commit/8ebf5ce

    Change-Id: Iac8048ad0e23ffb28b914fe880c4b6538bc41c86

commit b4e47007655e97625d1a88dad1a9f5501340658e
Author: Tim Burke <email address hidden>
Date: Wed Feb 20 14:59:49 2019 -0800

    py3: port some more middleware tests

      * name_check
      * quotas
      * ratelimit
      * read_only
      * recon

    Note that the middlewares themse...

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/swift 2.21.0 release.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/690102

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/690102
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=9bcce90be0c0d8a9cf0d2bb03cc02fb5ceb62970
Submitter: Zuul
Branch: stable/rocky

commit 9bcce90be0c0d8a9cf0d2bb03cc02fb5ceb62970
Author: Tim Burke <email address hidden>
Date: Fri Jan 4 23:06:06 2019 +0000

    s3api: Delete multipart uploads via multi-delete

    We have code that's *supposed* to do it, but we weren't reading the
    result of the bulk-delete, so we never actually deleted anything!

    Change-Id: I5c972749cadf903161456f34371a6f83ebc05eb9
    Closes-Bug: 1810567
    (cherry picked from commit b35fc41184edf8b6d88ccbf5a4e34a7924e9c46a)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/swift 2.19.2 release.