MPU cross-policy-segments writes manifest to wrong policy

[clayg]

If you carefully construct your +segments container for s3api such that MPU's segments go into a different (e.g. EC) policy than you source bucket (e.g. 3-replica manifests) then when you complete the multi-part-upload with the S3 API call the manifest will get written into the correct bucket - but the wrong policy.

https://review.opendev.org/c/openstack/swift/+/866508

The reconciler will try to fix the SLO-manifest policy but there's another issue there lp bug #2038458 - so better if we just don't do it in the first place.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/swift/+/897864

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Clay Gerrard <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/swift/+/866508
Reason: we're going to head over this way: https://review.opendev.org/c/openstack/swift/+/897864

[Alistair Coles (alistair-coles)]

The bug was introduced by introduced by https://review.opendev.org/c/openstack/swift/+/755587)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/swift/+/900817

[OpenStack Infra (hudson-openstack)]

Change abandoned by "ASHWIN A NAIR <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/swift/+/900817
Reason: we already have https://review.opendev.org/c/openstack/swift/+/897864

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/swift/+/897864
Committed: https://opendev.org/openstack/swift/commit/60c04f116b41f97a5dcf9c07ef6e77ae9bdfe61e
Submitter: "Zuul (22348)"
Branch: master

commit 60c04f116b41f97a5dcf9c07ef6e77ae9bdfe61e
Author: Alistair Coles <email address hidden>
Date: Tue Oct 10 16:54:21 2023 +0100

    s3api: Stop propagating storage policy to sub-requests

    The proxy_logging middleware needs an X-Backend-Storage-Policy-Index
    header to populate the storage policy field in logs, and will look in
    both request and response headers to find it.

    Previously, the s3api middleware would indiscriminately copy the
    X-Backend-Storage-Policy-Index from swift backend requests into the
    S3Request headers [1]. This works for logging but causes the header
    to leak between backend requests [2] and break mixed policy
    multipart uploads. This patch sets the X-Backend-Storage-Policy-Index
    header on s3api responses rather than requests.

    Additionally, the middleware now looks for the
    X-Backend-Storage-Policy-Index header in the swift backend request
    *and* response headers, in the same way that proxy_logging would
    (preferring a response header over a request header). This means that
    a policy index is now logged for bucket requests, which only have
    X-Backend-Storage-Policy-Index header in their response headers.

    The s3api adds the value from the *final* backend request/response
    pair to its response headers. Returning the policy index from the
    final backend request/response is consistent with swift.backend_path
    being set to that backend request's path i.e. proxy_logging will log
    the correct policy index for the logged path.

    The FakeSwift helper no longer looks in registered object responses
    for an X-Backend-Storage-Policy-Index header to update an object
    request. Real Swift object responses do not have an
    X-Backend-Storage-Policy-Index header. By default, FakeSwift will now
    update *all* object requests with an X-Backend-Storage-Policy-Index as
    follows:

      - If a matching container HEAD response has been registered then
        any X-Backend-Storage-Policy-Index found with that is used.
      - Otherwise the default policy index is used.

    Furthermore, FakeSwift now adds the X-Backend-Storage-Policy-Index
    header to the request *after* the request has been captured. Tests
    using FakeSwift.calls_wth_headers() to make assertions about captured
    headers no longer need to make allowance for the header that FakeSwift
    added.

    Co-Authored-By: Clay Gerrard <email address hidden>
    Closes-Bug: #2038459
    [1] Related-Change: I5fe5ab31d6b2d9f7b6ecb3bfa246433a78e54808
    [2] Related-Change: I40b252446b3a1294a5ca8b531f224ce9c16f9aba
    Change-Id: I2793e335a08ad373c49cbbe6759d4e97cc420867

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/swift 2.33.0 release.