S3 requests require multiple round-trips to Keystone

[Tim Burke]

The s3token middlware needs to make a request to Keystone to validate the request signature provided by the user. It picks out a token from the response and adds it to the request environment as an X-Auth-Token header. This then gets picked up the authtoken middleware which makes *another* request to validate the token *we just got*.

This seems rather silly, as the signature-verification response includes all of the (v2) auth params we need to satisfy the request. Further, it adds unnecessary load to the Keystone server (which has it hard enough as it is, since we shouldn't cache these responses) and unnecessary latency for the end-user.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/406424
Committed: https://git.openstack.org/cgit/openstack/swift3/commit/?id=6ffcc294e3881ccc59ff5840ff1568e44d9da4f8
Submitter: Jenkins
Branch: master

commit 6ffcc294e3881ccc59ff5840ff1568e44d9da4f8
Author: Tim Burke <email address hidden>
Date: Sat Dec 3 02:33:12 2016 +0000

    Only make one request to Keystone per S3 request

    Previously, s3token would make one request to Keystone to validate the
    request signature provided by the user, then use the response to add an
    X-Auth-Token header to the request environment. This would get picked up
    by the authtoken middleware which would make *another* request to
    validate the token *we just got*.

    Now, we'll populate the request environment with the following headers:

       * X-Identity-Status
       * X-Roles
       * X-User-Id
       * X-User-Name
       * X-Tenant-Id
       * X-Tenant-Name
       * X-Project-Id
       * X-Project-Name

    This allows Swift's keystoneauth middleware to function without needing
    the authtoken middleware at all.

    UpgradeImpact
    -------------
    The recommended pipeline ordering has changed. Whereas authoken previously
    had to be between s3token and keystoneauth like

      ... swift3 s3token authtoken keystoneauth ...

    it should now be placed before swift3, as in

      ... authtoken swift3 s3token keystoneauth ...

    Alternatively, if Keystone users should only ever access Swift through
    the S3 API, the authtoken middleware may be removed entirely. Note that
    the old pipeline ordering will continue to work, but still requires two
    Keystone requests per client request as before.

    To upgrade an existing cluster to take advantage of this change
    operators should, for each proxy server:

       1. Upgrade swift3
       2. Optionally, restart proxy-server
       3. Update proxy-server.conf with the new pipeline
       4. Restart proxy-server

    Updating proxy-server.conf *before* upgrading swift3 will prevent the
    proxy from starting if there is an unexpected reboot before the upgrade.

    Closes-Bug: #1653017
    Change-Id: I21e38884a2aefbb94b76c76deccd815f01db7362