non-ascii s3api usernames with invalid creds cause 500s

[clayg]

On a vsaio today you can have a non-ascii user setup and working with swift:

user_test☃_tester☃ = testing☃ .admin

You can also configure your aws-cli client and that works too:

aws_access_key_id = test☃:tester☃
aws_secret_access_key = testing☃

But if you *misconfigure* you aws-cli client

aws_access_key_id = test☃:tester☃
aws_secret_access_key = tetsing☃

... you'll get a traceback:

Mar 22 13:53:36 saio proxy-server: Expected a WSGI string; got '/v1/test☃:tester☃':
Traceback (most recent call last):
  File "/vagrant/swift/swift/common/middleware/s3api/s3api.py", line 349, in __call__
    resp = self.handle_request(req)
  File "/vagrant/swift/swift/common/middleware/s3api/s3api.py", line 389, in handle_request
    res = handler(req)
  File "/vagrant/swift/swift/common/middleware/s3api/controllers/service.py", line 35, in GET
    resp = req.get_response(self.app, query={'format': 'json'})
  File "/vagrant/swift/swift/common/middleware/s3api/s3request.py", line 1513, in get_response
    return self._get_response(app, method, container, obj,
  File "/vagrant/swift/swift/common/middleware/s3api/s3request.py", line 1412, in _get_response
    sw_resp = sw_req.get_response(app)
  File "/vagrant/swift/swift/common/swob.py", line 1165, in get_response
    status, headers, app_iter = self.call_application(application)
  File "/vagrant/swift/swift/common/swob.py", line 1149, in call_application
    app_iter = application(self.environ, start_response)
  File "/vagrant/swift/swift/common/middleware/s3api/s3api.py", line 195, in __call__
    return self.app(env, start_response)
  File "/vagrant/swift/swift/common/middleware/tempauth.py", line 349, in __call__
    return self.app(env, start_response)
  File "/vagrant/swift/swift/common/middleware/copy.py", line 233, in __call__
    return self.app(env, start_response)
  File "/vagrant/swift/swift/common/middleware/ratelimit.py", line 322, in __call__
    return self.app(env, start_response)
  File "/vagrant/swift/swift/common/middleware/slo.py", line 1894, in __call__
    return self.app(env, start_response)
  File "/vagrant/swift/swift/common/middleware/dlo.py", line 437, in __call__
    return self.app(env, start_response)
  File "/vagrant/swift/swift/common/middleware/versioned_writes/legacy.py", line 840, in __call__
    return self.app(env, start_response)
  File "/vagrant/swift/swift/common/middleware/versioned_writes/object_versioning.py", line 1475, in __call__
    return self.account_request(req, api_version, account,
  File "/vagrant/swift/swift/common/middleware/versioned_writes/object_versioning.py", line 1402, in account_request
    return account_ctx.list_containers(
  File "/vagrant/swift/swift/common/middleware/versioned_writes/object_versioning.py", line 1292, in list_containers
    app_resp = self._app_call(req.environ)
  File "/vagrant/swift/swift/common/wsgi.py", line 1138, in _app_call
    resp = reiterate(resp)
  File "/vagrant/swift/swift/common/utils/__init__.py", line 3749, in reiterate
    chunk = next(iterator)
  File "/vagrant/swift/swift/common/middleware/proxy_logging.py", line 437, in iter_response
    metric_name = self.statsd_metric_name(
  File "/vagrant/swift/swift/common/middleware/proxy_logging.py", line 363, in statsd_metric_name
    stat_type = self.get_metric_name_type(req)
  File "/vagrant/swift/swift/common/middleware/proxy_logging.py", line 351, in get_metric_name_type
    swift_path = req.environ.get('swift.backend_path', req.path)
  File "/vagrant/swift/swift/common/swob.py", line 1074, in path
    return wsgi_quote(self.environ.get('SCRIPT_NAME', '') +
  File "/vagrant/swift/swift/common/swob.py", line 314, in wsgi_quote
    raise TypeError('Expected a WSGI string; got %r' % wsgi_str)
TypeError: Expected a WSGI string; got '/v1/test☃:tester☃' (txn: txa08f5cb93853449dae13f-0065fd8d60)

the issue is related to how tempauth and probably other middlewares handle fixing the invalid string that s3api puts into PATH_INFO, probably s3api shouldn't pollute PATH_INFO with invalid strings.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/swift/+/913723
Committed: https://opendev.org/openstack/swift/commit/8424b02290c75a7e1eb2e36296b41926f041249a
Submitter: "Zuul (22348)"
Branch: master

commit 8424b02290c75a7e1eb2e36296b41926f041249a
Author: Tim Burke <email address hidden>
Date: Tue Mar 19 15:36:26 2024 -0700

    s3api: Fix handling of non-ascii access keys

    We stuff the access key into the request path until we get back a
    more-authoritative account name from auth. But it needs to be a WSGI
    string when we do!

    Closes-Bug: #2058748
    Change-Id: I34adb8141cc9e62d17a27f01c63f40d1dd25991c

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: feature/mpu
Review: https://review.opendev.org/c/openstack/swift/+/916416

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/swift/+/916416
Committed: https://opendev.org/openstack/swift/commit/ece8c28ce1fc6863226f24eae914679819a454ab
Submitter: "Zuul (22348)"
Branch: feature/mpu

commit 6387949a540e871e19062de8131503a8ac2ceef7
Author: Clay Gerrard <email address hidden>
Date: Thu Apr 4 10:36:36 2024 -0500

    lower-constraints: update to mock 3.0

    The main motivation here is that mock.call becomes a namedtuple and you
    can say `m.call_args_list[0].args` instead of `m.call_args_list[0][0]`

    Change-Id: Ibb1a64ef0bfdebf06d26636cdb6ea191c10705f7

commit 8424b02290c75a7e1eb2e36296b41926f041249a
Author: Tim Burke <email address hidden>
Date: Tue Mar 19 15:36:26 2024 -0700

    s3api: Fix handling of non-ascii access keys

    We stuff the access key into the request path until we get back a
    more-authoritative account name from auth. But it needs to be a WSGI
    string when we do!

    Closes-Bug: #2058748
    Change-Id: I34adb8141cc9e62d17a27f01c63f40d1dd25991c

commit f31b6f7353f6f8e91a452bf3586032b94b7898c8
Author: Tim Burke <email address hidden>
Date: Wed Mar 20 12:18:04 2024 -0700

    recon-cron: Tolerate missing directories

    Any of these directories may get unlinked between when we saw them in
    their parent's directory listing and when we go to descend.

    Change-Id: I1dfc0ee1d9e70cb0600557cde980bd5880bd40b3

commit 5d25108648a5676bf19a45e25ef714f0d4a2f9e1
Author: OpenStack Release Bot <email address hidden>
Date: Fri Mar 15 14:52:30 2024 +0000

    Update master for stable/2024.1

    Add file to the reno documentation build to show release notes for
    stable/2024.1.

    Use pbr instruction to increment the minor version number
    automatically so that master versions are higher than the versions on
    stable/2024.1.

    Sem-Ver: feature
    Change-Id: Ic940ff424aef9cc402bf54ebe5e5fc16330fc25c

commit 1c03d586431862c2772cb1b532aefc6cf2e671dc
Author: Tim Burke <email address hidden>
Date: Wed Jun 21 10:21:32 2023 -0700

    CI: Move probe tests to centos 9 stream

    Pin selenium to 3.x for now, until we can run down the issues with 4.x

    Change-Id: I596415d17f77f48a6e8a63a61b734a8ca0865847

commit af15ad53fb7818cdf14c92523fd6799bfabc12ec
Author: Tim Burke <email address hidden>
Date: Thu Mar 14 20:25:36 2024 -0700

    tests: Update CORS geckodriver

    Change-Id: I5ab762dfe0f85e346c4868ec4540884ba5f0a7f4