Comment 6 for bug 1856303

Iiiinteresting...

So if I take out one of my proxy-logging middlewares, I can (mostly) repro:

$ curl $(aws s3 presign s3://bucket/obj) -svI >/dev/null
* Trying 10.0.2.15...
* TCP_NODELAY set
* Connected to saio (10.0.2.15) port 8080 (#0)
> HEAD /bucket/obj?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=test%3Atester%2F20191217%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20191217T221750Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=f338a86dbd7d7c8edbf3556087f70fa1085717c900498fe4cbebffe397a26481 HTTP/1.1
> Host: saio:8080
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 403 Forbidden
< Content-Type: application/xml
< x-amz-id-2: tx6bd6efa997a04ee58c633-005df9540e
< x-amz-request-id: tx6bd6efa997a04ee58c633-005df9540e
< X-Trans-Id: tx6bd6efa997a04ee58c633-005df9540e
< X-Openstack-Request-Id: tx6bd6efa997a04ee58c633-005df9540e
< Date: Tue, 17 Dec 2019 22:17:50 GMT
< Transfer-Encoding: chunked
<
* Excess found in a non pipelined read: excess = 5 url = /bucket/obj?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=test%3Atester%2F20191217%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20191217T221750Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=f338a86dbd7d7c8edbf3556087f70fa1085717c900498fe4cbebffe397a26481 (zero-length body)
* Connection #0 to host saio left intact

(There's a slight difference in that I get a 403 since I can't seem to convince awscli to presign a non-GET request, but the "Transfer-Encoding: chunked" and "Excess found in a non pipelined read: excess = 5" bad are still there.)

Can you try putting a second swift-proxy-logging in your pipeline? If it were me, I'd probably put it between healthcheck and cache, like

pipeline = catch_errors gatekeeper healthcheck swift-proxy-logging cache listing_formats bulk tempurl crossdomain authtoken s3api s3token keystoneauth staticweb copy container-quotas account-quotas slo dlo versioned_writes swift-proxy-logging proxy-server

Don't need to do any other config, I don't think; it'll use the same [filter:swift-proxy-logging] section for both instances. FWIW, we've recommended doubled-up logging filters like that for quite a while, since https://github.com/openstack/swift/commit/a622349 -- it helps you have a more complete picture of what (sub)requests are getting made and by whom.