Comment 12 for bug 1196932

Revision history for this message
Samuel Merritt (torgomatic) wrote : Re: Possibly DoS attack using object tombstones

So, a few things:

First: after a replication pass, all the superfluous tombstones are cleaned up.

Second, I can pretty easily reproduce this with these commands:

    curl -v -X PUT -H "X-Timestamp: 1373004425" --data-binary "helloworld" -H "X-Auth-Token: $TOKEN" http://192.168.22.2:8080/v1.0/AUTH_test/con/obj
    curl -v -X DELETE -H "X-Timestamp: 1373004424" -H "X-Auth-Token: $TOKEN" http://192.168.22.2:8080/v1.0/AUTH_test/con/obj
    curl -v -X DELETE -H "X-Timestamp: 1373004423" -H "X-Auth-Token: $TOKEN" http://192.168.22.2:8080/v1.0/AUTH_test/con/obj
    curl -v -X DELETE -H "X-Timestamp: 1373004422" -H "X-Auth-Token: $TOKEN" http://192.168.22.2:8080/v1.0/AUTH_test/con/obj
    curl -v -X DELETE -H "X-Timestamp: 1373004421" -H "X-Auth-Token: $TOKEN" http://192.168.22.2:8080/v1.0/AUTH_test/con/obj
    curl -v -X DELETE -H "X-Timestamp: 1373004420" -H "X-Auth-Token: $TOKEN" http://192.168.22.2:8080/v1.0/AUTH_test/con/obj
    curl -v -X GET -H "X-Auth-Token: $TOKEN" http://192.168.22.2:8080/v1.0/AUTH_test/con/obj # gets "helloworld" as it should

Obviously, fill in your own Swift cluster's IP and auth token.

So, the bug is real. Is it a security vulnerability? I don't know.

Let's do some napkin math here. Let's say you can create bogus tombstones at a rate of 10 per second, and that a replication pass takes two days. That means you can get 86400 * 2 * 10 = 1728000 tombstones before they get cleaned up. Assuming that an os.listdir() returns strings that take 100 bytes of memory each (for reference counting, overhead, and whatever) and you get that an os.listdir() will require ~165 MiB to return its results. That's bad, but not the end of the world on decently-sized hardware, so I don't think that counts as a DoS attack. And that's worst-case, too... if your req/s is lower, or your replicators are faster, or whatever, then the memory usage goes down.

I'm not enough of an XFS expert to comment on the inode exhaustion problem.