object decryption should be able to validate that correct key is being used

[Alistair Coles]

When decrypting an object and/or it's metadata, it would be useful to be able to validate that the key being used is in fact the same as the one used to encrypt the object. One use case for this would be a future key rotation/re-keying feature that might crawl a cluster to re-wrap object body keys - in this case it would be good to check that the 'old key' being used to unwrap the body key is definitely the correct key.

I think we already have the capability to do such a check on object *PUT* data because we have an HMAC of the etag persisted in metadata. So given a presumed old key, K1, we can decrypt the encrypted etag using K1, calculate the HMAC of that etag using K1 and compare it with the persisted HMAC.

However, I'm not sure we have a similar mechanism when considering POST metadata that *may* have encrypted with a different key K2 than the PUT data (in some future use case where we may be rotating keys and therefore have more than one key in use). We *do* have key id metadata that the keymaster has provided - but is that sufficient to verify that the presumed key value is in fact K2?

One idea would be to add some encrypted/HMAC'd magic value to the key id metadata, that can be used to vaidate the decryption key. This would be a relatively trivial addition. Although not required until a future use case such as key rotation is supported, including some key validation mechanism would future proof objects.

[Janie Richling (jrichli)]

@Ruslan Hello. I hope you don't mind, but I was thinking about this wishlist bug this weekend and have come up with some questions and a patch (#473149) to demonstrate what I was thinking. But I see that you are the assignee on this bug. It would be great to get your feedback on what I have proposed. Have you started work on this issue? If so, we can work together on the solution if you would like. You can reach me in #openstack-swift as jrichli

@Alistair Great wishlist idea. I uploaded the patch for implementing the check against the etag HMAC. As for verifying keys used for user-metadata, as you know, there is more discussion to be had in order to select some encrypted/HMAC'd magic value to the key id metadata.

IMHO, we are able to safely store off the HMAC of the etag because that value is not very sensitive. But when trying to think about persisting something new for a key verification check tends to lead to persisting data that we don't want to be in the business of persisting. I realize that some systems persist a hash of the key itself for verification purposes, but a determinate solution on a key means that you can tell when two are identical. Anyway, I don't want to get into all the thoughts here, but just wanted to give an example to communicate why I think there should be more than one patch for this.

Given that our user metadata is now encrypted with the same object key as the body wrapping and etag encryption, this check covers validation as the code stands today. This added check could be merged fairly quickly if separate, as compared to the probably more controversial solution that will most likely require persisting a new value.

[Janie Richling (jrichli)]

A couple more thoughts on this. There are still a couple cases where the decryption key is not verified (unless we persist something new for user-meta) even though the same object key is used for both body and headers at this point.

The on 'X-Object-Sysmeta-Crypto-Etag-Mac' only exists if the body was encrypted at PUT time.
So, if the body was unencrypted, but there was a POST later with encrypted headers, the key used for those headers could not be verified.

Secondly, there will be no way to verify the container key when decrypting etags on a container listing.