Nova instance backup with rotation 0 creates backup and deletes it immediately

[Bhagyashri Shewale]

If you create an instance backup by passing rotation 0 to the backup api, nova creates an image and takes a snapshot of an instance and then deletes it immediately. Any malicious user can use this loophole by calling backup api for all instances belonging to his/her project causing consumption of glance and nova resources at the expense of service provider.

Steps to reproduce:

1. Create the instance

$ nova boot --flavor <flavor_id> --image <image_id> <instance_name>

2. Create the instance backup using the backup api by giving the rotation parameter value as 0

$ nova backup <instance_id> <name> daily 0

3. Check the glance image list

$ glance image-list

Output:
In glance images you will not found backup image as it will be deleted immediately after creation.

Expected result:
If user passes rotation 0, then nova shouldn’t create an image and take a snapshot of an instance and upload it to glance service or minimum value of rotation should be changed from 0 to 1.

[Morgan Fainberg (mdrnstm)]

Since this report concerns a possible security risk, an incomplete security advisory task has been added while the core security reviewers for the affected project or projects confirm the bug and discuss the scope of any vulnerability along with potential solutions.

[Matt Riedemann (mriedem)]

I'm not sure how much of a DoS threat this is compared to just writing a script on the client side that loops continually creating server snapshots and deleting the created image repeatedly.

[Sean Dague (sdague)]

I agree, I am not convinced that this is anything more than a standard level of DOS that is doable with other API calls. As there is no API rate limiting built into OpenStack services any more, this is just something we leave for the operator to solve.

[Jeremy Stanley (fungi)]

I've switched the security advisory to opinion state and subscribed the security team reviewers to see whether this falls into the scope of existing work taken on by the security note editors.

[Tushar Patil (tpatil)]

It is true that it's not a DOS attack and it won't bring any of the OpenStack services down (glance/nova). But any user can call backup instance passing rotation as 0 consuming CPU/memory/network/disk resources of glance and nova hosts at the expense of service provider. In this situation, user cannot be charged a cent for backing up instance even though it utilized valuable resources of service provider.

[Jeremy Stanley (fungi)]

An attacker bent on "consuming CPU/memory/network/disk resources" isn't going to care whether the (stolen) credit card they signed up with gets charged. A failure to be able to bill users for resources they consume, while possibly still a bug worth fixing, doesn't on its own constitute a security vulnerability.

[Luke Hinds (lhinds)]

I am not convinced this is a vulnerability, its a functional bug on a lack of input validation to filter out 0.

[Travis McPeak (travis-mcpeak)]

Yeah, I agree, nothing here that isn't achievable with other API calls.

[Bhagyashri Shewale (bhagyashri-shewale)]

Hi All,

I want to propose 3 solution when rotation parameter is 0 while taking instance backup.

Solution 1: If user passed rotation 0 to create instance backup then avoid the backup image creation at api side and delete all the previous backup images in api side itself.

Pros: 1. Solution 1 will require minimum changes in code and will not make call to the compute node.

Cons: 2. The drawback of this solution is, it might impact on the response time if there are too many backups for that instance.

Solution 2: Avoid the backup image creation at api side and call the compute node by passing the image_id parameter as None and delete all the previous backup images at the compute node for the given instance.

Cons : 1. This solution need maximum changes at several places.
       2. Calling the compute node only for previous backup image deletion doesn't make any sense.

Solution 3: Change the api-spec that rotation parameter value should be greater than 0.

Pros: 1. Minimal changes required

Cons: 1. User will need to delete the last backup image explicitly.

To get the backup images of the specific instance for the explicit deletion, user needs to find out by using the following command:

$ glance image-list --property-filter <KEY=VALUE>

For example: $ glance image-list --property-filter instance_uuid=6d2b9b03-929e-4f9c-9977-05f3c2f3a47c

I want to recommend solution 1 and 3 is better one out of the three solutions.

Please let me know your opinions about the same.

[Tony Breeds (o-tony)]

My vote is Option 3. Making '0' rotations magic and deleting stuff is overreach IMO.

I feel like to can switch this to OSSN, and make it public. We can then work this in the open.

[Bhagyashri Shewale (bhagyashri-shewale)]

Please refer attached patch which resolves this issue.

[Sean Dague (sdague)]

I agree, make this public so that this can just go into gerrit.

It's still a super minor issue.

[Jeremy Stanley (fungi)]

Consensus seems to be that this discussion does not need to continue under embargo. I'm treating it as a class D (potential security hardening opportunity) report, in case the Nova developers are willing to consider accepting a patch to implement some additional validation on the rotation field to reject 0. https://security.openstack.org/vmt-process.html#incident-report-taxonomy

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/409644

[OpenStack Infra (hudson-openstack)]

Change abandoned by Matt Riedemann (<email address hidden>) on branch: master
Review: https://review.openstack.org/409644
Reason: The spec for this was abandoned:

https://review.openstack.org/#/c/511825/

And the code is also abandoned clearly.

Also: http://lists.openstack.org/pipermail/openstack-dev/2018-November/136496.html