non exploitable resource exhaustion with malicious vmdk

Bug #2004565 reported by Felix Huettner
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Cinder
In Progress
High
Unassigned

Bug Description

This was found after playing with [1].
The vulnerability is currently not exploitable (see description below).

In case you have a vmdk file like the following:
```
version=1
CID=b1a17f47
parentCID=ffffffff
createType="monolithicFlat"

RW 1 FLAT "/etc/passwd" 0
RW 1 FLAT "/etc/passwd" 0
[... assume here are now 1000000 lines as above ...]
RW 1 FLAT "/etc/passwd" 0
RW 1 FLAT "/etc/passwd" 0

ddb.virtualHWVersion = "4"
ddb.geometry.cylinders = "20805"
ddb.geometry.heads = "16"
ddb.geometry.sectors = "63"
ddb.adapterType = "ide"
ddb.toolsVersion = "2147483647"
```

In this case the initial `qemu-img info` call in
`_create_from_image_cache_or_download` will take a long time to complete
and consume a large amount of memory. `qemu-img` will be terminated by
the `image_conversion_cpu_limit` after 60 seconds per default. It will
have consumed a full cpu core for the entire time of it running.

If you now assume a user can create a large amount of volumes in
parallel from such a malicious image, the user would cause a
denial-of-service of the node cinder-volume runs on by exhausing the cpu
cores available there.

As `qemu-img info` will only close the files after all lines have been
processes we can limit the processing by setting a maximum number of
open files.

I choose 1024 here as this is already accidentially enforced by default
and we can be quite sure that no reasonable image will ever need 1024 files
open at once during conversion. If this is set then the `qemu-img info` call
terminates after 0.5 seconds. This siginificantly reduces the denial-of-service
potential.

Why this is currently not exploitable:
1. The call in `_create_from_image_cache_or_download` for `qemu_img_info`
   is using the default `run_as_root=True`
2. `qemu_img_info` specifies a `prlimit` for address space and cpu time
3. The processes being executed then are (in order)
   1. `oslo_concurrency.prlimit` calling
   2. `sudo` calling
   3. `cinder-rootwrap` calling
   4. `qemu-img`
4. the resource limits applied are:
   1. for `oslo_concurrency.prlimit`: the limits for address space and
      cpu time
   2. for `cinder-rootwrap`: a limit for the amount of file descriptors
      (to 1024 per default) at [2]
5. the "accidential" limit by `cinder-rootwrap` here (see comments at [2])
   prevents exploitation. However from the comments of `olso.rootwrap` this
   does not look like an actual security feature, but rather a fix for the
   shortcomings of python2. Therefor we should still set the value
   explicitly on the cinder side to prevent exploitation once the code
   of `oslo.rootwrap` might be removed in the future.

[1]: https://bugs.launchpad.net/nova/+bug/1996188
[2]: https://opendev.org/openstack/oslo.rootwrap/src/commit/349603af2ea0d2da21adc50b317936c58480c202/oslo_rootwrap/cmd.py#L91-L118

Tags: hardening qemu
Changed in cinder:
status: New → In Progress
Changed in cinder:
importance: Undecided → High
Revision history for this message
Sofia Enriquez (lsofia-enriquez) wrote :
tags: added: hardening qemu
Revision history for this message
Brian Rosmaita (brian-rosmaita) wrote :
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.