[OSSA 2013-029] Unchecked qcow2 root disk sizes DoS
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
| OpenStack Compute (nova) |
High
|
Pádraig Brady | ||
| Folsom |
High
|
Pádraig Brady | ||
| Grizzly |
High
|
Pádraig Brady | ||
| Havana |
High
|
Pádraig Brady | ||
| OpenStack Security Advisory |
Medium
|
Thierry Carrez |
Bug Description
When doing QA for SUSE on bug 1177830
I found that the fix is incomplete,
because it assumed that the cached image would be mostly sparse.
However, I can easily create non-sparse small compressed qcow2 images with
perl -e 'for(1.
qemu-img convert -c -O qcow2 img img.qcow2
glance image-create --name=11gb --is-public=True --disk-format=qcow2 --container-
nova boot --image 11gb --flavor m1.small testvm
which (in Grizzly and Essex) results in one (or two in Essex) 11GB large files being created in /var/lib/
still allowing attackers to fill up disk space of compute nodes
because the size check is only done after the uncompressing / caching
Bernhard M. Wiedemann (ubuntubmw) wrote : | #1 |
Russell Bryant (russellb) wrote : | #3 |
Added Pádraig Brady to help review this.
Pádraig Brady (p-draigbrady) wrote : | #4 |
Refreshing my memory for the flow of these images ... http://
Yes this does seem like a valid issue, though not as serious as the related bug 1177830
as there is only an allocation per image rather than per instance.
Also any sizes over the quote size etc. for an instance wouldn't be thus referenced and so
could be auto deleted at some stage.
But yes this is a valid DOS vector given the difference between input and output sizes.
Now if we did limit sizes, I suppose we could limit to the max flavor size?
But that doesn't help much. One could still upload many small images up to the max flavor size to fill up the _base/
Maybe we need quotas glance side for max actual disk usage per user for all their images?
Thierry Carrez (ttx) wrote : | #5 |
Looks like we'll need an OSSA on this then.
Changed in ossa: | |
importance: | Undecided → Medium |
status: | Incomplete → Confirmed |
Thierry Carrez (ttx) wrote : | #6 |
Anyone on the current subscription list up for a patch ?
Thierry Carrez (ttx) wrote : | #7 |
Padraig, Mikal: any chance you could produce a patch to fix this ?
Changed in nova: | |
importance: | Undecided → High |
status: | New → Confirmed |
Pádraig Brady (p-draigbrady) wrote : | #8 |
Ok I'll take this (was away for a couple of weeks). I might get to it in the next couple of days
Changed in nova: | |
assignee: | nobody → Pádraig Brady (p-draigbrady) |
Thierry Carrez (ttx) wrote : | #9 |
@Pádraig: any progress ?
Pádraig Brady (p-draigbrady) wrote : | #10 |
Lets summarize _this_ bug as "per _image_ qcow2 root disk sizes DoS"
and bug 1177830 as "per _instance_ qcow2 root disk sizes DoS".
Unfortunately I think that even bug 1177830 mightn't be addressed in the case
where use_cow_
On initial inspection it seems like the large qcow2 files could be copied to each instance here too.
So thoughts on addressing this.
1. Always set min_disk = virt_size for an image in glance.
This is already checked in nova before we download anything.
That has the advantage of saving on network bandwidth and is
honored by cinder too for volumes.
2. Pass/Inspect size into nova.virt.
and disallow disk.get_
Doing either should address both bugs I think,
but I'm leaning towards 1 as a more general solution.
Pádraig Brady (p-draigbrady) wrote : | #11 |
Actually we probably should do both.
Better to have nova not rely on a particular glance implementation or version or glance at all for that matter.
I'll do the nova change now...
Changed in ossa: | |
assignee: | nobody → Thierry Carrez (ttx) |
Thierry Carrez (ttx) wrote : | #12 |
@Padraig: we'll take the Nova patch as the vulnerability fix. We'll consider the Glance as a hardening improvement.
Thierry Carrez (ttx) wrote : | #13 |
First attempt at an impact description, not sure I got this right:
-------
Title: Potential Nova denial of service through compressed disk images
Reporter: Bernhard M. Wiedemann (SUSE)
Products: Nova
Affects: All versions
Description:
Bernhard M. Wiedemann from SUSE reported a vulnerability in Nova's control of the size of disk images. By using malicious compressed qcow2 disk images, an authenticated user may consume large amounts of disk space, potentially resulting in a Denial of Service attack on Nova compute nodes. This issue is slightly different from CVE-2013-2096 which was addressed in OSSA 2013-012.
-------
Changed in ossa: | |
status: | Confirmed → Triaged |
Pádraig Brady (p-draigbrady) wrote : | #14 |
Re impact description, I'd mention that CVE-2013-2096 wasn't fully addressed either
in the non default case where use_cow_
are being transferred from glance. If mentioning both it's probably worth mentioning
the original CVE-2013-2096 was a per instance issue, while this new one is per image.
Changed in nova: | |
status: | Confirmed → In Progress |
Changed in nova: | |
milestone: | none → havana-rc1 |
tags: | added: havana-rc-potential libvirt |
Thierry Carrez (ttx) wrote : | #18 |
RC1 will probably be out before we can properly embargo and release this as an OSSA. This should stay as havana-rc-potential just in case it can make it prerelease.
Would be good to get some core pre-approvals on the proposed patches. Feel free to subscribe fellow core developers to accelerate that.
Changed in nova: | |
milestone: | havana-rc1 → none |
Russell Bryant (russellb) wrote : | #19 |
Added a few nova developers that could help review these patches
Thierry Carrez (ttx) wrote : | #20 |
Hey guys, please review proposed patches !
Russell Bryant (russellb) wrote : | #21 |
+2 on the patch
Changed in nova: | |
milestone: | none → icehouse-1 |
Pádraig Brady (p-draigbrady) wrote : | #22 |
respinning with just improved commit messages
Pádraig Brady (p-draigbrady) wrote : | #23 |
Pádraig Brady (p-draigbrady) wrote : | #24 |
Pádraig Brady (p-draigbrady) wrote : | #25 |
Thierry Carrez (ttx) wrote : | #26 |
Nova core: Please review proposed patches
tags: |
added: havana-backport-potential removed: havana-rc-potential |
Thierry Carrez (ttx) wrote : | #27 |
New attempt at impact description:
-------
Title: Potential Nova denial of service through compressed disk images
Reporter: Bernhard M. Wiedemann (SUSE)
Products: Nova
Affects: All versions
Description:
Bernhard M. Wiedemann from SUSE reported a vulnerability in Nova's control of the size of disk images. By using malicious compressed qcow2 disk images, an authenticated user may consume large amounts of disk space for each image, potentially resulting in a Denial of Service attack on Nova compute nodes. While fixing this issue, Pádraig Brady from Red Hat additionally discovered that OSSA 2013-012 did not fully address CVE-2013-2096 in the non-default case where use_cow_
-------
Vish Ishaya (vishvananda) wrote : | #28 |
+2 on patches
Pádraig Brady (p-draigbrady) wrote : | #29 |
+1 on impact description
Thierry Carrez (ttx) wrote : | #31 |
CVE-2013-4463
Thierry Carrez (ttx) wrote : | #32 |
Proposed public disclosure date/time: Thursday, October 31, 1500UTC.
Changed in ossa: | |
status: | In Progress → Fix Committed |
Thierry Carrez (ttx) wrote : | #33 |
CVE SPLIT w/ CVE-2013-4469:
Title: Potential Nova denial of service through compressed disk images
Reporter: Bernhard M. Wiedemann (SUSE) & Pádraig Brady (Red Hat)
Products: Nova
Affects: All versions
Description:
Bernhard M. Wiedemann from SUSE reported a vulnerability in Nova's
control of the size of disk images. By using malicious compressed qcow2
disk images, an authenticated user may consume large amounts of disk
space for each image, potentially resulting in a Denial of Service
attack on Nova compute nodes (CVE-2013-4463). While fixing this issue, Pádraig Brady from Red Hat additionally discovered that OSSA 2013-012 did not fully address CVE-2013-2096 in the non-default case where use_cow_
Thierry Carrez (ttx) wrote : | #34 |
Patches test runs:
master: tox, smoke PASS
havana: tox, neutron PASS
grizzly: tox, full PASS
information type: | Private Security → Public Security |
Fix proposed to branch: stable/havana
Review: https:/
master: https:/
stable/havana: https:/
stable/grizzly: https:/
Thierry Carrez (ttx) wrote : | #37 |
[OSSA 2013-029]
summary: |
- Unchecked qcow2 root disk sizes DoS + [OSSA 2013-029] Unchecked qcow2 root disk sizes DoS |
Reviewed: https:/
Committed: http://
Submitter: Jenkins
Branch: master
commit f6810be4ae1a6c9
Author: Pádraig Brady <email address hidden>
Date: Fri Sep 27 04:07:14 2013 +0100
ensure we don't boot oversized images
Since we can't generally shrink incoming images, add extra checks
to ensure oversized images are not allowed through.
All cases when populating the libvirt image cache are now handled,
including the initial download from glance, where we avoid
converting to raw, as that could generate non sparse images
much larger than the downloaded image.
* nova/virt/
of the max_size parameter.
* nova/virt/images.py (fetch_to_raw): Accept the max_size parameter,
and use it to discard images with larger (virtual) sizes.
* nova/virt/
refactored function to identify and raise exception to oversized images.
(Raw.
Also enforce virtual image size checking for already fetched images,
as this class (despite the name) can be handling qcow files.
(Qcow2.
or verify the virtual size for the instance as done previously.
(Lvm.
Also check the size before transferring to the volume to improve
efficiency by not even attempting the transfer of oversized images.
(Rbd.
* nova/tests/
* nova/tests/
Add a case to check oversized images are discarded.
* nova/tests/
(test_
Fixes bug: 1177830
Fixes bug: 1206081
Change-Id: I3d47adaa2ad074
Changed in nova: | |
status: | In Progress → Fix Committed |
tags: | added: in-stable-havana |
Reviewed: https:/
Committed: http://
Submitter: Jenkins
Branch: stable/havana
commit 3cdfe894ab58f7b
Author: Pádraig Brady <email address hidden>
Date: Fri Sep 27 04:07:14 2013 +0100
ensure we don't boot oversized images
Since we can't generally shrink incoming images, add extra checks
to ensure oversized images are not allowed through.
All cases when populating the libvirt image cache are now handled,
including the initial download from glance, where we avoid
converting to raw, as that could generate non sparse images
much larger than the downloaded image.
* nova/virt/
of the max_size parameter.
* nova/virt/images.py (fetch_to_raw): Accept the max_size parameter,
and use it to discard images with larger (virtual) sizes.
* nova/virt/
refactored function to identify and raise exception to oversized images.
(Raw.
Also enforce virtual image size checking for already fetched images,
as this class (despite the name) can be handling qcow files.
(Qcow2.
or verify the virtual size for the instance as done previously.
(Lvm.
Also check the size before transferring to the volume to improve
efficiency by not even attempting the transfer of oversized images.
(Rbd.
* nova/tests/
* nova/tests/
Add a case to check oversized images are discarded.
* nova/tests/
(test_
Fixes bug: 1177830
Fixes bug: 1206081
Change-Id: I3d47adaa2ad074
Reviewed: https:/
Committed: http://
Submitter: Jenkins
Branch: stable/grizzly
commit 135faa7b5d98553
Author: Pádraig Brady <email address hidden>
Date: Fri Sep 27 04:07:14 2013 +0100
ensure we don't boot oversized images
Since we can't generally shrink incoming images, add extra checks
to ensure oversized images are not allowed through.
All cases when populating the libvirt image cache are now handled,
including the initial download from glance, where we avoid
converting to raw, as that could generate non sparse images
much larger than the downloaded image.
* nova/virt/
of the max_size parameter.
* nova/virt/images.py (fetch_to_raw): Accept the max_size parameter,
and use it to discard images with larger (virtual) sizes.
* nova/virt/
refactored function to identify and raise exception to oversized images.
(Raw.
Also enforce virtual image size checking for already fetched images,
as this class (despite the name) can be handling qcow files.
(Qcow2.
or verify the virtual size for the instance as done previously.
(Lvm.
Also check the size before transferring to the volume to improve
efficiency by not even attempting the transfer of oversized images.
(Rbd.
* nova/tests/
* nova/tests/
Add a case to check oversized images are discarded.
* nova/tests/
Adjust to avoid the fetch size check.
Fixes bug: 1177830
Fixes bug: 1206081
Conflicts:
nova/
nova/
Change-Id: Idc35fce580be4f
Changed in ossa: | |
status: | Fix Committed → Fix Released |
tags: | removed: havana-backport-potential |
Changed in nova: | |
status: | Fix Committed → Fix Released |
tags: | removed: in-stable-havana |
Changed in nova: | |
milestone: | icehouse-1 → 2014.1 |
Russell, Mikal: could you confirm the vulnerability ?