[OSSA 2013-020] Denial of Service in Nova network source security groups (CVE-2013-4185)
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenStack Compute (nova) |
Fix Released
|
High
|
Vish Ishaya | ||
Grizzly |
Fix Released
|
High
|
Vish Ishaya | ||
OpenStack Security Advisory |
Fix Released
|
High
|
Jeremy Stanley |
Bug Description
The code for retrieving the list of ips to make source security groups is very inefficient. It is possible to create a set of rules using source security groups that will lead to an explosion of get_network_info calls.
Example:
# create a security group
nova secgroup-create foo foo
# add source group rules referencing the same group
for i in {1..10}; do
nova secgroup-
done
# launch 10 instances
nova boot --flavor 1 --image precise --num-instances 10 --security-groups foo test
When each instance boots it sends out a notification to each of the other instances to update their iptables rules. Here is what happens when the 10th instance boots:
for num instances in group that is referenced in a source rule: 10
for num rules referencing a source group: 10
for num instances in the source group: 10
call get_nw_info()
That means 1000 individual requests to get_nw_info for a single instance launch. In this case you have You can see how these numbers could get out of hand very quickly, for example launching 20 instances:
instance number(n) : num calls (n^2) * 10
1 : 10
2 : 20
3 : 90
...
18 : 3240
19 : 3610
20 : 4000
...
$ python -c "print sum(n * n * 10 for n in xrange(1, 21))"
28700
28,700 calls to get_network info by the time the launch completes. In reality this number of calls creates a DOS where nova-network can't respond in time and instance launches start to timeout.
There are multiple fixes needed:
a) constructing a list of all needed instances instead of making a separate request for each rule.
b) a single call to retrieve nw_info (or at least fixed ip info) for multiple instances at once.
CVE References
Changed in ossa: | |
assignee: | nobody → Michael Still (mikalstill) |
Changed in ossa: | |
status: | Incomplete → Confirmed |
assignee: | Michael Still (mikalstill) → Jeremy Stanley (fungi) |
Changed in ossa: | |
importance: | Undecided → Medium |
Changed in nova: | |
status: | Triaged → In Progress |
information type: | Private Security → Public Security |
Changed in ossa: | |
status: | Triaged → In Progress |
summary: |
- Potential Denial of Service using source_security_groups + Denial of Service in Nova network source security groups (CVE-2013-4185) |
tags: | removed: in-stable-folsom in-stable-grizzly |
Changed in ossa: | |
status: | In Progress → Fix Committed |
summary: |
- Denial of Service in Nova network source security groups (CVE-2013-4185) + [OSSA 2013-020] Denial of Service in Nova network source security groups + (CVE-2013-4185) |
Changed in ossa: | |
status: | Fix Committed → Fix Released |
Changed in nova: | |
milestone: | none → havana-3 |
status: | Fix Committed → Fix Released |
Changed in nova: | |
milestone: | havana-3 → 2013.2 |
no longer affects: | nova/folsom |
Not totally convinced we should issue an OSSA about this one... It's a bit of a slippery slope since authenticated users have a lot of ways to consume resources . Only wildly assymetric attacks should trigger OSSAs, but that may well be the case here.
That only affects Nova when used with nova-network (not Quantum), right ?