All nova apis relying on Instance.save(expected_*_state) for safety contain a race condition
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenStack Compute (nova) |
Fix Released
|
High
|
Matthew Booth |
Bug Description
Take, for example, resize_instance(). In manager.py, we assert that the instance is in RESIZE_PREP state with:
instance.
This should mean that the first resize will succeed, and any subsequent will fail. However, the underlying db implementation does not lock the instance during the update, and therefore doesn't guarantee this.
Specifically, _instance_update() in db/sqlalchemy/
SQLAlchemy seems to support select ... for update, as do MySQL and PostgreSQL, although MySQL will fall back to whole table locks for non-InnoDB tables, which would likely be a significant performance hit.
tags: | added: db |
tags: | added: compute |
Changed in nova: | |
importance: | Undecided → High |
status: | New → Confirmed |
Changed in nova: | |
assignee: | Jay Pipes (jaypipes) → Matthew Booth (mbooth-9) |
Changed in nova: | |
milestone: | none → liberty-2 |
status: | Fix Committed → Fix Released |
Changed in nova: | |
milestone: | none → liberty-3 |
status: | Fix Committed → Fix Released |
Changed in nova: | |
milestone: | liberty-3 → 12.0.0 |
https:/ /bugs.launchpad .net/nova/ +bug/1268569
looks like this was also caused by multiple API call concurrently