Race conditions with builds and deletes in Ironic driver
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Ironic |
Invalid
|
Medium
|
Unassigned | ||
OpenStack Compute (nova) |
Expired
|
Undecided
|
Unassigned |
Bug Description
While perhaps not officially supported by nova, when running 2 nova-computes for Ironic, you can get a delete to happen at the same time as a build. The ironic virt driver for nova skips unprovision when the node is not in certain states... It happens that unprovision is skipped when a Node is still deploying... Nova then tries to unset the instance_uuid and ends up failing and retrying on a 409 until ironic finishes the build. It then succeeds and you end up with a Node in ironic that is 'active' but has no instance_uuid.
In addition to this, there's no provision_state checking in the virt driver when reporting resources to the compute manager. There's checking on if instance_uuid is assigned.. and if there's not one, it assumes the node is free. In this case, it turns out the node is not really free... but somewhat orphaned. Scheduling should probably skip these if they happen.
Changed in ironic: | |
status: | New → In Progress |
assignee: | nobody → Chris Behrens (cbehrens) |
Changed in ironic: | |
importance: | Undecided → Medium |
tags: | added: nova-driver |
Changed in nova: | |
status: | New → Confirmed |
importance: | Undecided → Medium |
Changed in ironic: | |
assignee: | Jim Rollenhagen (jim-rollenhagen) → nobody |
Fix proposed to branch: master /review. openstack. org/104649
Review: https:/