unshelve leak allocation if update port fails
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenStack Compute (nova) |
Fix Released
|
Medium
|
Balazs Gibizer | ||
Pike |
Fix Released
|
Undecided
|
Unassigned | ||
Queens |
Fix Released
|
Undecided
|
Unassigned | ||
Rocky |
Fix Released
|
Undecided
|
Unassigned | ||
Stein |
Fix Released
|
Undecided
|
Unassigned | ||
Train |
Fix Released
|
Undecided
|
Unassigned |
Bug Description
If updating the port binding during unshelve of an offloaded server fails then nova leaks placement allocation.
Steps to reproduce
==================
1) boot a server with a neutron port
2) shelve and offload the server
3) disable the original host of the server to force scheduling during unshelve to select a differetn host. This is important as it triggers a non empty port update during unshelve
4) unshelve the server and inject network fault in the communication between nova and neutron. You can try to simply shut down neutron-server at the right moment as well. Right means just before the target compute tries to send the port update
5) observer that the unshelve fails, the server goes back to offloaded state, but the placement allocation on the target host remains.
Triage: the problem is cause by a missing fault handling code in the compute manager[1]. The compute manager has proper error handling if the unshelve fails in the virt driver spawn call, but it does not handle failure if the neutron communication fails. The compute manager method simply logs and re-raises the neutron exceptions. This means that the exception is dropped as the unshelve_instance compute RPC is a cast.
Changed in nova: | |
status: | New → Triaged |
importance: | Undecided → Medium |
assignee: | nobody → Balazs Gibizer (balazs-gibizer) |
tags: | added: shelve |
Fix proposed to branch: master /review. opendev. org/706867
Review: https:/