nova boot GPU instance will attach more one GPU pci device when reschedule happened
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenStack Compute (nova) |
In Progress
|
Undecided
|
guolei |
Bug Description
Description
===========
When we boot a GPU instance, on nova-compute's instance_claim
https:/
input instance object's attribute 'pci_devices' had update from [] to [PciDevice], it include a calculated GPU PCI device object.
Ok, now we pay attention to claim's code flow:
https:/
it cloned input instance object, set to self.instance
https:/
abort func will abort instance's claim with self.instance, it a cloned one, not the origin input instance object.
Now, we can see, if spawn instance failed, claim.abort will be called, it revert the cloned instance object's
'pci_devices' attribute to [], and pci_device in db had reverted from allocate to free too. but the origin input instance object not, origin instance object's 'pci_devices' is still [PciDevice], and it will send to nova-conductor to do reschedule, and on next node, after claim, instance.
And then, spawn instance will have two GPU pci device, or raise a LibvirtError, "Device xxx is in used"
Steps to reproduce
==================
1. build libvirt error on all compute nodes
2. nova boot a GPU instance
3. show guest xml in nova-compute.log
Expected result
===============
on reschedule node, guest xml had just one GPU pci device
Actual result
=============
on reschedule node, guest xml had more then one GPU pci device
Changed in nova: | |
assignee: | nobody → guolei (guolei-5) |
Fix proposed to branch: master /review. opendev. org/759544
Review: https:/