GPU device is missing "vendor_id" and "product_id" in the alias, so nova instance ended up with a device from a random passthrough device, in my case, on a compute host with both Crypto and GPU devices, even though the nova flavor specifies "pci_passthrough:alias"="gpu:1", eventually the Crypto VF got used.
My expectation is the whole gpu pf should have been passed to nova instance.
# Here are the devices on the compute:
[sysadmin@controller-0 ~(keystone_admin)]$ system host-device-list compute-1
+------------------+--------------+----------+-----------+-----------+---------------------------+---------------------------------+----------------------------------------+-----------+---------+
| name | address | class id | vendor id | device id | class name | vendor name | device name | numa_node | enabled |
+------------------+--------------+----------+-----------+-----------+---------------------------+---------------------------------+----------------------------------------+-----------+---------+
| pci_0000_0b_00_0 | 0000:0b:00.0 | 0b4000 | 8086 | 0435 | Co-processor | Intel Corporation | DH895XCC Series QAT | 0 | True |
| pci_0000_0f_00_0 | 0000:0f:00.0 | 030000 | 102b | 0522 | VGA compatible controller | Matrox Electronics Systems Ltd. | MGA G200e [Pilot] ServerEngines (SEP1) | 0 | True |
+------------------+--------------+----------+-----------+-----------+---------------------------+---------------------------------+----------------------------------------+-----------+---------+
GPU device is missing "vendor_id" and "product_id" in the alias, so nova instance ended up with a device from a random passthrough device, in my case, on a compute host with both Crypto and GPU devices, even though the nova flavor specifies "pci_passthroug h:alias" ="gpu:1" , eventually the Crypto VF got used.
My expectation is the whole gpu pf should have been passed to nova instance.
# Here are the devices on the compute: controller- 0 ~(keystone_admin)]$ system host-device-list compute-1 ------- -----+- ------- ------+ ------- ---+--- ------- -+----- ------+ ------- ------- ------- ------+ ------- ------- ------- ------- -----+- ------- ------- ------- ------- ------- ----+-- ------- --+---- -----+ ------- -----+- ------- ------+ ------- ---+--- ------- -+----- ------+ ------- ------- ------- ------+ ------- ------- ------- ------- -----+- ------- ------- ------- ------- ------- ----+-- ------- --+---- -----+ ------- -----+- ------- ------+ ------- ---+--- ------- -+----- ------+ ------- ------- ------- ------+ ------- ------- ------- ------- -----+- ------- ------- ------- ------- ------- ----+-- ------- --+---- -----+
[sysadmin@
+------
| name | address | class id | vendor id | device id | class name | vendor name | device name | numa_node | enabled |
+------
| pci_0000_0b_00_0 | 0000:0b:00.0 | 0b4000 | 8086 | 0435 | Co-processor | Intel Corporation | DH895XCC Series QAT | 0 | True |
| pci_0000_0f_00_0 | 0000:0f:00.0 | 030000 | 102b | 0522 | VGA compatible controller | Matrox Electronics Systems Ltd. | MGA G200e [Pilot] ServerEngines (SEP1) | 0 | True |
+------
# flavor extra specs: h:alias" : "gpu:1"} |
| extra_specs | {"hw:cpu_policy": "dedicated", "hw:mem_page_size": "2048", "pci_passthroug
# nova conf for passthrough devices: compute- 1-532206f8- lvn77 -it -- grep -E "whitelist|alias" /etc/nova/nova.conf whitelist = {"address": "0000:0b:00.0"} whitelist = {"address": "0000:0f:00.0"}
controller-0:~$ kubectl exec -n openstack nova-compute-
alias = {"vendor_id": "8086", "product_id": "0435", "name": "qat-dh895xcc-pf"}
alias = {"vendor_id": "8086", "product_id": "0443", "name": "qat-dh895xcc-vf"}
alias = {"vendor_id": "8086", "product_id": "37c8", "name": "qat-c62x-pf"}
alias = {"vendor_id": "8086", "product_id": "37c9", "name": "qat-c62x-vf"}
alias = {"name": "gpu"}
passthrough_
passthrough_
# from VM:
localhost:~# lspci -nn | grep -E "0443|0522"
00:05.0 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT Virtual Function [8086:0443]