Instance failed to launch with pci_passthrough:alias

[Senthil Mukundakumar]

Brief Description
-----------------
GPU passthrough Instance launched with pci_passthrough:alias in flavor failed to launch

Severity
--------
Major

Steps to Reproduce
------------------ --------
1. Add extra parameter to flavor to support GPU Passthrough

nova flavor-key flavor_name set pci_passthrough:alias=gpu:1
[wrsroot@controller-0 ~(keystone_admin)]$ nova device-list
+-----------+-----------+-----------+----------+--------------------+--------------+--------------------+--------------+
| PCI Alias | Device Id | Vendor Id | Class Id | pci_pfs_configured | pci_pfs_used | pci_vfs_configured | pci_vfs_used |
+-----------+-----------+-----------+----------+--------------------+--------------+--------------------+--------------+
| gpu | * | * | 030000 | 3 | 0 | 0 | 0 |
+-----------+-----------+-----------+----------+--------------------+--------------+--------------------+--------------+
[wrsroot@controller-0 ~(keystone_admin)]$

2. Launch linux guest using the following image:
download following image centos-67-cloud-gpu.img

Make Glance Image:
glance image-create --property hypervisor_type=QEMU --name "Centos-6.7-Cloud-Matrox-GPU" --container-format bare --disk-format qcow2 --property os_type=linux --file /home/wrsroot/images/centos-67-cloud-gpu.img --progress

reate Keypair:
Reference: https://www.mirantis.com/blog/openstack-security-tip-create-a-keypair-for-accessing-vms/

Create a key pair in Horizon

The first step is to create the actual key pair, if you don’t already have one:

Click “Compute” under the “Project” option in the Horizon left-hand menu.
Select “Access & Security”.
Click the “Key Pairs” tab.

openstack-horizon-access-security

Click “+Create Key Pair”.
Name your new key pair and click “Create Key Pair”.

openstack-create-key-pair

4. Launch VM using the image and flavor created above.

Expected Behavior
------------------
VM expected to be launched and become ACTIVE

Actual Behavior
----------------

Failed to launch
[wrsroot@controller-0 ~(keystone_admin)]$ nova list --a
+--------------------------------------+--------------+----------------------------------+--------+------------+-------------+----------+
| ID | Name | Tenant ID | Status | Task State | Power State | Networks |
+--------------------------------------+--------------+----------------------------------+--------+------------+-------------+----------+
| 943fe930-da47-4596-aaa5-c7cef7cafa0f | gpu instance | 59015934e48941f08f9d812a1ba0e21d | ERROR | - | NOSTATE | |
+--------------------------------------+--------------+----------------------------------+--------+------------+-------------+----------+
[wrsroot@controller-0 ~(keystone_admin)]$ nova show 943fe930-da47-4596-aaa5-c7cef7cafa0f
+--------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Property | Value |
+--------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| OS-DCF:diskConfig | AUTO |
| OS-EXT-AZ:availability_zone | nova |
| OS-EXT-SRV-ATTR:host | - |
| OS-EXT-SRV-ATTR:hostname | gpu-instance |
| OS-EXT-SRV-ATTR:hypervisor_hostname | - |
| OS-EXT-SRV-ATTR:instance_name | instance-00000002 |
| OS-EXT-SRV-ATTR:kernel_id | |
| OS-EXT-SRV-ATTR:launch_index | 0 |
| OS-EXT-SRV-ATTR:ramdisk_id | |
| OS-EXT-SRV-ATTR:reservation_id | r-xcxdv8hv |
| OS-EXT-SRV-ATTR:root_device_name | /dev/vda |
| OS-EXT-SRV-ATTR:user_data | - |
| OS-EXT-STS:power_state | 0 |
| OS-EXT-STS:task_state | - |
| OS-EXT-STS:vm_state | error |
| OS-SRV-USG:launched_at | - |
| OS-SRV-USG:terminated_at | - |
| accessIPv4 | |
| accessIPv6 | |
| config_drive | |
| created | 2018-10-04T20:07:39Z |
| description | - |
| fault | {"message": "No valid host was found. There are not enough hosts available. controller-0: (RetryFilter) Previously tried: [[u'controller-0', u'controller-0']]", "code": 501, "details": " controller-0: (RetryFilter) Previously tried: [[u'controller-0', u'controller-0']]", "created": "2018-10-04T20:08:46Z"} |
| flavor:disk | 20 |
| flavor:ephemeral | 0 |
| flavor:extra_specs | {"hw:cpu_model": "SandyBridge", "hw:cpu_policy": "dedicated", "aggregate_instance_extra_specs:storage": "local_image", "pci_passthrough:alias": "gpu:1", "hw:mem_page_size": "2048"} |
| flavor:original_name | medium.dpdk |
| flavor:ram | 2048 |
| flavor:swap | 0 |
| flavor:vcpus | 2 |
| hostId | |
| host_status | |
| id | 943fe930-da47-4596-aaa5-c7cef7cafa0f |
| image | Attempt to boot from volume - no image supplied |
| key_name | gpu |
| locked | False |
| metadata | {} |
| name | gpu instance |
| os-extended-volumes:volumes_attached | [{"id": "2fa19c96-4916-4e54-9b9f-2788b79e2742", "delete_on_termination": false}] |
| status | ERROR |
| tags | [] |
| tenant_id | 59015934e48941f08f9d812a1ba0e21d |
| updated | 2018-10-04T20:08:51Z |
| user_id | 6580b1dbc803412aa9df2ba0c671fd69 |
| wrs-if:nics | |
| wrs-res:pci_devices | |
| wrs-res:topology | node:0, 2048MB, pgsize:2M, 1s,2c,1t, vcpus:0,1, pcpus:6,8, pol:ded, thr:pre |
| wrs-res:vcpus | [2, 2, 2] |
| wrs-sg:server_group | |
+--------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

Reproducibility
---------------
Reproducible (2/2 times)

System Configuration
--------------------
Any

Branch/Pull Time/Commit
-----------------------
StarlingX master as of 2018-10-03_20-18-00

============================================

[Ghada Khalil (gkhalil)]

Based on review by Steve, this issue was introduced in mid-September by the changes made for:
https://storyboard.openstack.org/#!/story/2002877

This breaks pci device configuration. Targeting stx.2018.10

[Ghada Khalil (gkhalil)]

The duplicate bug is:
fixed in master by: https://github.com/starlingx-staging/stx-nova/pull/10
fixed in r/2018.10 by: https://github.com/starlingx-staging/stx-nova/pull/11