STX: VM boot up failed by 'PciDevice' object has no attribute 'get'

[Peng Peng]

Brief Description
-----------------
Failed to boot up a VM with pci-passthrough interface,

Severity
--------
Major

Steps to Reproduce
------------------
boot up a VM with pci-passthrough interface,

Expected Behavior
------------------
VM Boot up successfully

Actual Behavior
----------------
Boot up failed

Reproducibility
---------------
Reproducible (10/10)

System Configuration
--------------------
Dedicated storage

Branch/Pull Time/Commit
-----------------------
stx.2018.10 release branch build as of 2018-10-08_01-52-01

Timestamp/Logs
--------------
[2018-10-09 10:57:52,859] 262 DEBUG MainThread ssh.send :: Send 'nova --os-username 'tenant2' --os-password 'Li69nux*' --os-project-name tenant2 --os-auth-url http://[abba::2]:5000/v3 --os-user-domain-name Default --os-project-domain-name Default --os-region-name RegionOne boot --flavor fa764368-37bc-4871-b9da-bce16f44bc44 --key-name keypair-tenant2 --boot-volume f8bb1d7b-2b38-461b-8257-8bb0d4e598e3 --nic vif-model=virtio,net-id=0dfc7e0c-3980-461a-9efd-7df8114c768c --nic vif-model=pci-passthrough,net-id=bc7abfcd-fa61-4c76-90f9-27aa6b0018e6 tenant2-pci-passthrough-8 --poll'
[2018-10-09 10:58:23,515] 382 DEBUG MainThread ssh.expect :: Output:
+--------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Property | Value |
+--------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | |
| OS-EXT-STS:power_state | 0 |
| OS-EXT-STS:task_state | scheduling |
| OS-EXT-STS:vm_state | building |
| OS-SRV-USG:launched_at | - |
| OS-SRV-USG:terminated_at | - |
| accessIPv4 | |
| accessIPv6 | |
| adminPass | TqQTHVts6X2A |
| config_drive | |
| created | 2018-10-09T10:57:56Z |
| description | - |
| flavor:disk | 2 |
| flavor:ephemeral | 0 |
| flavor:extra_specs | {"hw:wrs:pci_numa_affinity": "strict", "hw:cpu_policy": "dedicated", "hw:pci_irq_affinity_mask": "1,3", "aggregate_instance_extra_specs:storage": "local_image"} |
| flavor:original_name | dedicated_pci_extras |
| flavor:ram | 1024 |
| flavor:swap | 0 |
| flavor:vcpus | 4 |
| hostId | |
| id | 633d012d-1ae0-4ddf-92d7-a068ee001621 |
| image | Attempt to boot from volume - no image supplied |
| key_name | keypair-tenant2 |
| locked | False |
| metadata | {} |
| name | tenant2-pci-passthrough-8 |
| os-extended-volumes:volumes_attached | [] |
| progress | 0 |
| security_groups | default |
| status | BUILD |
| tags | [] |
| tenant_id | 34781d4047294360927fd6a691a3b982 |
| updated | 2018-10-09T10:57:56Z |
| user_id | 18ced9e400074afd8c7cf73ee4ecae9f |
| wrs-if:nics | |
| wrs-res:pci_devices | |
| wrs-res:topology | node:-, 1024MB, pgsize:4K, vcpus:4 |
| wrs-res:vcpus | [4, 4, 4] |
| wrs-sg:server_group | |
+--------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+

Server building... 0% complete
Server building... 0% complete
Server building... 0% complete
Server building... 0% complete
Server building... 0% complete
Error building server
ERROR (ResourceInErrorState): `Server` resource is in the error state due to 'Exceeded maximum number of retries. Exceeded max scheduling attempts 3 for instance 633d012d-1ae0-4ddf-92d7-a068ee001621. Last exception: 'PciDevice' object has no attribute 'get''.

[Ghada Khalil (gkhalil)]

Based on review by Steve, this issue was introduced in mid-September by the changes made for:
https://storyboard.openstack.org/#!/story/2002877

This breaks the pci-pt configuration. Targeting stx.2018.10

[Steven Webster (swebster-wr)]

fixed in master by: https://github.com/starlingx-staging/stx-nova/pull/10
fixed in r/2018.10 by: https://github.com/starlingx-staging/stx-nova/pull/11