STX: pci-passthrough interface VM not reachable via internal net

[Peng Peng]

Brief Description
-----------------
Boot a vm with pci-passthrough vif model on internal net, then Add vlan to pci-passthrough interface for VM; VM could reach via mgmet net, but failed to ping VM over internal net

Severity
--------
Major

Steps to Reproduce
------------------
1. Boot a vm with pci-passthrough vif model on internal net
2. Add vlan to pci-passthrough interface for VM
3. Ping vm over internal nets from base vm

Expected Behavior
------------------
Ping success

Actual Behavior
----------------
Ping failed

Reproducibility
---------------
Reproducible (10/10)

System Configuration
--------------------
Dedicated storage

Branch/Pull Time/Commit
-----------------------
master as of 2018-10-14_20-18-00

Timestamp/Logs
--------------
[2018-10-15 14:35:34,413] 262 DEBUG MainThread ssh.send :: Send 'nova --os-username 'tenant2' --os-password 'Li69nux*' --os-project-name tenant2 --os-auth-url http://[abba::2]:5000/v3 --os-user-domain-name Default --os-project-domain-name Default --os-region-name RegionOne boot --key-name keypair-tenant2 --flavor 9083b707-ddd1-4e40-8013-a50821b82af7 --boot-volume 180f46ae-93a0-4c9e-a902-97861076183e --nic net-id=b5f625f5-21b0-47ed-97fe-44a90d693004,vif-model=virtio --nic net-id=184f47d4-7248-4a16-8a04-aa415e3fdc71,vif-model=pci-passthrough tenant2-pci-passthrough-2 --poll'

[2018-10-15 14:35:57,840] 262 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://[abba::2]:5000/v3 --os-user-domain-name Default --os-project-domain-name Default --os-region-name RegionOne list --all-tenants'
[2018-10-15 14:35:59,906] 382 DEBUG MainThread ssh.expect :: Output:
+--------------------------------------+---------------------------+----------------------------------+--------+------------+-------------+-------------------------------------------------------------+
| ID | Name | Tenant ID | Status | Task State | Power State | Networks |
+--------------------------------------+---------------------------+----------------------------------+--------+------------+-------------+-------------------------------------------------------------+
| b49da9b8-d3fa-4eff-96e6-e2abffbba549 | tenant2-pci-passthrough-2 | fe4087b77d0448ef9d8d5e11a1a0f07c | ACTIVE | - | Running | internal0-net0-2=10.2.2.136; tenant2-mgmt-net=192.168.231.3 |
| f8eca1ff-540f-4f00-a4a4-216091398656 | tenant2-vm-1 | fe4087b77d0448ef9d8d5e11a1a0f07c | ACTIVE | - | Running | internal0-net0-2=10.2.2.137; tenant2-mgmt-net=192.168.231.6 |
+--------------------------------------+---------------------------+----------------------------------+--------+------------+-------------+-------------------------------------------------------------+
[wrsroot@controller-0 ~(keystone_admin)]$

[2018-10-15 14:38:22,090] 262 DEBUG MainThread ssh.send :: Send 'ping -c 5 10.2.2.136'
[2018-10-15 14:38:36,200] 382 DEBUG MainThread ssh.expect :: Output:
PING 10.2.2.136 (10.2.2.136) 56(84) bytes of data.

--- 10.2.2.136 ping statistics ---
5 packets transmitted, 0 received, 100% packet loss, time 3999ms

]0;root@tenant2-pci-passthrough-2:~tenant2-pci-passthrough-2:~#
[2018-10-15 14:38:36,200] 262 DEBUG MainThread ssh.send :: Send 'echo $?'
[2018-10-15 14:38:36,303] 382 DEBUG MainThread ssh.expect :: Output:
1
]0;root@tenant2-pci-passthrough-2:~tenant2-pci-passthrough-2:~#
[2018-10-15 14:38:36,303] 2631 INFO MainThread network_helper.ping_server:: retry in 3 seconds
[2018-10-15 14:38:39,306] 2638 WARNING MainThread network_helper.ping_server:: Ping from 192.168.231.3 to 10.2.2.136 failed.
[2018-10-15 14:38:39,307] 1536 INFO MainThread vm_helper._ping_vms:: Retry in 3 seconds
[2018-10-15 14:38:42,310] 419 DEBUG MainThread ssh.exec_cmd:: Executing command...
[2018-10-15 14:38:42,310] 262 DEBUG MainThread ssh.send :: Send 'ping -c 5 192.168.231.3'
[2018-10-15 14:38:46,416] 382 DEBUG MainThread ssh.expect :: Output:
PING 192.168.231.3 (192.168.231.3) 56(84) bytes of data.
64 bytes from 192.168.231.3: icmp_seq=1 ttl=64 time=0.017 ms
64 bytes from 192.168.231.3: icmp_seq=2 ttl=64 time=0.028 ms
64 bytes from 192.168.231.3: icmp_seq=3 ttl=64 time=0.016 ms
64 bytes from 192.168.231.3: icmp_seq=4 ttl=64 time=0.028 ms
64 bytes from 192.168.231.3: icmp_seq=5 ttl=64 time=0.018 ms

--- 192.168.231.3 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 3999ms
rtt min/avg/max/mdev = 0.016/0.021/0.028/0.006 ms

[Ghada Khalil (gkhalil)]

This issue was reported previously in the following launchpad:
https://bugs.launchpad.net/starlingx/+bug/1790174
https://bugs.launchpad.net/starlingx/+bug/1790191

The investigation indicated that this is an issue in the way this system is setup to run the test. It's not a software issue.