Comment 19 for bug 1838676

I have installed the same HW lab with a recent load 2019-12-01_20-00-00 and run the same automated testcase. Now suspect new behavior related to LP-1841189

The instance survived several live migrations but now fails ping test after iteration8
(originating host was compute-1 and target host was compute-6)

...
[2019-12-04 19:36:42,818] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter8
[2019-12-04 19:36:46,801] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 358526fa-0f85-45e9-b246-63be77384e3d from compute-1 started.
[2019-12-04 19:36:46,802] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 358526fa-0f85-45e9-b246-63be77384e3d
[2019-12-04 19:36:48,847] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort
[2019-12-04 19:36:51,184] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING
[2019-12-04 19:36:51,184] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE
[2019-12-04 19:37:11,844] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 358526fa-0f85-45e9-b246-63be77384e3d completed
[2019-12-04 19:37:13,951] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 358526fa-0f85-45e9-b246-63be77384e3d successfully migrated from compute-1 to compute-6
[2019-12-04 19:37:13,951] 1484 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client...
[2019-12-04 19:37:16,074] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'358526fa-0f85-45e9-b246-63be77384e3d': ['192.168.131.50']}
[2019-12-04 19:37:16,074] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.131.50 from host 128.224.186.181
[2019-12-04 19:37:28,281] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds
[2019-12-04 19:37:31,284] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.131.50 failed.
[2019-12-04 19:37:31,284] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.131.50': 100}

Name tenant1-volume-stress-1
ID 358526fa-0f85-45e9-b246-63be77384e3d
    tenant1-net10 172.16.10.135
    tenant1-mgmt-net 192.168.131.50

tenant1-volume-stress-1:~# ip add
[ 3752.763497] -bash[1597]: HISTORY: PID=1597 UID=0 ip add
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether fa:16:3e:77:6e:55 brd ff:ff:ff:ff:ff:ff
    inet 192.168.131.50/27 brd 192.168.131.63 scope global dynamic eth0
       valid_lft 82655sec preferred_lft 82655sec
    inet6 fe80::f816:3eff:fe77:6e55/64 scope link
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether fa:16:3e:1a:98:e7 brd ff:ff:ff:ff:ff:ff
    inet 172.16.10.135/24 brd 172.16.10.255 scope global dynamic eth1
       valid_lft 82661sec preferred_lft 82661sec
    inet6 fe80::f816:3eff:fe1a:98e7/64 scope link
       valid_lft forever preferred_lft forever

(20b14c7e-0453) 192.168.131.1 Active Internal Interface UP
(3006e0df-0a95) 192.168.131.65 Active Internal Interface UP
(69f7186e-f008) 192.168.131.33 Active Internal Interface UP
(72149874-f1fe) 192.168.31.2 Active External Gateway UP

Pings failures from the vm
tenant1-volume-stress-1:~# ping 192.168.131.1
[ 3770.922772] -bash[1597]: HISTORY: PID=1597 UID=0 ping 192.168.131.1
PING 192.168.131.1 (192.168.131.1) 56(84) bytes of data.
From 192.168.131.50 icmp_seq=1 Destination Host Unreachable
From 192.168.131.50 icmp_seq=2 Destination Host Unreachable
From 192.168.131.50 icmp_seq=3 Destination Host Unreachable
From 192.168.131.50 icmp_seq=4 Destination Host Unreachable

--- 192.168.131.1 ping statistics ---
5 packets transmitted, 0 received, +4 errors, 100% packet loss, time 4001ms
pipe 4
tenant1-volume-stress-1:~# ping 192.168.131.2
[ 3777.060447] -bash[1597]: HISTORY: PID=1597 UID=0 ping 192.168.131.2
PING 192.168.131.2 (192.168.131.2) 56(84) bytes of data.
From 192.168.131.50 icmp_seq=1 Destination Host Unreachable
From 192.168.131.50 icmp_seq=2 Destination Host Unreachable
From 192.168.131.50 icmp_seq=3 Destination Host Unreachable
From 192.168.131.50 icmp_seq=4 Destination Host Unreachable
From 192.168.131.50 icmp_seq=5 Destination Host Unreachable
From 192.168.131.50 icmp_seq=6 Destination Host Unreachable
From 192.168.131.50 icmp_seq=7 Destination Host Unreachable
From 192.168.131.50 icmp_seq=8 Destination Host Unreachable
From 192.168.131.50 icmp_seq=9 Destination Host Unreachable
From 192.168.131.50 icmp_seq=10 Destination Host Unreachable
From 192.168.131.50 icmp_seq=11 Destination Host Unreachable
From 192.168.131.50 icmp_seq=12 Destination Host Unreachable
--- 192.168.131.2 ping statistics ---
13 packets transmitted, 0 received, +12 errors, 100% packet loss, time 12003ms
pipe 4

tenant1-volume-stress-1:~# ping 192.168.31.2
[ 3791.826266] -bash[1597]: HISTORY: PID=1597 UID=0 ping 192.168.31.2
PING 192.168.31.2 (192.168.31.2) 56(84) bytes of data.
From 192.168.131.50 icmp_seq=1 Destination Host Unreachable
From 192.168.131.50 icmp_seq=2 Destination Host Unreachable
From 192.168.131.50 icmp_seq=3 Destination Host Unreachable
From 192.168.131.50 icmp_seq=4 Destination Host Unreachable
From 192.168.131.50 icmp_seq=5 Destination Host Unreachable
From 192.168.131.50 icmp_seq=6 Destination Host Unreachable
From 192.168.131.50 icmp_seq=7 Destination Host Unreachable
From 192.168.131.50 icmp_seq=8 Destination Host Unreachable

--- 192.168.31.2 ping statistics ---
9 packets transmitted, 0 received, +8 errors, 100% packet loss, time 8003ms
pipe 4