[R3.2 vcenter_only]Ports getting deleted on restart of vcenter_plugin

[aswani kumar]

Seeing on
R3.2 full build 67 and cb-build 116 onwards
R4.1 cb-build 94

On restarting vcenter plugin, ports are getting deleted.
Attached the vcenter_plugin log at that particular time frame

Vcenter_log
1c, primary vlan = 154, isolated vlan = 155>]
[2018-02-15 10:55:04,957] [INFO ] [Thread-3:ApiConnectorImpl@231][] [>> Request: GET, /virtual-machine/50242087-37ab-4a87-8a3c-4f3c864c8e86]
[2018-02-15 10:55:04,961] [INFO ] [Thread-3:ApiConnectorImpl@263][] [<< Response Status: HTTP/1.1 200 OK]
[2018-02-15 10:55:04,963] [INFO ] [Thread-3:ApiConnectorImpl@231][] [>> Request: GET, /virtual-machine/50242087-37ab-4a87-8a3c-4f3c864c8e86]
[2018-02-15 10:55:04,967] [INFO ] [Thread-3:ApiConnectorImpl@263][] [<< Response Status: HTTP/1.1 200 OK]
[2018-02-15 10:55:04,968] [INFO ] [Thread-3:ApiConnectorImpl@231][] [>> Request: DELETE, /virtual-machine/50242087-37ab-4a87-8a3c-4f3c864c8e86]
[2018-02-15 10:55:05,022] [INFO ] [Thread-3:ApiConnectorImpl@263][] [<< Response Status: HTTP/1.1 200 OK]
[2018-02-15 10:55:05,022] [INFO ] [Thread-3:VncDB@626][] [Deleted VM <10.204.216.183, host null, 50242087-37ab-4a87-8a3c-4f3c864c8e86>]
[2018-02-15 10:55:05,023] [INFO ] [Thread-3:ApiConnectorImpl@231][] [>> Request: GET, /virtual-machine-interface/c33745b8-3c4e-4a77-b5e1-b55b74ee8892]
[2018-02-15 10:55:05,026] [INFO ] [Thread-3:ApiConnectorImpl@263][] [<< Response Status: HTTP/1.1 200 OK]
[2018-02-15 10:55:05,029] [INFO ] [Thread-3:VncDB@1137][] [Delete instance IP: [ab3e4ef0-a75e-47ef-92c0-b8e21cb77ecc]]
[2018-02-15 10:55:05,029] [INFO ] [Thread-3:ApiConnectorImpl@231][] [>> Request: GET, /instance-ip/ab3e4ef0-a75e-47ef-92c0-b8e21cb77ecc]
[2018-02-15 10:55:05,039] [INFO ] [Thread-3:ApiConnectorImpl@263][] [<< Response Status: HTTP/1.1 200 OK]
[2018-02-15 10:55:05,041] [INFO ] [Thread-3:ApiConnectorImpl@231][] [>> Request: DELETE, /instance-ip/ab3e4ef0-a75e-47ef-92c0-b8e21cb77ecc]
[2018-02-15 10:55:05,131] [INFO ] [Thread-3:ApiConnectorImpl@263][] [<< Response Status: HTTP/1.1 200 OK]
[2018-02-15 10:55:05,131] [INFO ] [Thread-3:VncDB@1141][] [Deleted Ip Instance ab3e4ef0-a75e-47ef-92c0-b8e21cb77ecc]
[2018-02-15 10:55:05,131] [INFO ] [Thread-3:ApiConnectorImpl@231][] [>> Request: GET, /virtual-machine-interface/c33745b8-3c4e-4a77-b5e1-b55b74ee8892]
[2018-02-15 10:55:05,136] [INFO ] [Thread-3:ApiConnectorImpl@263][] [<< Response Status: HTTP/1.1 200 OK]
[2018-02-15 10:55:05,139] [INFO ] [Thread-3:VncDB@765][] [SecurityGroup association exists for VMInterface:c33745b8-3c4e-4a77-b5e1-b55b74ee8892]
[2018-02-15 10:55:05,139] [INFO ] [Thread-3:ApiConnectorImpl@231][] [>> Request: GET, /security-group/2499a5b0-2d74-4ac2-9503-30d94714767d]
[2018-02-15 10:55:05,145] [INFO ] [Thread-3:ApiConnectorImpl@263][] [<< Response Status: HTTP/1.1 200 OK]
[2018-02-15 10:55:05,151] [INFO ] [Thread-3:ApiConnectorImpl@228][] [>> Request: PUT, /virtual-machine-interface/c33745b8-3c4e-4a77-b5e1-b55b74ee8892, {"virtual-machine-interface":{"port_security_enabled":false,"virtual_machine_interface_disable_policy":false,"security_group_refs":[],"name":"c33745b8-3c4e-4a77-b5e1-b55b74ee8892","uuid":"c33745b8-3c4e-4a77-b5e1-b55b74ee8892","fq_name":["default-virtual-machine","c33745b8-3c4e-4a77-b5e1-b55b74ee8892"],"parent_type":"virtual-machine"}}]
[2018-02-15 10:55:05,172] [INFO ] [Thread-3:ApiConnectorImpl@263][] [<< Response Status: HTTP/1.1 200 OK]
[2018-02-15 10:55:05,172] [INFO ] [Thread-3:VncDB@779][] [Removed SecurityGroup association for VMInterface:c33745b8-3c4e-4a77-b5e1-b55b74ee8892]
[2018-02-15 10:55:05,173] [INFO ] [Thread-3:ApiConnectorImpl@231][] [>> Request: GET, /virtual-machine-interface/c33745b8-3c4e-4a77-b5e1-b55b74ee8892]
[2018-02-15 10:55:05,186] [INFO ] [Thread-3:ApiConnectorImpl@263][] [<< Response Status: HTTP/1.1 200 OK]
[2018-02-15 10:55:05,189] [INFO ] [Thread-3:ApiConnectorImpl@231][] [>> Request: DELETE, /virtual-machine-interface/c33745b8-3c4e-4a77-b5e1-b55b74ee8892]
[2018-02-15 10:55:05,238] [INFO ] [Thread-3:ApiConnectorImpl@263][] [<< Response Status: HTTP/1.1 200 OK]
[2018-02-15 10:55:05,238] [INFO ] [Thread-3:VncDB@734][] [Deleted VMI <10.204.216.181, ctest-vn40-97150023, c33745b8-3c4e-4a77-b5e1-b55b74ee8892, 87.122.41.4, 00:50:56:a4:82:d7, primary vlan = 100, isolated vlan = 101>]

Test log before restarting

Ping beteween vms was going fine

2018-02-15 10:54:32,244 - DEBUG - ping -s 56 -c 5 -W 1 87.122.41.4
2018-02-15 10:54:37,966 - DEBUG - sudo: unable to resolve host ubuntu^M
PING 87.122.41.4 (87.122.41.4) 56(84) bytes of data.^M
64 bytes from 87.122.41.4: icmp_req=1 ttl=64 time=1.84 ms^M
64 bytes from 87.122.41.4: icmp_req=2 ttl=64 time=0.939 ms^M
64 bytes from 87.122.41.4: icmp_req=3 ttl=64 time=1.17 ms^M
64 bytes from 87.122.41.4: icmp_req=4 ttl=64 time=1.17 ms^M
64 bytes from 87.122.41.4: icmp_req=5 ttl=64 time=1.03 ms^M
^M
--- 87.122.41.4 ping statistics ---^M
5 packets transmitted, 5 received, 0% packet loss, time 4005ms^M
rtt min/avg/max/mdev = 0.939/1.230/1.840/0.318 ms
2018-02-15 10:54:37,966 - INFO - Ping to IP 87.122.41.4 from VM ctest-vm1_plugin-32404622 passed
2018-02-15 10:54:37,999 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 5 -W 1 87.122.41.3, host_string: ubuntu@169.254.0.4, password: ubuntugateway: root@10.204.216.181, gateway password: c0ntrail123
2018-02-15 10:54:37,999 - DEBUG - ping -s 56 -c 5 -W 1 87.122.41.3
2018-02-15 10:54:42,995 - DEBUG - sudo: unable to resolve host ubuntu^M
PING 87.122.41.3 (87.122.41.3) 56(84) bytes of data.^M
64 bytes from 87.122.41.3: icmp_req=1 ttl=64 time=1.47 ms^M
64 bytes from 87.122.41.3: icmp_req=2 ttl=64 time=1.00 ms^M
64 bytes from 87.122.41.3: icmp_req=3 ttl=64 time=1.05 ms^M
64 bytes from 87.122.41.3: icmp_req=4 ttl=64 time=1.00 ms^M
64 bytes from 87.122.41.3: icmp_req=5 ttl=64 time=1.15 ms^M
^M
--- 87.122.41.3 ping statistics ---^M
5 packets transmitted, 5 received, 0% packet loss, time 4005ms^M
rtt min/avg/max/mdev = 1.006/1.139/1.479/0.183 ms

Thanks,
Aswani Kumar

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/40122
Submitter: amudha ramachandran (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/40122
Committed: http://github.com/Juniper/contrail-vcenter-plugin/commit/4249d2330242b7591cb6cd3b435bf9f8736f0953
Submitter: Zuul (<email address hidden>)
Branch: R3.2

commit 4249d2330242b7591cb6cd3b435bf9f8736f0953
Author: Amudha <email address hidden>
Date: Mon Feb 26 21:02:18 2018 -0800

vCenter-only: Fix to scan VMI

Reverting readVirtualMachineInterfaces related change from below commit
https://github.com/Juniper/contrail-vcenter-plugin/commit/2639f8535453982841a9f1d829dc10dee19ee826

Skipping for unmanaged VMI is taken care of in readVirtualMachineInterfaces().
So, call readVirtualMachineInterfaces before ignoring VM read.

Change-Id: Ia2023d53590cdda99376d9eab36399dd52826c0e
Closes-Bug: #1749905

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/40871
Submitter: amudha ramachandran (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/40872
Submitter: amudha ramachandran (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/40873
Submitter: amudha ramachandran (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/40873
Committed: http://github.com/Juniper/contrail-vcenter-plugin/commit/694fa444a1ccf3a4ab63c5ce335404eeef83cdfe
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit 694fa444a1ccf3a4ab63c5ce335404eeef83cdfe
Author: Amudha <email address hidden>
Date: Mon Feb 26 21:02:18 2018 -0800

vCenter-only: Fix to scan VMI

Reverting readVirtualMachineInterfaces related change from below commit
https://github.com/Juniper/contrail-vcenter-plugin/commit/2639f8535453982841a9f1d829dc10dee19ee826

Skipping for unmanaged VMI is taken care of in readVirtualMachineInterfaces().
So, call readVirtualMachineInterfaces before ignoring VM read.

Change-Id: Ia2023d53590cdda99376d9eab36399dd52826c0e
Closes-Bug: #1749905
(cherry picked from commit 4249d2330242b7591cb6cd3b435bf9f8736f0953)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/40872
Committed: http://github.com/Juniper/contrail-vcenter-plugin/commit/109b6aa193e562d8cdd0f9649cc77a3b618d9652
Submitter: Zuul (<email address hidden>)
Branch: R4.1

commit 109b6aa193e562d8cdd0f9649cc77a3b618d9652
Author: Amudha <email address hidden>
Date: Mon Feb 26 21:02:18 2018 -0800

vCenter-only: Fix to scan VMI

Reverting readVirtualMachineInterfaces related change from below commit
https://github.com/Juniper/contrail-vcenter-plugin/commit/2639f8535453982841a9f1d829dc10dee19ee826

Skipping for unmanaged VMI is taken care of in readVirtualMachineInterfaces().
So, call readVirtualMachineInterfaces before ignoring VM read.

Change-Id: Ia2023d53590cdda99376d9eab36399dd52826c0e
Closes-Bug: #1749905
(cherry picked from commit 4249d2330242b7591cb6cd3b435bf9f8736f0953)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/40871
Committed: http://github.com/Juniper/contrail-vcenter-plugin/commit/647783e0b4a35c68f160be5e3174f8a9bf3d0eb8
Submitter: Zuul (<email address hidden>)
Branch: R4.0

commit 647783e0b4a35c68f160be5e3174f8a9bf3d0eb8
Author: Amudha <email address hidden>
Date: Mon Feb 26 21:02:18 2018 -0800

vCenter-only: Fix to scan VMI

Reverting readVirtualMachineInterfaces related change from below commit
https://github.com/Juniper/contrail-vcenter-plugin/commit/2639f8535453982841a9f1d829dc10dee19ee826

Skipping for unmanaged VMI is taken care of in readVirtualMachineInterfaces().
So, call readVirtualMachineInterfaces before ignoring VM read.

Change-Id: Ia2023d53590cdda99376d9eab36399dd52826c0e
Closes-Bug: #1749905
(cherry picked from commit 4249d2330242b7591cb6cd3b435bf9f8736f0953)