[2.1-Build 28] Tor-Agent crash while creating logical ports continually

[Daisuke Nakajima]

While many logical ports without delay by REST/API of Contrail, the contrail-tor-agent was crashed and WebUI showed alart of TOR-agent was partial information available.
The following APIs were done each 8 sec.

--- APIs ---
Neutron API (Port create)
{"port": { "name": "${name}", "network_id": "${network_id}", "admin_state_up": ${admin_state_up}, "mac_address": "${mac_addres}"}}
Contrail API (logical-interface create)
{"logical-interface": { "fq_name": ["default-global-system-config", "${router}", "${parent}", "${name}"], "parent_type": "physical-interface", "parent_uuid": "${physical_interface_uuid}", "name":"${name}", "logical_interface_vlan_tag": ${vlan}, "virtual_machine_interface_refs": [{"to" : ["default-domain", "demo", "${port_fqdn}"]}], "logical_interface_type": "L2"}}

[Daisuke Nakajima (dnakajima)]

The tor-agent is rebooting regularly. According to var/log/syslog, there are many contrail-tor-agent crash logs.
I suppose it is buffer overflow of tor-agent.

Feb 13 09:41:20 system003 kernel: [86392.405023] contrail-tor-ag[30843]: segfault at 94 ip 00000000008e311a sp 00007f4a0ce5a810 error 4 in contrail-tor-agent[400000+bac000]
Feb 13 09:42:04 system003 kernel: [86436.227015] contrail-tor-ag[9781]: segfault at 94 ip 00000000008e311a sp 00007ff30bbfd810 error 4 in contrail-tor-agent[400000+bac000]
Feb 13 09:42:59 system003 kernel: [86490.554476] contrail-tor-ag[21143]: segfault at 94 ip 00000000008e311a sp 00007fb90602f810 error 4 in contrail-tor-agent[400000+bac000]
Feb 13 09:43:27 system003 kernel: [86518.993215] contrail-tor-ag[32506]: segfault at 94 ip 00000000008e311a sp 00007f68b71d5750 error 4 in contrail-tor-agent[400000+bac000]
Feb 13 09:44:39 system003 kernel: [86590.726996] contrail-tor-ag[10916]: segfault at 94 ip 00000000008e311a sp 00007f87d27d3810 error 4 in contrail-tor-agent[400000+bac000]
Feb 13 09:44:41 system003 dhclient: DHCPREQUEST of 172.27.113.87 on em1 to 172.27.115.153 port 67 (xid=0x714ad4a3)
Feb 13 09:44:41 system003 dhclient: DHCPACK of 172.27.113.87 from 172.27.115.153
Feb 13 09:44:41 system003 dhclient: bound to 172.27.113.87 -- renewal in 283 seconds.
Feb 13 09:45:55 system003 kernel: [86666.870521] contrail-tor-ag[22288]: segfault at 94 ip 00000000008e311a sp 00007fd6f95ae810 error 4 in contrail-tor-agent[400000+bac000]
Feb 13 09:46:22 system003 kernel: [86694.179209] contrail-tor-ag[1218]: segfault at 94 ip 00000000008e311a sp 00007f4ad7aaf750 error 4 in contrail-tor-agent[400000+bac000]
Feb 13 09:46:51 system003 kernel: [86722.297055] contrail-tor-ag[12117]: segfault at 94 ip 00000000008e311a sp 00007f527fbf1750 error 4 in contrail-tor-agent[400000+bac000]
Feb 13 09:47:16 system003 kernel: [86748.221803] contrail-tor-ag[22951]: segfault at 94 ip 00000000008e311a sp 00007f1871515750 error 4 in contrail-tor-agent[400000+bac000]
Feb 13 09:48:44 system003 kernel: [86835.723508] contrail-tor-ag[1388]: segfault at 94 ip 00000000008e311a sp 00007f868dad9810 error 4 in contrail-tor-agent[400000+bac000]
Feb 13 09:49:24 system003 dhclient: DHCPREQUEST of 172.27.113.87 on em1 to 172.27.115.153 port 67 (xid=0x714ad4a3)
Feb 13 09:49:24 system003 dhclient: DHCPACK of 172.27.113.87 from 172.27.115.153
Feb 13 09:49:24 system003 dhclient: bound to 172.27.113.87 -- renewal in 232 seconds.
Feb 13 09:49:39 system003 kernel: [86890.207943] contrail-tor-ag[12779]: segfault at 94 ip 00000000008e311a sp 00007f0387571810 error 4 in contrail-tor-agent[400000+bac000]
Feb 13 09:50:22 system003 kernel: [86934.057985] contrail-tor-ag[24155]: segfault at 94 ip 00000000008e311a sp 00007fb33f736810 error 4 in contrail-tor-agent[400000+bac000]
Feb 13 09:50:52 system003 kernel: [86963.482072] contrail-tor-ag[3077]: segfault at 94 ip 00000000008e311a sp 00007f3321fb4750 error 4 in contrail-tor-agent[400000+bac000]
Feb 13 09:51:40 system003 kernel: [87011.610614] contrail-tor-ag[13925]: segfault at 94 ip 00000000008e311a sp 00007f71c09e1810 error 4 in contrail-tor-agent[400000+bac000]
Feb 13 09:53:16 system003 dhclient: DHCPREQUEST of 172.27...

[Hari Prasad Killi (haripk)]

R2.1 fixes to handle this scenario:
https://github.com/Juniper/contrail-controller/commit/179a8f694626a365b3f90bf92ab389b6e5ecca31
https://github.com/Juniper/contrail-controller/commit/b4af4d67cef56ac85da5bb0d50bb55f882c70538
https://github.com/Juniper/contrail-controller/commit/8abc6b9a1071438ddaeff1d2647e6406c48395b4