Activity log for bug #1484788

Date Who What changed Old value New value Message
2015-08-14 05:22:55 Anoop Kumar Sahu bug added bug
2015-08-14 05:22:55 Anoop Kumar Sahu attachment added HTTP Introspect.html https://bugs.launchpad.net/bugs/1484788/+attachment/4444537/+files/HTTP%20Introspect.html
2015-08-14 05:23:19 Anoop Kumar Sahu juniperopenstack: importance Undecided Critical
2015-08-14 05:26:37 Anoop Kumar Sahu description On a HA setup(3-CN/3-TSN), Step 1: Rebooted CN-1. waited it to come online and active Step 2: Rebooted CN-2. Waited it to come online and active Step 3: Rebooted CN-3. Waited it to com online and active After effect of above 3 steps lead to all the configs were deleted on the QFX side by the Controller. It has been almost 5-6 hours and still the configs are not pushed to the QFX TORs with tor agent active and all SSL connection intact . Please refer to the tor-agent-1 which has around 3K VNs. Attached is the http introspect page http://10.94.63.103:9010/Snh_SandeshTraceRequest?x=Ovsdb%20Pkt root@Host1-CN1:/var/crashes# contrail-status vRouter is NOT PRESENT == Contrail vRouter == supervisor-vrouter: inactive (disabled on boot) unix:///tmp/supervisord_vrouter.sockno == Contrail Control == supervisor-control: active contrail-control active contrail-control-nodemgr active contrail-dns active contrail-named active == Contrail Analytics == supervisor-analytics: active contrail-analytics-api active contrail-analytics-nodemgr active contrail-collector active contrail-query-engine active contrail-snmp-collector active contrail-topology active == Contrail Config == supervisor-config: active contrail-api:0 active contrail-config-nodemgr active contrail-device-manager active contrail-discovery:0 active contrail-schema backup contrail-svc-monitor active ifmap active == Contrail Web UI == supervisor-webui: active contrail-webui active contrail-webui-middleware active == Contrail Support Services == supervisor-support-service: active rabbitmq-server active ========Run time service failures============= /var/crashes/core.contrail-contro.2267.Host1-CN1.1439507792 root@Host5-TSN2:~# contrail-status == Contrail vRouter == supervisor-vrouter: active contrail-tor-agent-1 active contrail-tor-agent-2 initializing (ToR:vdc-vcf-s2 connection down) contrail-tor-agent-3 initializing (ToR:vdc-vcf-s4 connection down) contrail-tor-agent-4 initializing (ToR:vdc-vcf-l1 connection down) contrail-tor-agent-5 initializing (ToR:vdc-vcf-l2 connection down) contrail-tor-agent-6 active contrail-tor-agent-7 initializing (ToR:vdc-vcf-l7 connection down) contrail-vrouter-agent active contrail-vrouter-nodemgr initializing (NTP state unsynchronized.) On a HA setup(3-CN/3-TSN), Step 1: Rebooted CN-1. waited it to come online and active Step 2: Rebooted CN-2. Waited it to come online and active Step 3: Rebooted CN-3. Waited it to com online and active After effect of above 3 steps lead to all the configs were deleted on the QFX side by the Controller. It has been almost 5-6 hours and still the configs are not pushed to the QFX TORs with tor agent active and all SSL connection intact . Please refer to the tor-agent-1 which has around 3K VNs. Attached is the http introspect page + ovsdb.dump file (at 192.168.61.1 (/var/www/html/pub)) http://10.94.63.103:9010/Snh_SandeshTraceRequest?x=Ovsdb%20Pkt {master:0} root@vdc-vcf-s1> show ovsdb interface Interface VLAN ID Bridge-domain xe-0/0/22:0 xe-0/0/22:1 xe-0/0/22:2 xe-0/0/22:3 xe-1/0/15 xe-1/0/23 {master:0} root@vdc-vcf-s1> show ovsdb controller VTEP controller information: Controller IP address: 10.94.191.153 Controller protocol: ssl Controller port: 6645 Controller connection: up Controller seconds-since-connect: 22109 Controller seconds-since-disconnect: 22112 Controller last-eror: Connection reset by peer Controller connection status: active {master:0} root@vdc-vcf-s1> show system core-dumps fpc0: -------------------------------------------------------------------------- /var/tmp/*core*: No such file or directory fpc1: -------------------------------------------------------------------------- /var/tmp/*core*: No such file or directory {master:0} root@vdc-vcf-s1> root@Host1-CN1:/var/crashes# contrail-status vRouter is NOT PRESENT == Contrail vRouter == supervisor-vrouter: inactive (disabled on boot) unix:///tmp/supervisord_vrouter.sockno == Contrail Control == supervisor-control: active contrail-control active contrail-control-nodemgr active contrail-dns active contrail-named active == Contrail Analytics == supervisor-analytics: active contrail-analytics-api active contrail-analytics-nodemgr active contrail-collector active contrail-query-engine active contrail-snmp-collector active contrail-topology active == Contrail Config == supervisor-config: active contrail-api:0 active contrail-config-nodemgr active contrail-device-manager active contrail-discovery:0 active contrail-schema backup contrail-svc-monitor active ifmap active == Contrail Web UI == supervisor-webui: active contrail-webui active contrail-webui-middleware active == Contrail Support Services == supervisor-support-service: active rabbitmq-server active ========Run time service failures============= /var/crashes/core.contrail-contro.2267.Host1-CN1.1439507792 root@Host5-TSN2:~# contrail-status == Contrail vRouter == supervisor-vrouter: active contrail-tor-agent-1 active contrail-tor-agent-2 initializing (ToR:vdc-vcf-s2 connection down) contrail-tor-agent-3 initializing (ToR:vdc-vcf-s4 connection down) contrail-tor-agent-4 initializing (ToR:vdc-vcf-l1 connection down) contrail-tor-agent-5 initializing (ToR:vdc-vcf-l2 connection down) contrail-tor-agent-6 active contrail-tor-agent-7 initializing (ToR:vdc-vcf-l7 connection down) contrail-vrouter-agent active contrail-vrouter-nodemgr initializing (NTP state unsynchronized.)
2015-08-17 05:27:04 Hari Prasad Killi tags blocker bms qfx blocker bms qfx vrouter
2015-08-17 05:27:15 Hari Prasad Killi juniperopenstack: assignee Ashok Singh (ashoksr)
2015-08-17 05:27:21 Hari Prasad Killi nominated for series juniperopenstack/r2.20
2015-08-17 05:27:21 Hari Prasad Killi bug task added juniperopenstack/r2.20
2015-08-17 05:27:30 Hari Prasad Killi juniperopenstack/r2.20: importance Undecided High
2015-08-17 05:27:33 Hari Prasad Killi juniperopenstack: importance Critical High
2015-08-17 05:27:41 Hari Prasad Killi juniperopenstack/r2.20: assignee Ashok Singh (ashoksr)
2015-08-17 05:27:50 Hari Prasad Killi juniperopenstack/r2.20: milestone r2.21
2015-08-17 05:27:56 Hari Prasad Killi juniperopenstack: milestone r2.30-fcs
2015-08-24 22:11:11 Ignatious Johnson Christopher juniperopenstack: assignee Ashok Singh (ashoksr) Ignatious Johnson Christopher (ijohnson-x)
2015-08-24 22:11:13 Ignatious Johnson Christopher juniperopenstack/r2.20: assignee Ashok Singh (ashoksr) Ignatious Johnson Christopher (ijohnson-x)
2015-08-24 22:24:25 Vinay Mahuli nominated for series juniperopenstack/trunk
2015-08-24 22:24:25 Vinay Mahuli bug task added juniperopenstack/trunk
2015-08-24 22:24:25 Vinay Mahuli bug task added juniperopenstack/trunk
2015-08-25 06:04:42 OpenContrail Admin juniperopenstack/trunk: status In Progress Fix Committed
2015-08-25 17:45:34 Vinay Mahuli juniperopenstack/r2.20: status New In Progress
2015-08-26 17:04:24 OpenContrail Admin juniperopenstack/r2.20: status In Progress Fix Committed
2015-09-03 10:27:11 Hari Prasad Killi information type Proprietary Public