On a HA setup, on shutting down one of the CFGM nodes, query engine remain in init state
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Juniper Openstack |
Fix Committed
|
High
|
c mishra | ||
R2.0 |
Fix Committed
|
High
|
c mishra |
Bug Description
On a Ha cluster with 3 Openstack/cfgm nodes , on shutting down one , Query engine remains in init state.
root@cs-scale-3:~# contrail-status | grep query
contrail-
contrail-status | grep query
contrail-
looks like ebenthough teh config has 3 Cassandra nodes , it is still trying to connect to 1
Version : contrail-analytics 2.0-11 11 Ubuntu 12.04.3 Icehouse
root@cs-scale-3:~# tail -f /var/log/
2014-12-07 Sun 07:26:17:449.297 PST cs-scale-3 [Thread 140559956899648, Pid 4810]: RAC_Connect status: 0 0x1fb69e0
2014-12-07 Sun 07:26:17:449.321 PST cs-scale-3 [Thread 140559956899648, Pid 4810]: Connected to REDIS...
2014-12-07 Sun 07:26:17:449.347 PST cs-scale-3 [Thread 140559956899648, Pid 4810]: RAC_Connect status: 0 0x1fb7260
2014-12-07 Sun 07:26:17:449.359 PST cs-scale-3 [Thread 140559956899648, Pid 4810]: Connected to REDIS...
2014-12-07 Sun 08:01:11:998.110 PST cs-scale-3 [Thread 140559956899648, Pid 4810]: SANDESH: Send FAILED: 1417968071998094 TCP [SYS_DEBUG]: TcpSessionMessa
2014-12-07 Sun 08:08:29:628.355 PST cs-scale-3 [Thread 140559956899648, Pid 4810]: SANDESH: Send FAILED: 1417968509628320 TCP [SYS_DEBUG]: TcpSessionMessa
2014-12-07 Sun 08:22:11:949.665 PST cs-scale-3 [Thread 140559956899648, Pid 4810]: SANDESH: Send FAILED: 1417969331949646 TCP [SYS_DEBUG]: TcpSessionMessa
^C
root@cs-scale-3:~# tail -f /var/log/
contrail-
root@cs-scale-3:~# tail -f /var/log/
Thrift: Sun Dec 7 21:23:59 2014 TSocketPool::open failed <Host: 13.1.0.1 Port: 9160>: connect() failed: Connection timed out
Thrift: Sun Dec 7 21:23:59 2014 TSocketPool::open failed <Host: 13.1.0.1 Port: 9160>: connect() failed: Connection timed out
Thrift: Sun Dec 7 21:23:59 2014 TSocketPool::open failed <Host: 13.1.0.1 Port: 9160>: connect() failed: Connection timed out
Thrift: Sun Dec 7 21:23:59 2014 TSocketPool::open failed <Host: 13.1.0.1 Port: 9160>: connect() failed: Connection timed out
Thrift: Sun Dec 7 21:23:59 2014 TSocketPool::open failed <Host: 13.1.0.1 Port: 9160>: connect() failed: Connection timed out
Thrift: Sun Dec 7 21:23:59 2014 TSocketPool::open failed <Host: 13.1.0.1 Port: 9160>: connect() failed: Connection timed out
Thrift: Sun Dec 7 21:23:59 2014 TSocketPool::open failed <Host: 13.1.0.1 Port: 9160>: connect() failed: Connection timed out
Thrift: Sun Dec 7 21:23:59 2014 TSocketPool::open failed <Host: 13.1.0.1 Port: 9160>: connect() failed: Connection timed out
Thrift: Sun Dec 7 21:23:59 2014 TSocketPool::open failed <Host: 13.1.0.1 Port: 9160>: connect() failed: Connection timed out
Thrift: Sun Dec 7 21:23:59 2014 TSocketPool::open failed <Host: 13.1.0.1 Port: 9160>: connect() failed: Connection timed out
description: | updated |
Changed in juniperopenstack: | |
importance: | Undecided → Critical |
importance: | Critical → High |
Changed in juniperopenstack: | |
assignee: | Raj Reddy (rajreddy) → c mishra (cdmishra) |
Changed in juniperopenstack: | |
status: | New → In Progress |
information type: | Proprietary → Public |
tags: | added: analytics |
Changed in juniperopenstack: | |
milestone: | none → r2.20-fcs |
Changed in juniperopenstack: | |
milestone: | r2.20-fcs → none |
Prabhakar, Please triage.