3073: ubuntu-14-04 mitaka SMLite single node containers restart as zookeeper connection is flapping

Bug #1689377 reported by Sudheendra Rao
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Juniper Openstack
New
Critical
Abhay Joshi

Bug Description

The containers (controller and analytics) are restarting as zookeeper connection is falpping on mainline build 3073 mitaka SMLite single node.

logs of contrail-api:

WARNING:api-0:Connection dropped: socket connection error: Connection refused
INFO:api-0:Connecting to 10.204.216.232:2181
WARNING:api-0:Connection dropped: socket connection error: Connection refused
ERROR:contrail-api:Session Event: TCP Connect Fail
ERROR:contrail-api:SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = nodek12 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.204.216.232:2181, ] status = Initializing description = >>, << type = Collector name = server_addrs = [ 10.204.216.232:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Zookeeper:Zookeeper[], Collector connection down >>, ] >>
ERROR:contrail-api:SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = nodek12 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.204.216.232:2181, ] status = Initializing description = >>, << type = Collector name = server_addrs = [ 10.204.216.232:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Zookeeper:Zookeeper[], Collector connection down >>, ] >>
ERROR:contrail-api:SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = nodek12:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 30511 http_port = 8084 start_time = 1494268876738826 collector_name = collector_ip = 10.204.216.232:8086 collector_list = [ 10.204.216.232:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >>
INFO:api-0:Connecting to 10.204.216.232:2181DEBUG:api-0:Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=400000, session_id=0, passwd='\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\
x00\x00', read_only=None)INFO:api-0:Zookeeper connection established, state: CONNECTED

Rabbitmq connection is also down:

=WARNING REPORT==== 8-May-2017::23:22:55 ===
closing AMQP connection <0.5303.0> (127.0.0.1:54454 -> 127.0.0.1:5672):
connection_closed_abruptly

=WARNING REPORT==== 8-May-2017::23:22:55 ===
closing AMQP connection <0.2167.0> (127.0.0.1:54132 -> 127.0.0.1:5672):
connection_closed_abruptly

Changed in juniperopenstack:
milestone: none → r4.0
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.