Juniper Openstack

Vcenter-as-compute: Collector crashes at Kafka new topic -uve-x Err during fab setup_all

  1. Series r3.1
  2. Bug #1657623
Bug #1657623 reported by Sarath
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Juniper Openstack
Status tracked in Trunk
R3.1
New
Critical
Raj Reddy
Juniper Openstack r3.1.2.0
R3.2
New
Critical
Raj Reddy
Juniper Openstack r3.2.1.0
Trunk
New
Critical
Raj Reddy
Juniper Openstack r4.0.0.0-fcs "r4.0.0.0"

Bug Description

This issue seen during Fresh install of Vcenter-as-compute build 3.1.2.0-55 ( mitaka) with ubuntu 14.0.4.5 and kernel is 3.13.0-100

[New LWP 22259]
[New LWP 21463]
[New LWP 21461]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/bin/contrail-collector --conf_file /etc/contrail/contrail-keystone-auth.co'.
Program terminated with signal SIGABRT, Aborted.
#0 0x00007f245ee6cc37 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
56 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0 0x00007f245ee6cc37 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1 0x00007f245ee70028 in __GI_abort () at abort.c:89
#2 0x00007f245ee65bf6 in __assert_fail_base (fmt=0x7f245efb63b8 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0x7f553e "0",
    file=file@entry=0x7ec2e4 "controller/src/base/task.cc", line=line@entry=291, function=function@entry=0x7ede60 "virtual tbb::task* TaskImpl::execute()") at assert.c:92
#3 0x00007f245ee65ca2 in __GI___assert_fail (assertion=0x7f553e "0", file=0x7ec2e4 "controller/src/base/task.cc", line=291, function=0x7ede60 "virtual tbb::task* TaskImpl::execute()") at assert.c:101
#4 0x0000000000469a73 in ?? ()
#5 0x00007f24603f4b3a in ?? () from /usr/lib/libtbb.so.2
#6 0x00007f24603f0816 in ?? () from /usr/lib/libtbb.so.2
#7 0x00007f24603eff4b in ?? () from /usr/lib/libtbb.so.2
#8 0x00007f24603ec0ff in ?? () from /usr/lib/libtbb.so.2
#9 0x00007f24603ec2f9 in ?? () from /usr/lib/libtbb.so.2
#10 0x00007f2460610184 in start_thread (arg=0x7f2454469700) at pthread_create.c:312
#11 0x00007f245ef3037d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
(gdb)

>>> collector logs as relevant
2017-01-18 Wed 13:53:10:950.996 PST a6s38 [Thread 139794240262976, Pid 21455]: Kafka new Prod
2017-01-18 Wed 13:53:10:951.156 PST a6s38 [Thread 139794240262976, Pid 21455]: Kafka new topic -uve-0 Err
2017-01-18 Wed 13:53:10:951.184 PST a6s38 [Thread 139794240262976, Pid 21455]: Kafka new topic -uve-1 Err
2017-01-18 Wed 13:53:10:951.205 PST a6s38 [Thread 139794240262976, Pid 21455]: Kafka new topic -uve-2 Err
2017-01-18 Wed 13:53:10:951.227 PST a6s38 [Thread 139794240262976, Pid 21455]: Kafka new topic -uve-3 Err
2017-01-18 Wed 13:53:10:951.246 PST a6s38 [Thread 139794240262976, Pid 21455]: Kafka new topic -uve-4 Err
2017-01-18 Wed 13:53:10:951.266 PST a6s38 [Thread 139794240262976, Pid 21455]: Kafka new topic -uve-5 Err
2017-01-18 Wed 13:53:10:951.288 PST a6s38 [Thread 139794240262976, Pid 21455]: Kafka new topic -uve-6 Err
2017-01-18 Wed 13:53:10:951.306 PST a6s38 [Thread 139794240262976, Pid 21455]: Kafka new topic -uve-7 Err
2017-01-18 Wed 13:53:10:951.325 PST a6s38 [Thread 139794240262976, Pid 21455]: Kafka new topic -uve-8 Err
2017-01-18 Wed 13:53:10:951.346 PST a6s38 [Thread 139794240262976, Pid 21455]: Kafka new topic -uve-9 Err
2017-01-18 Wed 13:53:10:951.365 PST a6s38 [Thread 139794240262976, Pid 21455]: Kafka new topic -uve-10 Err
2017-01-18 Wed 13:53:10:951.385 PST a6s38 [Thread 139794240262976, Pid 21455]: Kafka new topic -uve-11 Err
2017-01-18 Wed 13:53:10:951.404 PST a6s38 [Thread 139794240262976, Pid 21455]: Kafka new topic -uve-12 Err
2017-01-18 Wed 13:53:10:951.423 PST a6s38 [Thread 139794240262976, Pid 21455]: Kafka new topic -uve-13 Err
2017-01-18 Wed 13:53:10:951.443 PST a6s38 [Thread 139794240262976, Pid 21455]: Kafka new topic -uve-14 Err
2017-01-18 Wed 13:53:10:951.463 PST a6s38 [Thread 139794240262976, Pid 21455]: Kafka new topic -uve-15 Err
2017-01-18 Wed 13:53:10:951.483 PST a6s38 [Thread 139794240262976, Pid 21455]: Kafka new topic -uve-16 Err
2017-01-18 Wed 13:53:10:951.502 PST a6s38 [Thread 139794240262976, Pid 21455]: Kafka new topic -uve-17 Err
2017-01-18 Wed 13:53:10:951.521 PST a6s38 [Thread 139794240262976, Pid 21455]: Kafka new topic -uve-18 Err
2017-01-18 Wed 13:53:10:951.542 PST a6s38 [Thread 139794240262976, Pid 21455]: Kafka new topic -uve-19 Err
2017-01-18 Wed 13:53:10:951.561 PST a6s38 [Thread 139794240262976, Pid 21455]: Kafka new topic -uve-20 Err
2017-01-18 Wed 13:53:10:951.579 PST a6s38 [Thread 139794240262976, Pid 21455]: Kafka new topic -uve-21 Err
2017-01-18 Wed 13:53:10:951.600 PST a6s38 [Thread 139794240262976, Pid 21455]: Kafka new topic -uve-22 Err
2017-01-18 Wed 13:53:10:951.618 PST a6s38 [Thread 139794240262976, Pid 21455]: Kafka new topic -uve-23 Err
2017-01-18 Wed 13:53:10:951.637 PST a6s38 [Thread 139794240262976, Pid 21455]: Kafka new topic -uve-24 Err
2017-01-18 Wed 13:53:10:951.658 PST a6s38 [Thread 139794240262976, Pid 21455]: Kafka new topic -uve-25 Err
2017-01-18 Wed 13:53:10:951.676 PST a6s38 [Thread 139794240262976, Pid 21455]: Kafka new topic -uve-26 Err
2017-01-18 Wed 13:53:10:951.696 PST a6s38 [Thread 139794240262976, Pid 21455]: Kafka new topic -uve-27 Err
2017-01-18 Wed 13:53:10:951.715 PST a6s38 [Thread 139794240262976, Pid 21455]: Kafka new topic -uve-28 Err
2017-01-18 Wed 13:53:10:951.732 PST a6s38 [Thread 139794240262976, Pid 21455]: Kafka new topic -uve-29 Err
2017-01-18 Wed 13:53:24:424.785 PST a6s38 [Thread 139794240262976, Pid 21455]: SANDESH: No Client: 1484776404424682 SandeshModuleClientTrace: data= [ name = a6s38:Analytics:contrail-collector:0 client_info= [ status = Idle successful_connections = 0 pid = 21455 http_port = 8089 start_time = 1484776404424402 collector_name = primary = 0.0.0.0:0 secondary = 0.0.0.0:0 rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_diff= [ [ _iter99->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 4 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 ], _iter99->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 ], ] ] tx_msg_diff= [ [ _iter103->first = dropped_no_client _iter103->second = 5, ] ] ]
2017-01-18 Wed 13:53:24:425.849 PST a6s38 [Thread 139794009462528, Pid 21455]: SANDESH: Send FAILED: 1484776404425061 NodeStatusUVE: data= [ name = a6s38 process_status= [ [ [ module_id = contrail-collector instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Idle : EvStart -> Disconnect ], [ type = Database name = a6s38:Global server_addrs= [ [ (*_iter6) = 10.84.13.38:9042, (*_iter6) = 10.84.13.32:9042, (*_iter6) = 10.84.13.33:9042, ] ] status = Up description = ], [ type = Redis-UVE name = From server_addrs= [ [ (*_iter6) = 127.0.0.1:6379, ] ] status = Initializing description = ], [ type = Redis-UVE name = To server_addrs= [ [ (*_iter6) = 127.0.0.1:6379, ] ] status = Initializing description = ], [ type = KafkaPub name = 10.84.13.38:9092,10.84.13.32:9092,10.84.13.33:9092 server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Initializing description = ], ] ] description = Number of connections:5, Expected:7 Missing: Discovery:ApiServer,Discovery:C

>>> stdout
2017-01-18 13:53:10,953:21455(0x7f2462085340):ZOO_INFO@log_env@712: Client environment:zookeeper.version=zookeeper C client 3.4.5
2017-01-18 13:53:10,953:21455(0x7f2462085340):ZOO_INFO@log_env@716: Client environment:host.name=a6s38
2017-01-18 13:53:10,954:21455(0x7f2462085340):ZOO_INFO@log_env@723: Client environment:os.name=Linux
2017-01-18 13:53:10,954:21455(0x7f2462085340):ZOO_INFO@log_env@724: Client environment:os.arch=3.13.0-100-generic
2017-01-18 13:53:10,954:21455(0x7f2462085340):ZOO_INFO@log_env@725: Client environment:os.version=#147-Ubuntu SMP Tue Oct 18 16:48:51 UTC 2016
2017-01-18 13:53:10,955:21455(0x7f2462085340):ZOO_INFO@log_env@733: Client environment:user.name=(null)
2017-01-18 13:53:10,955:21455(0x7f2462085340):ZOO_INFO@log_env@741: Client environment:user.home=/var/lib/contrail
2017-01-18 13:53:10,955:21455(0x7f2462085340):ZOO_INFO@log_env@753: Client environment:user.dir=/
2017-01-18 13:53:10,956:21455(0x7f2462085340):ZOO_INFO@zookeeper_init@786: Initiating client connection, host=10.84.13.32:2181,10.84.13.33:2181,10.84.13.38:2181 sessionTimeout=10000 watcher=(nil) sessionId=0 sessionPasswd=<null> context=(nil) flags=0
2017-01-18 13:53:10,956:21455(0x7f2462085340):ZOO_DEBUG@start_threads@221: starting threads...
2017-01-18 13:53:10,957:21455(0x7f2454469700):ZOO_DEBUG@do_completion@459: started completion thread
2017-01-18 13:53:10,957:21455(0x7f2454c6a700):ZOO_DEBUG@do_io@367: started IO thread
2017-01-18 13:53:10,958:21455(0x7f2454c6a700):ZOO_INFO@check_events@1703: initiated connection to server [10.84.13.33:2181]
2017-01-18 13:53:10,973:21455(0x7f2454c6a700):ZOO_INFO@check_events@1750: session establishment complete on server [10.84.13.33:2181], sessionId=0x259b38fa410000a, negotiated timeout=10000
2017-01-18 13:53:10,973:21455(0x7f2454c6a700):ZOO_DEBUG@check_events@1756: Calling a watcher for a ZOO_SESSION_EVENT and the state=ZOO_CONNECTED_STATE
2017-01-18 13:53:10,973:21455(0x7f2454469700):ZOO_DEBUG@process_completions@2107: Calling a watcher for node [], type = -1 event=ZOO_SESSION_EVENT
2017-01-18 13:53:11,960:21455(0x7f2462085340):ZOO_DEBUG@zoo_acreate@2756: Sending request xid=0x587fe3c7 for path [/collector] to 10.84.13.33:2181
2017-01-18 13:53:11,971:21455(0x7f2454c6a700):ZOO_DEBUG@process_sync_completion@1868: Processing sync_completion with type=6 xid=0x587fe3c7 rc=-110
2017-01-18 13:53:11,971:21455(0x7f2462085340):ZOO_DEBUG@zoo_awget@2655: Sending request xid=0x587fe3c8 for path [/collector] to 10.84.13.33:2181
2017-01-18 13:53:11,973:21455(0x7f2454c6a700):ZOO_DEBUG@process_sync_completion@1868: Processing sync_completion with type=2 xid=0x587fe3c8 rc=0
2017-01-18 13:53:12,973:21455(0x7f2462085340):ZOO_DEBUG@zoo_acreate@2756: Sending request xid=0x587fe3c9 for path [/collector] to 10.84.13.33:2181
2017-01-18 13:53:12,992:21455(0x7f2454c6a700):ZOO_DEBUG@process_sync_completion@1868: Processing sync_completion with type=6 xid=0x587fe3c9 rc=-110
2017-01-18 13:53:12,992:21455(0x7f2462085340):ZOO_DEBUG@zoo_awget@2655: Sending request xid=0x587fe3ca for path [/collector] to 10.84.13.33:2181
2017-01-18 13:53:12,992:21455(0x7f2454c6a700):ZOO_DEBUG@process_sync_completion@1868: Processing sync_completion with type=2 xid=0x587fe3ca rc=0
2017-01-18 13:53:13,992:21455(0x7f2462085340):ZOO_DEBUG@zoo_acreate@2756: Sending request xid=0x587fe3cb for path [/collector] to 10.84.13.33:2181
2017-01-18 13:53:14,004:21455(0x7f2454c6a700):ZOO_DEBUG@process_sync_completion@1868: Processing sync_completion with type=6 xid=0x587fe3cb rc=-110
2017-01-18 13:53:14,004:21455(0x7f2462085340):ZOO_DEBUG@zoo_awget@2655: Sending request xid=0x587fe3cc for path [/collector] to 10.84.13.33:2181
2017-01-18 13:53:14,006:21455(0x7f2454c6a700):ZOO_DEBUG@process_sync_completion@1868: Processing sync_completion with type=2 xid=0x587fe3cc rc=0
2017-01-18 13:53:15,006:21455(0x7f2462085340):ZOO_DEBUG@zoo_acreate@2756: Sending request xid=0x587fe3cd for path [/collector] to 10.84.13.33:2181
2017-01-18 13:53:15,022:21455(0x7f2454c6a700):ZOO_DEBUG@process_sync_completion@1868: Processing sync_completion with type=6 xid=0x587fe3cd rc=-110
2017-01-18 13:53:15,022:21455(0x7f2462085340):ZOO_DEBUG@zoo_awget@2655: Sending request xid=0x587fe3ce for path [/collector] to 10.84.13.33:2181
2017-01-18 13:53:15,022:21455(0x7f2454c6a700):ZOO_DEBUG@process_sync_completion@1868: Processing sync_completion with type=2 xid=0x587fe3ce rc=0
2017-01-18 13:53:16,022:21455(0x7f2462085340):ZOO_DEBUG@zoo_acreate@2756: Sending request xid=0x587fe3cf for path [/collector] to 10.84.13.33:2181
2017-01-18 13:53:16,036:21455(0x7f2454c6a700):ZOO_DEBUG@process_sync_completion@1868: Processing sync_completion with type=6 xid=0x587fe3cf rc=-110
2017-01-18 13:53:16,036:21455(0x7f2462085340):ZOO_DEBUG@zoo_awget@2655: Sending request xid=0x587fe3d0 for path [/collector] to 10.84.13.33:2181
2017-01-18 13:53:16,037:21455(0x7f2454c6a700):ZOO_DEBUG@process_sync_completion@1868: Processing sync_completion with type=2 xid=0x587fe3d0 rc=0
2017-01-18 13:53:17,037:21455(0x7f2462085340):ZOO_DEBUG@zoo_acreate@2756: Sending request xid=0x587fe3d1 for path [/collector] to 10.84.13.33:2181
2017-01-18 13:53:17,058:21455(0x7f2454c6a700):ZOO_DEBUG@process_sync_completion@1868: Processing sync_completion with type=6 xid=0x587fe3d1 rc=-110
2017-01-18 13:53:17,058:21455(0x7f2462085340):ZOO_DEBUG@zoo_awget@2655: Sending request xid=0x587fe3d2 for path [/collector] to 10.84.13.33:2181
2017-01-18 13:53:17,059:21455(0x7f2454c6a700):ZOO_DEBUG@process_sync_completion@1868: Processing sync_completion with type=2 xid=0x587fe3d2 rc=0
2017-01-18 13:53:18,059:21455(0x7f2462085340):ZOO_DEBUG@zoo_acreate@2756: Sending request xid=0x587fe3d3 for path [/collector] to 10.84.13.33:2181
2017-01-18 13:53:18,076:21455(0x7f2454c6a700):ZOO_DEBUG@process_sync_completion@1868: Processing sync_completion with type=6 xid=0x587fe3d3 rc=-110
2017-01-18 13:53:18,076:21455(0x7f2462085340):ZOO_DEBUG@zoo_awget@2655: Sending request xid=0x587fe3d4 for path [/collector] to 10.84.13.33:2181
2017-01-18 13:53:18,077:21455(0x7f2454c6a700):ZOO_DEBUG@process_sync_completion@1868: Processing sync_completion with type=2 xid=0x587fe3d4 rc=0
2017-01-18 13:53:19,077:21455(0x7f2462085340):ZOO_DEBUG@zoo_acreate@2756: Sending request xid=0x587fe3d5 for path [/collector] to 10.84.13.33:2181
2017-01-18 13:53:19,091:21455(0x7f2454c6a700):ZOO_DEBUG@process_sync_completion@1868: Processing sync_completion with type=6 xid=0x587fe3d5 rc=-110
2017-01-18 13:53:19,091:21455(0x7f2462085340):ZOO_DEBUG@zoo_awget@2655: Sending request xid=0x587fe3d6 for path [/collector] to 10.84.13.33:2181
2017-01-18 13:53:19,091:21455(0x7f2454c6a700):ZOO_DEBUG@process_sync_completion@1868: Processing sync_completion with type=2 xid=0x587fe3d6 rc=0
2017-01-18 13:53:20,092:21455(0x7f2462085340):ZOO_DEBUG@zoo_acreate@2756: Sending request xid=0x587fe3d7 for path [/collector] to 10.84.13.33:2181
2017-01-18 13:53:20,139:21455(0x7f2454c6a700):ZOO_DEBUG@process_sync_completion@1868: Processing sync_completion with type=6 xid=0x587fe3d7 rc=-110
2017-01-18 13:53:20,139:21455(0x7f2462085340):ZOO_DEBUG@zoo_awget@2655: Sending request xid=0x587fe3d8 for path [/collector] to 10.84.13.33:2181
2017-01-18 13:53:20,150:21455(0x7f2454c6a700):ZOO_DEBUG@process_sync_completion@1868: Processing sync_completion with type=2 xid=0x587fe3d8 rc=0
2017-01-18 13:53:21,150:21455(0x7f2462085340):ZOO_DEBUG@zoo_acreate@2756: Sending request xid=0x587fe3d9 for path [/collector] to 10.84.13.33:2181
2017-01-18 13:53:21,175:21455(0x7f2454c6a700):ZOO_DEBUG@process_sync_completion@1868: Processing sync_completion with type=6 xid=0x587fe3d9 rc=-110
2017-01-18 13:53:21,175:21455(0x7f2462085340):ZOO_DEBUG@zoo_awget@2655: Sending request xid=0x587fe3da for path [/collector] to 10.84.13.33:2181
2017-01-18 13:53:21,185:21455(0x7f2454c6a700):ZOO_DEBUG@process_sync_completion@1868: Processing sync_completion with type=2 xid=0x587fe3da rc=0
2017-01-18 13:53:22,185:21455(0x7f2462085340):ZOO_DEBUG@zoo_acreate@2756: Sending request xid=0x587fe3db for path [/collector] to 10.84.13.33:2181
2017-01-18 13:53:22,228:21455(0x7f2454c6a700):ZOO_DEBUG@process_sync_completion@1868: Processing sync_completion with type=6 xid=0x587fe3db rc=-110
2017-01-18 13:53:22,228:21455(0x7f2462085340):ZOO_DEBUG@zoo_awget@2655: Sending request xid=0x587fe3dc for path [/collector] to 10.84.13.33:2181
2017-01-18 13:53:22,229:21455(0x7f2454c6a700):ZOO_DEBUG@process_sync_completion@1868: Processing sync_completion with type=2 xid=0x587fe3dc rc=0
2017-01-18 13:53:23,229:21455(0x7f2462085340):ZOO_DEBUG@zoo_acreate@2756: Sending request xid=0x587fe3dd for path [/collector] to 10.84.13.33:2181
2017-01-18 13:53:23,241:21455(0x7f2454c6a700):ZOO_DEBUG@process_sync_completion@1868: Processing sync_completion with type=6 xid=0x587fe3dd rc=-110
2017-01-18 13:53:23,241:21455(0x7f2462085340):ZOO_DEBUG@zoo_awget@2655: Sending request xid=0x587fe3de for path [/collector] to 10.84.13.33:2181
2017-01-18 13:53:23,251:21455(0x7f2454c6a700):ZOO_DEBUG@process_sync_completion@1868: Processing sync_completion with type=2 xid=0x587fe3de rc=0
2017-01-18 13:53:24,251:21455(0x7f2462085340):ZOO_DEBUG@zoo_acreate@2756: Sending request xid=0x587fe3df for path [/collector] to 10.84.13.33:2181
2017-01-18 13:53:24,263:21455(0x7f2454c6a700):ZOO_DEBUG@process_sync_completion@1868: Processing sync_completion with type=6 xid=0x587fe3df rc=0
2017-01-18 13:53:24,409:21455(0x7f2462085340):ZOO_DEBUG@zoo_adelete@2796: Sending request xid=0x587fe3e0 for path [/collector] to 10.84.13.33:2181
2017-01-18 13:53:24,423:21455(0x7f2454c6a700):ZOO_DEBUG@process_sync_completion@1868: Processing sync_completion with type=0 xid=0x587fe3e0 rc=0
2017-01-18 13:53:24,423:21455(0x7f2454c6a700):ZOO_DEBUG@do_io@446: IO thread terminated
2017-01-18 13:53:24,423:21455(0x7f2454469700):ZOO_DEBUG@do_completion@469: completion thread terminated
2017-01-18 13:53:24,423:21455(0x7f2462085340):ZOO_INFO@zookeeper_close@2505: Closing zookeeper sessionId=0x259b38fa410000a to [10.84.13.33:2181]

contrail-collector: controller/src/base/task.cc:291: virtual tbb::task* TaskImpl::execute(): Assertion `0' failed.
2017-01-18 13:53:30,496:22612(0x7fc2264d6340):ZOO_INFO@log_env@712: Client environment:zookeeper.version=zookeeper C client 3.4.5
2017-01-18 13:53:30,496:22612(0x7fc2264d6340):ZOO_INFO@log_env@716: Client environment:host.name=a6s38
2017-01-18 13:53:30,496:22612(0x7fc2264d6340):ZOO_INFO@log_env@723: Client environment:os.name=Linux
2017-01-18 13:53:30,496:22612(0x7fc2264d6340):ZOO_INFO@log_env@724: Client environment:os.arch=3.13.0-100-generic
2017-01-18 13:53:30,496:22612(0x7fc2264d6340):ZOO_INFO@log_env@725: Client environment:os.version=#147-Ubuntu SMP Tue Oct 18 16:48:51 UTC 2016
2017-01-18 13:53:30,496:22612(0x7fc2264d6340):ZOO_INFO@log_env@733: Client environment:user.name=(null)
2017-01-18 13:53:30,496:22612(0x7fc2264d6340):ZOO_INFO@log_env@741: Client environment:user.home=/var/lib/contrail
2017-01-18 13:53:30,496:22612(0x7fc2264d6340):ZOO_INFO@log_env@753: Client environment:user.dir=/
2017-01-18 13:53:30,496:22612(0x7fc2264d6340):ZOO_INFO@zookeeper_init@786: Initiating client connection, host=10.84.13.32:2181,10.84.13.33:2181,10.84.13.38:2181 sessionTimeout=10000 watcher=(nil) sessionId=0 sessionPasswd=<null> context=(nil) flags=0
2017-01-18 13:53:30,496:22612(0x7fc2264d6340):ZOO_DEBUG@start_threads@221: starting threads...
2017-01-18 13:53:30,497:22612(0x7fc2188ba700):ZOO_DEBUG@do_completion@459: started completion thread
2017-01-18 13:53:30,497:22612(0x7fc2190bb700):ZOO_DEBUG@do_io@367: started IO thread
2017-01-18 13:53:30,497:22612(0x7fc2190bb700):ZOO_INFO@check_events@1703: initiated connection to server [10.84.13.38:2181]
2017-01-18 13:53:30,511:22612(0x7fc2190bb700):ZOO_INFO@check_events@1750: session establishment complete on server [10.84.13.38:2181], sessionId=0x359b390440f0003, negotiated timeout=10000
2017-01-18 13:53:30,511:22612(0x7fc2190bb700):ZOO_DEBUG@check_events@1756: Calling a watcher for a ZOO_SESSION_EVENT and the state=ZOO_CONNECTED_STATE
2017-01-18 13:53:30,511:22612(0x7fc2188ba700):ZOO_DEBUG@process_completions@2107: Calling a watcher for node [], type = -1 event=ZOO_SESSION_EVENT
2017-01-18 13:53:31,497:22612(0x7fc2264d6340):ZOO_DEBUG@zoo_acreate@2756: Sending request xid=0x587fe3db for path [/collector] to 10.84.13.38:2181
2017-01-18 13:53:31,509:22612(0x7fc2190bb700):ZOO_DEBUG@process_sync_completion@1868: Processing sync_completion with type=6 xid=0x587fe3db rc=0
2017-01-18 13:53:31,656:22612(0x7fc2264d6340):ZOO_DEBUG@zoo_adelete@2796: Sending request xid=0x587fe3dc for path [/collector] to 10.84.13.38:2181
2017-01-18 13:53:31,668:22612(0x7fc2190bb700):ZOO_DEBUG@process_sync_completion@1868: Processing sync_completion with type=0 xid=0x587fe3dc rc=0
2017-01-18 13:53:31,668:22612(0x7fc2190bb700):ZOO_DEBUG@do_io@446: IO thread terminated
2017-01-18 13:53:31,669:22612(0x7fc2188ba700):ZOO_DEBUG@do_completion@469: completion thread terminated
2017-01-18 13:53:31,669:22612(0x7fc2264d6340):ZOO_INFO@zookeeper_close@2505: Closing zookeeper sessionId=0x359b390440f0003 to [10.84.13.38:2181]

Revision history for this message
Sarath (nsarath) wrote :

nsarath@ubuntu-build04:/auto/cores/1657623$ ls -l
total 1301848
-rwxrwxrwx 1 nsarath test 265103360 Jan 18 18:37 Ctrl-A-log.tar
-rwxrwxrwx 1 nsarath test 8591360 Jan 18 18:37 Ctrl-A-Utils.tar
-rwxrwxrwx 1 nsarath test 511559680 Jan 18 18:37 Ctrl-B-log.tar
-rwxrwxrwx 1 nsarath test 542586880 Jan 18 18:37 Ctrl-C-log.tar
nsarath@ubuntu-build04:/auto/cores/1657623$

Revision history for this message
Raj Reddy (rajreddy) wrote :

From the collector log it shows same as the other PR..

     92 server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = ], ] ] description = Number of connections:6, Expected:7 Missing: Discovery:Collector ], ] ] ]
  53 2017-01-18 Wed 13:53:29:427.667 PST a6s38 [Thread 139794009462528, Pid 21455]: !!!! ERROR !!!! Task caught fatal exception: basic_filebuf::underflow error reading the file Tas kImpl: 0x7f2457e7b840

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.