k8s:collector core is observed while running the k8s sanity

[Venkatesh Velpula]

Build :5.1.0-190
HOSTOS:CentOS 7.5

Provisioning was done by using the sensible deployer . haven't seen any functional impact as such though core was generated

[root@testbed-1-vm4 ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
testbed-1-vm2 Ready <none> 16h v1.9.2
testbed-1-vm3 Ready <none> 16h v1.9.2
testbed-1-vm4 NotReady master 16h v1.9.2

(analytics-collector)[root@testbed-1-vm1 /]$ contrail-version
Package Version Build-ID | Repo | RPM Name
-------------------------------------- ------------------------------ ----------------------------------
contrail-analytics 5.1.0-190.el7 @contrail
contrail-lib 5.1.0-190.el7 @contrail
python-contrail 5.1.0-190.el7 @contrail
contrail-utils 5.1.0-190.el7 @contrail
contrail-setup 5.1.0-190.el7 @contrail

[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `/usr/bin/contrail-collector'.
Program terminated with signal 11, Segmentation fault.
#0 0x00007f42b1f6fab7 in abort () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install contrail-analytics-5.1.0-190.el7.x86_64
(gdb) bt
#0 0x00007f42b1f6fab7 in abort () from /lib64/libc.so.6
#1 0x00007f42b1f67096 in __assert_fail_base () from /lib64/libc.so.6
#2 0x00007f42b1f67142 in __assert_fail () from /lib64/libc.so.6
#3 0x000000000070cedd in OpServerProxy::DeleteUVEs(std::string const&, std::string const&, std::string const&, std::string const&) ()
#4 0x0000000000625016 in SandeshGenerator::DisconnectSession(VizSession*) ()
#5 0x000000000061616c in Collector::DisconnectSession(SandeshSession*) ()
#6 0x000000000082eccd in SandeshServerConnection::ProcessDisconnect(SandeshSession*) ()
#7 0x000000000082b900 in ssm::Established::react(ssm::EvTcpClose const&) ()
#8 0x000000000082bc28 in boost::statechart::simple_state<ssm::Established, SandeshStateMachine, boost::mpl::list<mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, (boost::statechart::history_mode)0>::react_impl(boost::statechart::event_base const&, void const*) ()
#9 0x000000000082b70b in boost::statechart::state_machine<SandeshStateMachine, ssm::Idle, std::allocator<void>, boost::statechart::null_exception_translator>::send_event(boost::statechart::event_base const&) ()
#10 0x0000000000823fa5 in SandeshStateMachine::DequeueEvent(SandeshStateMachine::EventContainer&) ()
#11 0x000000000082aba5 in QueueTaskRunner<SandeshStateMachine::EventContainer, WorkQueue<SandeshStateMachine::EventContainer> >::RunQueue() ()
#12 0x000000000047504f in TaskImpl::execute() ()
#13 0x00007f42b33648ca in tbb::internal::custom_scheduler<tbb::internal::IntelSchedulerTraits>::local_wait_for_all(tbb::task&, tbb::task*) () from /lib64/libtbb.so.2
#14 0x00007f42b33605b6 in tbb::internal::arena::process(tbb::internal::generic_scheduler&) () from /lib64/libtbb.so.2
#15 0x00007f42b335fc8b in tbb::internal::market::process(rml::job&) () from /lib64/libtbb.so.2
#16 0x00007f42b335d67f in tbb::internal::rml::private_worker::run() () from /lib64/libtbb.so.2
#17 0x00007f42b335d879 in tbb::internal::rml::private_worker::thread_routine(void*) () from /lib64/libtbb.so.2
#18 0x00007f42b357fe25 in start_thread () from /lib64/libpthread.so.0
#19 0x00007f42b2036bad in clone () from /lib64/libc.so.6

(analytics-collector)[root@testbed-1-vm1 /]$ contrail-version
Package Version Build-ID | Repo | RPM Name
-------------------------------------- ------------------------------ ----------------------------------
contrail-analytics 5.1.0-190.el7 @contrail
contrail-lib 5.1.0-190.el7 @contrail
python-contrail 5.1.0-190.el7 @contrail
contrail-utils 5.1.0-190.el7 @contrail
contrail-setup 5.1.0-190.el7 @contrail
(analytics-collector)[root@testbed-1-vm1 /]$ [root@testbed-1-vm1 ~]#
[root@testbed-1-vm1 ~]# contrail-status
Pod Service Original Name State Status
analytics alarm-gen contrail-analytics-alarm-gen running Up 13 hours
analytics api contrail-analytics-api running Up 13 hours
analytics collector contrail-analytics-collector running Up 13 hours
analytics nodemgr contrail-nodemgr running Up 13 hours
analytics query-engine contrail-analytics-query-engine running Up 13 hours
config api contrail-controller-config-api running Up 12 hours
config device-manager contrail-controller-config-devicemgr running Up 13 hours
config nodemgr contrail-nodemgr running Up 13 hours
config schema contrail-controller-config-schema running Up 13 hours
config svc-monitor contrail-controller-config-svcmonitor running Up 13 hours
config-database cassandra contrail-external-cassandra running Up 13 hours
config-database nodemgr contrail-nodemgr running Up 13 hours
config-database rabbitmq contrail-external-rabbitmq running Up 13 hours
config-database zookeeper contrail-external-zookeeper running Up 13 hours
control control contrail-controller-control-control running Up 13 hours
control dns contrail-controller-control-dns running Up 13 hours
control named contrail-controller-control-named running Up 13 hours
control nodemgr contrail-nodemgr running Up 13 hours
database cassandra contrail-external-cassandra running Up 13 hours
database kafka contrail-external-kafka running Up 13 hours
database nodemgr contrail-nodemgr running Up 13 hours
database zookeeper contrail-external-zookeeper running Up 13 hours
kubernetes kube-manager contrail-kubernetes-kube-manager running Up 11 hours
webui job contrail-controller-webui-job running Up 13 hours
webui web contrail-controller-webui-web running Up 13 hours

== Contrail control ==
control: active
nodemgr: active
named: active
dns: active

== Contrail config-database ==
nodemgr: active
zookeeper: active
rabbitmq: active
cassandra: active

== Contrail kubernetes ==
kube-manager: backup

== Contrail database ==
kafka: active
nodemgr: active
zookeeper: active
cassandra: active

== Contrail analytics ==
nodemgr: active
api: active
collector: active
query-engine: active
alarm-gen: active

== Contrail webui ==
web: active
job: active

== Contrail config ==
svc-monitor: backup
nodemgr: active
device-manager: backup
api: active
schema: backup

[mkheni (mkheni)]

Where can I find the core for this?

[Andrei Bunghez (abunghez)]

I can also see this trace on a 5.0 based build with RHEL OpenStack 10, no kubernetes involved.