analytics containers are keeps restarting with ocata-5.0-80 build
Affects | Status | Importance | Assigned to | Milestone | ||
---|---|---|---|---|---|---|
Juniper Openstack | Status tracked in Trunk | |||||
R5.0 |
Invalid
|
Critical
|
Venkatesh Velpula | |||
Trunk |
Invalid
|
Critical
|
Venkatesh Velpula |
Bug Description
Below analytics containers are keeps restarting
contrail-
contrail-
contrail-
contrail-
With same latest ansible deployer code,on build79,we don't see this issue.
build :ocata-5.0-80
deployer :ansible-deployer
hostos :centos7.5
regitry :10.204.
setup:
controller/master :nodec19,
compute/minion . :nodei16,nodei18
docker log snippet
===================
+ mv /etc/contrail/
+ exec /usr/bin/python /usr/bin/
-c /etc/contrail/
05/31/2018 08:00:32 PM [contrail-
05/31/2018 08:00:32 PM [contrail-
05/31/2018 08:00:32 PM [contrail-
Traceback (most recent call last):
File "/usr/bin/
load_
File "/usr/lib/
opserver = OpServer(args_str)
File "/usr/lib/
staticmetho
AttributeError: type object 'ConnectionState' has no attribute 'get_process_
++++ echo 10.204.217.98
+++ local server_
+++ extended_
+++ '[' -n '10.204.217.52:9092 10.204.217.71:9092 10.204.217.98:9092 ' ']'
+++ echo '10.204.217.52:9092 10.204.217.71:9092 10.204.217.98:9092'
++ KAFKA_SERVERS=
++ ANALYTICS_API_VIP=
++ CONFIG_API_VIP=
++ LINKLOCAL_
++ LINKLOCAL_
++ LINKLOCAL_
++ IPFABRIC_
++ IPFABRIC_
++ XMPP_SSL_
++ XMPP_SERVER_
++ XMPP_SERVER_
++ XMPP_SERVER_
++ INTROSPECT_
++ INTROSPECT_
++ INTROSPECT_
++ INTROSPECT_
++ INTROSPECT_
++ SANDESH_
++ SANDESH_
++ SANDESH_
++ SANDESH_
++ is_enabled False
++ local val=false
++ [[ false == \t\r\u\e ]]
++ [[ false == \y\e\s ]]
++ [[ false == \e\n\a\b\l\e\d ]]
++ read -r -d '' sandesh_
++ true
++ is_enabled False
++ local val=false
++ [[ false == \t\r\u\e ]]
++ [[ false == \y\e\s ]]
++ [[ false == \e\n\a\b\l\e\d ]]
++ xmpp_certs_config=
++ METADATA_
++ METADATA_
++ METADATA_
++ METADATA_
++ METADATA_
++ RABBITMQ_VHOST=/
++ RABBITMQ_USER=guest
++ RABBITMQ_
++ RABBITMQ_
++ RABBITMQ_SSL_VER=
++ RABBITMQ_
++ RABBITMQ_
++ RABBITMQ_
++ read -r -d '' rabbitmq_config
++ true
++ read -r -d '' rabbit_config
++ true
++ is_enabled False
++ local val=false
++ [[ false == \t\r\u\e ]]
++ [[ false == \y\e\s ]]
++ [[ false == \e\n\a\b\l\e\d ]]
++ AGENT_MODE=kernel
++ DPDK_UIO_
++ CPU_CORE_MASK=0x01
++ HUGE_PAGES=1024
++ HUGE_PAGES_
++ DPDK_MEM_
++ VHOST_CONFIG_
++ TSN_EVPN_MODE=False
++ VROUTER_
++ VROUTER_
++ VROUTER_
++ DIST_SNAT_
++ FLOW_EXPORT_RATE=0
++ FABRIC_
++ PRIORITY_ID=
++ PRIORITY_BANDWIDTH=
++ PRIORITY_
++ QOS_QUEUE_ID=
++ QOS_LOGICAL_QUEUES=
++ QOS_DEF_
++ PRIORITY_
+ pre_start_init
+ wait_certs_
+ is_ssl_enabled
+ is_enabled False
+ local val=false
+ [[ false == \t\r\u\e ]]
+ [[ false == \y\e\s ]]
+ [[ false == \e\n\a\b\l\e\d ]]
+ is_enabled False
+ local val=false
+ [[ false == \t\r\u\e ]]
+ [[ false == \y\e\s ]]
+ [[ false == \e\n\a\b\l\e\d ]]
+ is_enabled False
+ local val=false
+ [[ false == \t\r\u\e ]]
+ [[ false == \y\e\s ]]
+ [[ false == \e\n\a\b\l\e\d ]]
+ is_enabled False
+ local val=false
+ [[ false == \t\r\u\e ]]
+ [[ false == \y\e\s ]]
+ [[ false == \e\n\a\b\l\e\d ]]
+ return
++ get_listen_
+++ find_my_
+++ cut -d ' ' -f 1
+++ local server_
+++ local server_list=
+++ IFS=,
+++ read -ra server_list
++++ get_local_ips
++++ tr '\n' ,
++++ cat /proc/net/fib_trie
++++ awk '/32 host/ { print f } {f=$2}'
+++ local local_ips=
+++ local ord=1
+++ for server in '"${server_
+++ [[ ,10.204.
+++ (( ord+=1 ))
+++ for server in '"${server_
+++ [[ ,10.204.
+++ echo 10.204.217.71 2
+++ return
++ local ip=10.204.217.71
++ [[ -z 10.204.217.71 ]]
++ echo 10.204.217.71
+ host_ip=
+ cat
+ add_ini_
+ local service_
+ local cfg_path=
+ local delim=__
++ grep '^ANALYTICS_
++ sort
++ cut -d = -f 1
++ set -o posix
++ set
++ sed 's/^ANALYTICS_
+ local vars=
+ local section=
+ set_third_
+ [[ noauth != \k\e\y\s\t\o\n\e ]]
+ return
+ set_vnc_api_lib_ini
+ local tmp_file=
+ cat
+ [[ noauth == \k\e\y\s\t\o\n\e ]]
[root@nodeg31 ~]# contrail-status
Pod Service Original Name State Status
analytics alarm-gen contrail-
analytics api contrail-
analytics collector contrail-
analytics nodemgr contrail-nodemgr running Up 2 hours
analytics query-engine contrail-
analytics snmp-collector contrail-
analytics topology contrail-
config api contrail-
config cassandra contrail-
config device-manager contrail-
config nodemgr contrail-nodemgr running Up About a minute
config rabbitmq contrail-
config schema contrail-
config svc-monitor contrail-
config zookeeper contrail-
control control contrail-
control dns contrail-
control named contrail-
control nodemgr contrail-nodemgr running Up 2 hours
database cassandra contrail-
database kafka contrail-
database nodemgr contrail-nodemgr running Up 2 hours
database zookeeper contrail-
kubernetes kube-manager contrail-
webui job contrail-
webui web contrail-
== Contrail control ==
control: active
nodemgr: active
named: active
dns: active
== Contrail kubernetes ==
kube-manager: active
== Contrail database ==
kafka: active
nodemgr: initializing (Disk for DB is too low. )
zookeeper: active
cassandra: active
== Contrail analytics ==
snmp-collector: inactive
query-engine: active
api: inactive<
alarm-gen: inactive
nodemgr: active
collector: active
topology: inactive
== Contrail webui ==
web: active
job: active
== Contrail config ==
api: active
zookeeper: active
svc-monitor: backup
nodemgr: initializing
device-manager: backup
cassandra: active
rabbitmq: active
schema: backup
[root@nodeg31 ~]#
adding instances.yaml file