SMLite:R3.2: Provision stops at database state with errors in cassandra when SM node has the cfgm role

[sundarkh]

SMLite:R3.2: Provision stops at database state with errors in cassandra when SM node has the cfgm role

1) Install SM Lite with R3.2 build 9 mitaka
2) Initiate provision on a multi interface setup with config and db in same node; Collecor and database are not in same node
3) Observe that the provision stops at database state, with errors in cassandra

root@nodec35:/etc/puppet/environments# sudo service contrail-database restart
 * Restarting Cassandra cassandra
start-stop-daemon: warning: failed to kill 5838: No such process
   ...done.
root@nodec35:/etc/puppet/environments# sudo service contrail-database status
 * could not access pidfile for Cassandra

Nitish is aware of the issue

[Nitish Krishna Kaveri (nitishk)]

Seems like when cfg0 and SM Lite are on same node, when config db separation is there, the database gets provisioned but cassandra service doesn't start.
if we move cfg0 to non SM Lite node it starts fine and provision completes.

Need to check if we have to change the database_dir in this scenario from the default

[sundarkh (sundar-kh)]

Seen in BUild 17 as well, repro is inconsistent though

server-manager-client display server --select id,cluster_id,roles,ip_address
+---------+----------------+---------------+----------------------------------------------------+
| id | cluster_id | ip_address | roles |
+---------+----------------+---------------+----------------------------------------------------+
| nodec33 | cluster_single | 10.204.221.59 | [u'control', u'webui', u'collector'] |
| nodec35 | cluster_single | 10.204.221.58 | [u'control', u'openstack', u'database', u'config'] |
| nodea4 | cluster_single | 10.204.221.60 | [u'compute'] |
| nodec57 | cluster_single | 10.204.221.61 | [u'compute'] |
+---------+----------------+---------------+----------------------------------------------------+
root@nodec35:~# service cassandra status
 * could not access pidfile for Cassandra
root@nodec35:~# service cassandra restart
 * Restarting Cassandra cassandra start-stop-daemon: warning: failed to kill 12532: No such process
                                                                                                                                                        [ OK ]
root@nodec35:~# vi /var/log/cassandra/system.log
root@nodec35:~# vi /var/log/cassandra/
debug.log gc.log.0.current status.log status-up system.log
root@nodec35:~# vi /var/log/cassandra/debug.log
root@nodec35:~# contrail-status
== Contrail Control ==
supervisor-control: active
contrail-control active
contrail-control-nodemgr active
contrail-dns active
contrail-named active

== Contrail Config ==
supervisor-config: active
contrail-api:0 active
contrail-config-nodemgr active
contrail-device-manager active
contrail-discovery active
contrail-schema active
contrail-svc-monitor active
ifmap active

== Contrail Database ==
contrail-database: inactive

== Contrail Supervisor Database ==
supervisor-database: active
contrail-database-nodemgr active
kafka active

== Contrail Support Services ==
supervisor-support-service: active
rabbitmq-server active

root@nodec35:~#

[Abhay Joshi (abhayj)]

Sudhee is going to discuss with Sundar and get back.

[sundarkh (sundar-kh)]

Issue not seen in 3.1 build 61 ; tested with mitaka

[Abhay Joshi (abhayj)]

Not applicable in container deployment.

[Abhay Joshi (abhayj)]

Please see above comment:

sundarkh (sundar-kh) wrote on 2017-01-24: #4

Issue not seen in 3.1 build 61 ; tested with mitaka