setup_all failes on keystone not coming up

Bug #1612833 reported by wenqing liang
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Juniper Openstack
Status tracked in Trunk
R3.1
Won't Fix
Medium
wenqing liang
R3.2
Won't Fix
Medium
wenqing liang
Trunk
Invalid
High
wenqing liang

Bug Description

Seen on r3.1.0.0-23 mitaka tgz install. see setup_all logs at /cs-shared/bugs/1612833.

From: Ignatious Johnson
Sent: Friday, August 12, 2016 12:22 PM
To: Amudha R <email address hidden>; Nagendra Prasath Maynattamai Prem Chandran <email address hidden>; Wenqing Liang <email address hidden>; Suresh Akula <email address hidden>
Cc: Jeya Ganesh Babu Jegatheesan <email address hidden>
Subject: Re: contrail services in problem state

Hi Wenqing,

setup_all has failed due to keystone not coming up.

Keystone is running natively using upstart instead of under the supervisor group. So supervisor-openstack couldn’t start keystone

Also spoke to Nagendra, He said TGZ is supported/shipped for R3.1

Thanks,
Ignatious

From: Amudha R <email address hidden>
Date: Friday, August 12, 2016 at 12:12 PM
To: Nagendra Prasath Maynattamai Prem Chandran <email address hidden>, Wenqing Liang <email address hidden>, Suresh Akula <email address hidden>
Cc: Ignatious Johnson <email address hidden>, Jeya Ganesh Babu Jegatheesan <email address hidden>
Subject: Re: contrail services in problem state

I did not port storage support during mitaka bring up. I believe Jeya Ganesh brought in the storage support.

-Amudha

From: Nagendra Prasath Maynattamai Prem Chandran <email address hidden>
Date: Friday, August 12, 2016 at 2:04 PM
To: Wenqing Liang <email address hidden>, Suresh Akula <email address hidden>, Amudha R <email address hidden>
Cc: Ignatious Johnson <email address hidden>, Jeya Ganesh Babu Jegatheesan <email address hidden>
Subject: Re: contrail services in problem state

+Amudha

Regards,
Nagendra Prasath

From: Wenqing Liang <email address hidden>
Date: Friday, August 12, 2016 at 11:55 AM
To: Suresh Akula <email address hidden>
Cc: Ignatious Johnson <email address hidden>, Nagendra Prasath <email address hidden>, Jeya Ganesh Babu Jegatheesan <email address hidden>
Subject: RE: contrail services in problem state

BTW, this is a tgz intall.

From: Suresh Akula
Sent: Friday, August 12, 2016 11:47 AM
To: Wenqing Liang <email address hidden>
Cc: Ignatious Johnson <email address hidden>; Nagendra Prasath Maynattamai Prem Chandran <email address hidden>; Jeya Ganesh Babu Jegatheesan <email address hidden>
Subject: Re: contrail services in problem state

FYI:
When we install Stroage WebUI on top of Contrail webui, first we will stop the webui service than we will install/patch up the Stroage WebUI.

As per logs earlier Contrail webui is not installed properly and webui service is not running... Due to this we see below error while installing Strorage.

Thanks,
--Suresh Akula.

On Aug 12, 2016, at 11:31 AM, Wenqing Liang <email address hidden> wrote:
Hi Ignatious/Nagendra,
R3.1.0.0-23 mitaka’s fab setup_storage aborted:

2016-08-12 11:06:10:433419: [10.87.140.197] out: [localhost] local: sudo mv storage.config.global.js.new /usr/src/contrail/contrail-web-storage/webroot/common/config/storage.config.global.js
2016-08-12 11:06:10:433491: [10.87.140.197] out: [10.87.140.197] run: sudo storage-webui-setup --storage-setup-mode setup
2016-08-12 11:06:10:440994: [10.87.140.197] out: [10.87.140.197] out: stopping... supervisor-webui service
2016-08-12 11:06:10:856995: [10.87.140.197] out: [10.87.140.197] out: [localhost] local: sudo service supervisor-webui stop
2016-08-12 11:06:10:857088: [10.87.140.197] out: [10.87.140.197] out: stop: Unknown instance:
2016-08-12 11:06:10:864454: [10.87.140.197] out: [10.87.140.197] out:
2016-08-12 11:06:10:864559: [10.87.140.197] out: [10.87.140.197] out: Fatal error: local() encountered an error (return code 1) while executing 'sudo service supervisor-webui stop'
2016-08-12 11:06:10:864604: [10.87.140.197] out: [10.87.140.197] out:
2016-08-12 11:06:10:864644: [10.87.140.197] out: [10.87.140.197] out: Aborting.
2016-08-12 11:06:10:864682: [10.87.140.197] out: [10.87.140.197] out:
2016-08-12 11:06:10:880163: [10.87.140.197] out:
2016-08-12 11:06:10:911649: [10.87.140.197] out:
2016-08-12 11:06:10:911727: [10.87.140.197] out: Fatal error: run() received nonzero return code 1 while executing!
2016-08-12 11:06:10:911765: [10.87.140.197] out:
2016-08-12 11:06:10:911802: [10.87.140.197] out: Requested: sudo storage-webui-setup --storage-setup-mode setup
2016-08-12 11:06:10:911837: [10.87.140.197] out: Executed: /bin/bash -l -c "sudo storage-webui-setup --storage-setup-mode setup"
2016-08-12 11:06:10:911873: [10.87.140.197] out:
2016-08-12 11:06:10:911909: [10.87.140.197] out: Aborting.
2016-08-12 11:06:10:911943: [10.87.140.197] out:

It looks to have to do with contrail service in problem states:

root@cmbu-ceph-perf1:/opt/contrail/utils# contrail-status
== Contrail Control ==
supervisor-control: inactive (disabled on boot)
unix:///var/run/supervisord_control.sockno

== Contrail Analytics ==
supervisor-analytics: inactive (disabled on boot)
unix:///var/run/supervisord_analytics.sockno

== Contrail Config ==
supervisor-config: active (disabled on boot)
contrail-api:0 initializing (Collector, Discovery:Collector[Subscribe - Connection Error], Database:Cassandra[] connection down)
contrail-config-nodemgr initializing (Collector, Discovery:Collector[Subscribe - Connection Error] connection down)
contrail-device-manager initializing (Collector, ApiServer:ApiServer[], Discovery:Collector[Subscribe - Connection Error] connection down)
contrail-discovery:0 initializing (Collector, Discovery:Collector[Subscribe - Connection Error], Database:Cassandra[] connection down)
contrail-schema backup
contrail-svc-monitor initializing (Collector, Database:Cassandra[] connection down)
ifmap active

== Contrail Web UI ==
supervisor-webui: inactive (disabled on boot)
unix:///var/run/supervisord_webui.sockno

== Contrail Database ==
contrail-database: active

== Contrail Supervisor Database ==
supervisor-database: active
contrail-database-nodemgr initializing (Collector, Discovery:Collector[Subscribe - Connection Error] connection down)
kafka active

== Contrail Support Services ==
supervisor-support-service: active
rabbitmq-server active

== Contrail Storage ==
contrail-storage-stats: active
root@cmbu-ceph-perf1:/opt/contrail/utils#

Can you pls have a look? The cluster IPs are 10.87.140.197-200, root/n1keenA.

Thanks,
Wenqing

wenqing liang (wliang)
description: updated
information type: Proprietary → Public
wenqing liang (wliang)
tags: removed: blocker
Revision history for this message
Jeba Paulaiyan (jebap) wrote :

This is seen only in tgz based install.

Revision history for this message
Jeba Paulaiyan (jebap) wrote :

tgz packaging is not yet productized. Will close based on 4.0.0.0 packages testing.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.