Script checking Contrail health for Helm deployment fails

Bug #1783759 reported by Pawel Rusak
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Juniper Openstack
Status tracked in Trunk
R5.0
Fix Committed
High
Pawel Rusak
Trunk
Fix Committed
High
Pawel Rusak

Bug Description

First test script for sanity run for Helm deployment fails, because contrail-status script reports nodes in non active state. Here are logs for actual issue:

2018-07-25 11:38:36,093 - DEBUG - defaultdict(, {'172.16.8.19': {'analytics-nodemgr': {'status': 'initializing', 'description': 'NTP state unsynchronized. '}, 'control-nodemgr': {'status': 'initializing', 'description': 'NTP state unsynchronized. '}, 'analyticsdb-nodemgr': {'status': 'initializing', 'description': 'Disk for DB is too low. NTP state unsynchronized. '}, 'config-nodemgr': {'status': 'initializing', 'description': 'NTP state unsynchronized. '}, 'vrouter-nodemgr': {'status': 'initializing', 'description': 'NTP state unsynchronized. '}}})
2018-07-25 11:38:36,093 - DEBUG - Not all services up. Sleeping for 10 seconds. iteration: 0
2018-07-25 11:38:46,094 - ERROR - Not all services up , Gave up!
2018-07-25 11:38:46,094 - INFO - Failed services are : defaultdict(, {'172.16.8.19': {'analytics-nodemgr': {'status': 'initializing', 'description': 'NTP state unsynchronized. '}, 'control-nodemgr': {'status': 'initializing', 'description': 'NTP state unsynchronized. '}, 'analyticsdb-nodemgr': {'status': 'initializing', 'description': 'Disk for DB is too low. NTP state unsynchronized. '}, 'config-nodemgr': {'status': 'initializing', 'description': 'NTP state unsynchronized. '}, 'vrouter-nodemgr': {'status': 'initializing', 'description': 'NTP state unsynchronized. '}}})
2018-07-25 11:38:46,094 - ERROR - contrail-status failed
2018-07-25 11:38:46,927 - DEBUG - Requesting: http://172.16.8.19:8085/Snh_AgentXmppConnectionStatusReq?
2018-07-25 11:38:46,931 - DEBUG - No XMPP flaps were noticed during the test
2018-07-25 11:38:46,931 - INFO -
2018-07-25 11:38:46,931 - INFO - END TEST : test_contrail_status : FAILED[0:00:12]

Full sanity test output can be found below:

http://logs.opencontrail.org/89/43989/14/experimental/contrail-sanity-ubuntu1604-helm-ocata/95f7e6c/sanity-logs/logs/analyticstestsanity.html

Tags: provisioning
Pawel Rusak (rusakp)
information type: Proprietary → Public
Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] master

Review in progress for https://review.opencontrail.org/44958
Submitter: Pawel Rusak (<email address hidden>)

Jeba Paulaiyan (jebap)
tags: added: provisioning
Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/44958
Committed: http://github.com/Juniper/contrail-helm-deployer/commit/b9762386e06425c0a55f671ac68c2e299808109f
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit b9762386e06425c0a55f671ac68c2e299808109f
Author: Pawel Rusak <email address hidden>
Date: Thu Jul 26 14:15:44 2018 +0200

Fix nodemgr failure in sanity run

To overcome problem with nodemgr failure reported by contrail-status
script, we need to change default expected disk storage space to
much lower values for sanity VMs.

Change-Id: Ic058b79b415c558a419da54a3511a993c7bf2f3e
Closes-Bug: #1783759

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R5.0

Review in progress for https://review.opencontrail.org/45435
Submitter: Pawel Rusak (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/45435
Committed: http://github.com/Juniper/contrail-helm-deployer/commit/22bddd8c3bdda48c22c4d13deba408cb1c544c5a
Submitter: Zuul v3 CI (<email address hidden>)
Branch: R5.0

commit 22bddd8c3bdda48c22c4d13deba408cb1c544c5a
Author: Pawel Rusak <email address hidden>
Date: Thu Jul 26 14:15:44 2018 +0200

Fix nodemgr failure in sanity run

To overcome problem with nodemgr failure reported by contrail-status
script, we need to change default expected disk storage space to
much lower values for sanity VMs.

Change-Id: Ic058b79b415c558a419da54a3511a993c7bf2f3e
Closes-Bug: #1783759
(cherry picked from commit b9762386e06425c0a55f671ac68c2e299808109f)

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.