Script checking Contrail health for Helm deployment fails

Bug #1783759 reported by Pawel Rusak on 2018-07-26
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Juniper Openstack
Status tracked in Trunk
R5.0
Fix Committed
High
Pawel Rusak
Trunk
Fix Committed
High
Pawel Rusak

Bug Description

First test script for sanity run for Helm deployment fails, because contrail-status script reports nodes in non active state. Here are logs for actual issue:

2018-07-25 11:38:36,093 - DEBUG - defaultdict(, {'172.16.8.19': {'analytics-nodemgr': {'status': 'initializing', 'description': 'NTP state unsynchronized. '}, 'control-nodemgr': {'status': 'initializing', 'description': 'NTP state unsynchronized. '}, 'analyticsdb-nodemgr': {'status': 'initializing', 'description': 'Disk for DB is too low. NTP state unsynchronized. '}, 'config-nodemgr': {'status': 'initializing', 'description': 'NTP state unsynchronized. '}, 'vrouter-nodemgr': {'status': 'initializing', 'description': 'NTP state unsynchronized. '}}})
2018-07-25 11:38:36,093 - DEBUG - Not all services up. Sleeping for 10 seconds. iteration: 0
2018-07-25 11:38:46,094 - ERROR - Not all services up , Gave up!
2018-07-25 11:38:46,094 - INFO - Failed services are : defaultdict(, {'172.16.8.19': {'analytics-nodemgr': {'status': 'initializing', 'description': 'NTP state unsynchronized. '}, 'control-nodemgr': {'status': 'initializing', 'description': 'NTP state unsynchronized. '}, 'analyticsdb-nodemgr': {'status': 'initializing', 'description': 'Disk for DB is too low. NTP state unsynchronized. '}, 'config-nodemgr': {'status': 'initializing', 'description': 'NTP state unsynchronized. '}, 'vrouter-nodemgr': {'status': 'initializing', 'description': 'NTP state unsynchronized. '}}})
2018-07-25 11:38:46,094 - ERROR - contrail-status failed
2018-07-25 11:38:46,927 - DEBUG - Requesting: http://172.16.8.19:8085/Snh_AgentXmppConnectionStatusReq?
2018-07-25 11:38:46,931 - DEBUG - No XMPP flaps were noticed during the test
2018-07-25 11:38:46,931 - INFO -
2018-07-25 11:38:46,931 - INFO - END TEST : test_contrail_status : FAILED[0:00:12]

Full sanity test output can be found below:

http://logs.opencontrail.org/89/43989/14/experimental/contrail-sanity-ubuntu1604-helm-ocata/95f7e6c/sanity-logs/logs/analyticstestsanity.html

Pawel Rusak (rusakp) on 2018-07-26
information type: Proprietary → Public

Review in progress for https://review.opencontrail.org/44958
Submitter: Pawel Rusak (<email address hidden>)

Jeba Paulaiyan (jebap) on 2018-07-31
tags: added: provisioning

Reviewed: https://review.opencontrail.org/44958
Committed: http://github.com/Juniper/contrail-helm-deployer/commit/b9762386e06425c0a55f671ac68c2e299808109f
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit b9762386e06425c0a55f671ac68c2e299808109f
Author: Pawel Rusak <email address hidden>
Date: Thu Jul 26 14:15:44 2018 +0200

Fix nodemgr failure in sanity run

To overcome problem with nodemgr failure reported by contrail-status
script, we need to change default expected disk storage space to
much lower values for sanity VMs.

Change-Id: Ic058b79b415c558a419da54a3511a993c7bf2f3e
Closes-Bug: #1783759

Review in progress for https://review.opencontrail.org/45435
Submitter: Pawel Rusak (<email address hidden>)

Reviewed: https://review.opencontrail.org/45435
Committed: http://github.com/Juniper/contrail-helm-deployer/commit/22bddd8c3bdda48c22c4d13deba408cb1c544c5a
Submitter: Zuul v3 CI (<email address hidden>)
Branch: R5.0

commit 22bddd8c3bdda48c22c4d13deba408cb1c544c5a
Author: Pawel Rusak <email address hidden>
Date: Thu Jul 26 14:15:44 2018 +0200

Fix nodemgr failure in sanity run

To overcome problem with nodemgr failure reported by contrail-status
script, we need to change default expected disk storage space to
much lower values for sanity VMs.

Change-Id: Ic058b79b415c558a419da54a3511a993c7bf2f3e
Closes-Bug: #1783759
(cherry picked from commit b9762386e06425c0a55f671ac68c2e299808109f)

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers