DC: helm operation failure during merging helm overrides

[Yuxing]

Brief Description
-----------------
During the deployment of a large scale of DC, the armada application apply will be failed during merging helm overrides

Severity
--------
Minor

Steps to Reproduce
------------------
Write down the steps to reproduce the issue

Expected Behavior
------------------
The application should be uploaded and applied successfully

Actual Behavior
----------------
The application is validated successfully by Armada and uploaded, the application failed before the Armada apply cmd

Reproducibility
---------------
It is intermittent, state the frequency of occurrence

System Configuration
--------------------
Distributed cloud with multiple nodes

Branch/Pull Time/Commit
-----------------------
11/02/2020

Last Pass
---------
na

Timestamp/Logs
--------------
In sysinv.log seeing error message:
sysinv.conductor.kube_app HelmTillerFailure: Helm operation failure: Failed to render helm chart: Helm operation failure: install.go:159: [debug] Original chart version: ""

This exception Should be produced by install_helm_chart_with_dry_run()

Test Activity
-------------
Developer test

Workaround
----------
Apply it again with cmd: system application-apply <application name>

[Ghada Khalil (gkhalil)]

stx.5.0 / medium priority - related to DC scaling feature

[Yuxing (yuxing)]

Add the sysinv.log to analyze this issue

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/762909

[Yuxing (yuxing)]

As the error message is unclear to indicate whether it is a tiller connection(most probably as it will succeed in the 2nd time apply) error or an error in the chart, improve the error handling to offer more info from the helm error message.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Yuxing Jiang (<email address hidden>) on branch: master
Review: https://review.opendev.org/762909
Reason: The buffering shouldn't impact the incomplete err message

[Bart Wensley (bartwensley)]

This is an issue in the helm/tiller code (not a DC issue). I am assigning this to Frank so the containers team can prioritize/investigate.

[Frank Miller (sensfan22)]

This issue is not seen recently and was only reported the one time. If the frequency of this issue increases then please open a new LP with a recent load.