Kubernetes: empty configuration for etcd slave node

[Alexander Tivelkov]

A "etcdctl member add" command which adds slave etcd nodes to the cluster may fail if the cluster is busy when it is run;
if it happens the command has to be repeated. The code which is doing that contains a bug: it attempts to repeat the command only 5 times with a 2-second intervals between attempts.

So, if after 5 tries the execution still had not been successful, the code is just returning an empty string as a configuration setting of the newly added node. This string, beeing passed to other cluster nodes, may cause them to malfunction at some later point of environment lifecycle.

The script needs to be rewritten to do more attempts (10 second is not enough), if if none of them succeed, the job should fail at this point, without passing incorrect config any further.

[Serg Melikyan (smelikyan)]

Fixed by https://review.openstack.org/222832 in Liberty

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/kilo
Review: https://review.openstack.org/251341

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/251341
Committed: https://git.openstack.org/cgit/openstack/murano-apps/commit/?id=b0e8122664c0289034144010a10fe06108743ab7
Submitter: Jenkins
Branch: stable/kilo

commit b0e8122664c0289034144010a10fe06108743ab7
Author: Alexander Tivelkov <email address hidden>
Date: Sat Sep 12 05:22:21 2015 +0300

    Fixed an issue occasionally causing an empty config for etcd slave nodes

    The script which adds new nodes to etcd cluster now does up to 30 attempts
    and fails (returns a non-zero exit code) if none are successful.

    A minor performance optimization has been done as well, so no unneeded
    waits happen when the command is successful and no temporary files are used.

    Closes-Bug: #1494997
    Change-Id: Ic0552f388518c119925da902b64797bbf96f979f
    (cherry picked from commit d1e5e3f323967115bd126b1f196dfb281d2fba76)