Cinder-vmware daemon does not restart if it can't connect to vCenter

[okosse]

When nodes with cinder-vmware role reload, If the nodes can’t connect to vCenter, cinder-volume-vmware-vcenter daemon will not start.
Step for reproduce:
1. Deploy cluster, including vCenter and a node with cinder-vmware role.
2. Destroy connectivity between vCenter and the nodes
3. Reboot a node with cinder-vmware
4. Restore connectivity between vCenter and the nodes
5. Run OSTF

Expected results: OSTF passed

Actual results: test "vCenter: Create volume and attach it to instance" is failed.
command "service cinder-volume-vmware-vcenter status" shows "cinder-volume-vmware-vcenter stop/waiting"

We can get the same result, if we restart vCenter and a node with cinder-vmware role at the same time

---------------------------iso version---------------------------------
api: '1.0'
astute_sha: a717657232721a7fafc67ff5e1c696c9dbeb0b95
auth_required: true
build_id: '288'
build_number: '288'
feature_groups:
- mirantis
fuel-agent_sha: 082a47bf014002e515001be05f99040437281a2d
fuel-library_sha: 121016a09b0e889994118aa3ea42fa67eabb8f25
fuel-nailgun-agent_sha: d7027952870a35db8dc52f185bb1158cdd3d1ebd
fuel-ostf_sha: 1f08e6e71021179b9881a824d9c999957fcc7045
fuelmain_sha: 6b83d6a6a75bf7bca3177fcf63b2eebbf1ad0a85
nailgun_sha: 93477f9b42c5a5e0506248659f40bebc9ac23943
openstack_version: 2015.1.0-7.0
production: docker
python-fuelclient_sha: 1ce8ecd8beb640f2f62f73435f4e18d1469979ac
release: '7.0'
release_versions:
  2015.1.0-7.0:
    VERSION:
      api: '1.0'
      astute_sha: a717657232721a7fafc67ff5e1c696c9dbeb0b95
      build_id: '288'
      build_number: '288'
      feature_groups:
      - mirantis
      fuel-agent_sha: 082a47bf014002e515001be05f99040437281a2d
      fuel-library_sha: 121016a09b0e889994118aa3ea42fa67eabb8f25
      fuel-nailgun-agent_sha: d7027952870a35db8dc52f185bb1158cdd3d1ebd
      fuel-ostf_sha: 1f08e6e71021179b9881a824d9c999957fcc7045
      fuelmain_sha: 6b83d6a6a75bf7bca3177fcf63b2eebbf1ad0a85
      nailgun_sha: 93477f9b42c5a5e0506248659f40bebc9ac23943
      openstack_version: 2015.1.0-7.0
      production: docker
      python-fuelclient_sha: 1ce8ecd8beb640f2f62f73435f4e18d1469979ac
      release: '7.0'

[Ilya Bumarskov (ibumarskov)]

Tested on ISO #295
Bug was fixed within "Refactor cinder-vmware role" feature.

[Ilya Bumarskov (ibumarskov)]

Seems we have regression in 9.0

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/310737

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/310883

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/310737
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=8febf5a5c7572391045fb7a00b4f51a8d6fa830f
Submitter: Jenkins
Branch: master

commit 8febf5a5c7572391045fb7a00b4f51a8d6fa830f
Author: Alexander Arzhanov <email address hidden>
Date: Thu Apr 28 14:20:20 2016 +0300

    Update cinder-vmware conf for cinder multi backend

    * Update cinder-vmware conf for cinder multi backend
    * Fix the spec framework for this module so the
      spec can be actually used
    * Disable non-working specs in the vmware modules
      somebody should fix them later.
    * Add spec for vmware::cinder::vmdk
      It tests only this templates and
      should be finished later too.
    * Enable spec for this module in the Fuel CI

    Change-Id: Id3b8cdb10acd0d7a63a085c03795169c5f304c13
    Closes-Bug: #1569874
    Closes-Bug: #1496009

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/310883
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=c4d3dfc9c07db9ac0ba703ac6f491447a12224d3
Submitter: Jenkins
Branch: stable/mitaka

commit c4d3dfc9c07db9ac0ba703ac6f491447a12224d3
Author: Alexander Arzhanov <email address hidden>
Date: Thu Apr 28 14:20:20 2016 +0300

    Update cinder-vmware conf for cinder multi backend

    * Update cinder-vmware conf for cinder multi backend
    * Fix the spec framework for this module so the
      spec can be actually used
    * Disable non-working specs in the vmware modules
      somebody should fix them later.
    * Add spec for vmware::cinder::vmdk
      It tests only this templates and
      should be finished later too.
    * Enable spec for this module in the Fuel CI

    Change-Id: Id3b8cdb10acd0d7a63a085c03795169c5f304c13
    Closes-Bug: #1569874
    Closes-Bug: #1496009
    (cherry picked from commit 8febf5a5c7572391045fb7a00b4f51a8d6fa830f)

[Ekaterina Khomyakova (ekhomyakova)]

Verified on Fuel 9.0 #402

Command "service cinder-volume-vmware-vcenter status" shows "cinder-volume-vmware-vcenter start/running"

OSTF "vCenter: Create volume and attach it to instance" still fails due to https://bugs.launchpad.net/fuel/+bug/1584862 (Won't Fix)