[vCenter] Cinder-volume service is disabled after reboot cinder-vmware node and lost connectivity with vCenter

[Ilya Bumarskov]

Fuel 9.0 mos-iso#387 with DVS plugin 3.0.0 build #418

Detailed bug description:
 cinder-volume service is disabled after reboot cinder-vmware node during lost connection with vCenter.

Steps to reproduce:

   - Create env with vCenter and DVS plugin
   - Add following nodes:
         * Controller
         * Controller
         * Controller
         * Cinder-vmware
         * Compute-vmware
   - Deploy env
   - Simulate lost connection with vCenter
   - Reboot cinder-vmware node
   - Restore connection with vCenter
   - Run OSTF "vCenter: Create volume and attach it to instance"

Expected results:
 OSTF is passed

Actual result:
 OSTF has failed. Cinder-volume in down state:
root@node-2:~# cinder service-list
+------------------+------------------------------+----------------+---------+-------+----------------------------+-----------------+
| Binary | Host | Zone | Status | State | Updated_at | Disabled Reason |
+------------------+------------------------------+----------------+---------+-------+----------------------------+-----------------+
| cinder-scheduler | node-2.test.domain.local | nova | enabled | up | 2016-05-23T15:51:36.000000 | - |
| cinder-volume | vcenter@VMwareVcVmdk-backend | vcenter-cinder | enabled | down | 2016-05-23T15:34:33.000000 | - |
+------------------+------------------------------+----------------+---------+-------+----------------------------+-----------------+

[Alexander Arzhanov (aarzhanov)]

Logs http://pastebin.com/zuNNFY71

Increase the following parameters is not helping:

vmware_api_retry_count
vmware_task_poll_interval

[Dina Belova (dbelova)]

Setting to confirmed. MOS Cinder team, please pay attention.

[Ivan Kolodyazhny (e0ne)]

Ilya, provide env to reproduce this bug. MOS-Cinder team doesn't have environment with VMWare

[Ivan Kolodyazhny (e0ne)]

Actually, it's a normal Cinder behaviour for all backends. If backend driver wasn't initialized on cinder-volume startup, Cinder will never try to do it again.

Fix of this issue should be done in scope of blueprint [1] because it's a improvement and/or feature request.

https://blueprints.launchpad.net/cinder/+spec/driver-initialization-after-fail