eqlx driver timeout error in periodic task

[John Haan]

Cinder volume driver gets the capacity information from periodic task.
Eqlx driver, which is EqualLogic Storage of Dell, gets the information using 'pool select default show' CLI command.
The CLI command brings up default pool information and all list of volumes.

Below is output of command.

TEST-PS6210> pool select default show
______________________________ Pool Information _______________________________
Name: default
Description:
Default: true
Data-Reduction: not-started
TotalVolumes: 4
VolumesOnline: 4
.
.
TotalCapacity: 9.1TB
VolumeReserve: 102.47GB
.
.
FreeSpace: 8.89TB
.
.
_______________________________________________________________________________
Name Status Model Version Disks Capacity FreeSpace Connections
---------- ------- ------- ---------- ----- ---------- ---------- -----------
PS6210XS-0 online 70-0425 V9.0.3 (R4 24 9.1TB 8.89TB 2
  1 27117)

___________________________________ Volumes ___________________________________

Name Size Snapshots Status Permission Connections
--------------- ---------- --------- -------- ---------- -----------
volume-d961ebbb 1GB 2 online read-write 0
  -ccfe-4983-95
  a1-cdb5faf7a4
  72

It could make a problem when cinder hava a lot of volumes in eqlx driver.

Below is cinder-volume log,
2017-02-02 11:29:44.540 151491 ERROR cinder.volume.drivers.eqlx [-] The EQL array has closed the connection.
2017-02-02 11:29:44.541 151491 ERROR cinder.volume.drivers.eqlx [req-2feb8b12-ea6f-45c2-81e6-09ff45078df4 - - - - -] Error running command.
2017-02-02 11:29:44.541 151491 ERROR cinder.volume.drivers.eqlx Traceback (most recent call last):
2017-02-02 11:29:44.541 151491 ERROR cinder.volume.drivers.eqlx File "/usr/local/lib/python2.7/dist-packages/cinder/volume/drivers/eqlx.py", line 263, in _run_ssh
2017-02-02 11:29:44.541 151491 ERROR cinder.volume.drivers.eqlx timeout=self.configuration.ssh_conn_timeout)
2017-02-02 11:29:44.541 151491 ERROR cinder.volume.drivers.eqlx File "/usr/local/lib/python2.7/dist-packages/cinder/volume/drivers/eqlx.py", line 94, in __inner
2017-02-02 11:29:44.541 151491 ERROR cinder.volume.drivers.eqlx res = gt.wait()
2017-02-02 11:29:44.541 151491 ERROR cinder.volume.drivers.eqlx File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 175, in wait
2017-02-02 11:29:44.541 151491 ERROR cinder.volume.drivers.eqlx return self._exit_event.wait()
2017-02-02 11:29:44.541 151491 ERROR cinder.volume.drivers.eqlx File "/usr/lib/python2.7/dist-packages/eventlet/event.py", line 121, in wait
2017-02-02 11:29:44.541 151491 ERROR cinder.volume.drivers.eqlx return hubs.get_hub().switch()
2017-02-02 11:29:44.541 151491 ERROR cinder.volume.drivers.eqlx File "/usr/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 294, in switch
2017-02-02 11:29:44.541 151491 ERROR cinder.volume.drivers.eqlx return self.greenlet.switch()
2017-02-02 11:29:44.541 151491 ERROR cinder.volume.drivers.eqlx File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 214, in main
2017-02-02 11:29:44.541 151491 ERROR cinder.volume.drivers.eqlx result = function(*args, **kwargs)
2017-02-02 11:29:44.541 151491 ERROR cinder.volume.drivers.eqlx File "/usr/local/lib/python2.7/dist-packages/cinder/volume/drivers/eqlx.py", line 211, in _ssh_execute
2017-02-02 11:29:44.541 151491 ERROR cinder.volume.drivers.eqlx self._get_output(chan)
2017-02-02 11:29:44.541 151491 ERROR cinder.volume.drivers.eqlx File "/usr/local/lib/python2.7/dist-packages/cinder/volume/drivers/eqlx.py", line 189, in _get_output
2017-02-02 11:29:44.541 151491 ERROR cinder.volume.drivers.eqlx raise exception.VolumeBackendAPIException(data=msg)
2017-02-02 11:29:44.541 151491 ERROR cinder.volume.drivers.eqlx VolumeBackendAPIException: Bad or unexpected response
from the storage volume backend API: The EQL array has closed the connection.

In our case, we can get the error when we have more than 220 volumes for both PS6110XV and PS5110XV model.