Comment 17 for bug 1661154

Revision history for this message
Deepa (dpaclt) wrote : Re: [Bug 1661154] Re: Dell eqlx/PS driver timeout error in periodic task

Hello Rajini

Yes it fixed with little more tweaks inside cinder.conf.We had to increase the time out value

Thanks again for the response

Sent from my iPhone

> On 27-Nov-2017, at 8:29 PM, Rajini Karthik <email address hidden> wrote:
>
> This fix has been backported to ocata, pike and newton. See above
> messages in the thread
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1661154
>
> Title:
> Dell eqlx/PS driver timeout error in periodic task
>
> Status in Cinder:
> Fix Released
>
> Bug description:
> Cinder volume driver gets the capacity information from periodic task.
> Eqlx driver, which is EqualLogic Storage of Dell, gets the information using 'pool select default show' CLI command.
> The CLI command brings up default pool information and all list of volumes.
>
> Below is output of command.
>
> TEST-PS6210> pool select default show
> ______________________________ Pool Information _______________________________
> Name: default
> Description:
> Default: true
> Data-Reduction: not-started
> TotalVolumes: 4
> VolumesOnline: 4
> .
> .
> TotalCapacity: 9.1TB
> VolumeReserve: 102.47GB
> .
> .
> FreeSpace: 8.89TB
> .
> .
> _______________________________________________________________________________
> Name Status Model Version Disks Capacity FreeSpace Connections
> ---------- ------- ------- ---------- ----- ---------- ---------- -----------
> PS6210XS-0 online 70-0425 V9.0.3 (R4 24 9.1TB 8.89TB 2
> 1 27117)
>
> ___________________________________ Volumes ___________________________________
>
> Name Size Snapshots Status Permission Connections
> --------------- ---------- --------- -------- ---------- -----------
> volume-d961ebbb 1GB 2 online read-write 0
> -ccfe-4983-95
> a1-cdb5faf7a4
> 72
>
> It could make a problem when cinder hava a lot of volumes in eqlx
> driver.
>
> Below is cinder-volume log,
> 2017-02-02 11:29:44.540 151491 ERROR cinder.volume.drivers.eqlx [-] The EQL array has closed the connection.
> 2017-02-02 11:29:44.541 151491 ERROR cinder.volume.drivers.eqlx [req-2feb8b12-ea6f-45c2-81e6-09ff45078df4 - - - - -] Error running command.
> 2017-02-02 11:29:44.541 151491 ERROR cinder.volume.drivers.eqlx Traceback (most recent call last):
> 2017-02-02 11:29:44.541 151491 ERROR cinder.volume.drivers.eqlx File "/usr/local/lib/python2.7/dist-packages/cinder/volume/drivers/eqlx.py", line 263, in _run_ssh
> 2017-02-02 11:29:44.541 151491 ERROR cinder.volume.drivers.eqlx timeout=self.configuration.ssh_conn_timeout)
> 2017-02-02 11:29:44.541 151491 ERROR cinder.volume.drivers.eqlx File "/usr/local/lib/python2.7/dist-packages/cinder/volume/drivers/eqlx.py", line 94, in __inner
> 2017-02-02 11:29:44.541 151491 ERROR cinder.volume.drivers.eqlx res = gt.wait()
> 2017-02-02 11:29:44.541 151491 ERROR cinder.volume.drivers.eqlx File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 175, in wait
> 2017-02-02 11:29:44.541 151491 ERROR cinder.volume.drivers.eqlx return self._exit_event.wait()
> 2017-02-02 11:29:44.541 151491 ERROR cinder.volume.drivers.eqlx File "/usr/lib/python2.7/dist-packages/eventlet/event.py", line 121, in wait
> 2017-02-02 11:29:44.541 151491 ERROR cinder.volume.drivers.eqlx return hubs.get_hub().switch()
> 2017-02-02 11:29:44.541 151491 ERROR cinder.volume.drivers.eqlx File "/usr/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 294, in switch
> 2017-02-02 11:29:44.541 151491 ERROR cinder.volume.drivers.eqlx return self.greenlet.switch()
> 2017-02-02 11:29:44.541 151491 ERROR cinder.volume.drivers.eqlx File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 214, in main
> 2017-02-02 11:29:44.541 151491 ERROR cinder.volume.drivers.eqlx result = function(*args, **kwargs)
> 2017-02-02 11:29:44.541 151491 ERROR cinder.volume.drivers.eqlx File "/usr/local/lib/python2.7/dist-packages/cinder/volume/drivers/eqlx.py", line 211, in _ssh_execute
> 2017-02-02 11:29:44.541 151491 ERROR cinder.volume.drivers.eqlx self._get_output(chan)
> 2017-02-02 11:29:44.541 151491 ERROR cinder.volume.drivers.eqlx File "/usr/local/lib/python2.7/dist-packages/cinder/volume/drivers/eqlx.py", line 189, in _get_output
> 2017-02-02 11:29:44.541 151491 ERROR cinder.volume.drivers.eqlx raise exception.VolumeBackendAPIException(data=msg)
> 2017-02-02 11:29:44.541 151491 ERROR cinder.volume.drivers.eqlx VolumeBackendAPIException: Bad or unexpected response
> from the storage volume backend API: The EQL array has closed the connection.
>
> In our case, we can get the error when we have more than 220 volumes
> for both PS6110XV and PS5110XV model.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/cinder/+bug/1661154/+subscriptions