PureISCSIDriver fails to initialize when failing to find available port

[Patrick East]

As seen here: http://ec2-54-67-51-189.us-west-1.compute.amazonaws.com/25/149725/3/check/PureISCSIDriver-tempest-dsvm-volume/39647b0/logs/screen-c-vol.txt.gz#_2015-02-03_19_09_10_429

If something goes wrong when looking up the iscsi port the error message does not actually explain what happened, and instead shows a generic python error of trying to access the field. This log message seems to be the culpret:

LOG.debug(("iSCSI discovery of port %(port_name)s at "
                           "%(port_portal)s failed with error: %(err_msg)s"),
                          {"port_name": self._iscsi_port["name"],
                           "port_portal": self._iscsi_port["portal"],
                           "err_msg": err.stderr})

[Patrick East (patrick-east)]

From the log this got a err code of 11 (ISCSI_ERR_PDU_TIMEOUT) from iscsiadm. The 3rd party CI system is in the process of being upgraded to faster *real* hardware so this should go away, but it points out an issue where the driver will fail to initialize if this timeout happens. We should really be retrying at least a couple of times to handle this.

[Patrick East (patrick-east)]

looks like http://ec2-54-67-51-189.us-west-1.compute.amazonaws.com/58/147758/9/check/PureISCSIDriver-tempest-dsvm-volume/5625c9d/logs/screen-c-bak.txt.gz was hit by this too

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/152693

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/152693
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=7b109bb6bf831dab08fe7a2db2800f2ae34b2bfd
Submitter: Jenkins
Branch: master

commit 7b109bb6bf831dab08fe7a2db2800f2ae34b2bfd
Author: Patrick East <email address hidden>
Date: Tue Feb 3 12:09:58 2015 -0800

    Make PureISCSIDriver iSCSI port discovery more flexible

    While searching for reachable iSCSI ports on the target flash array it
    might fail the discovery command. If this happens on driver setup it
    will throw an exception from attempting to use fields on an
    uninitialized class variable. This fixes the log message and adds a
    retry to the method to make this more flexible with network timeouts.

    Closes-Bug: 1417730
    Change-Id: I99ffb8de4dd5a421442df1bf745808505775480a