Storwize_svc driver doesn't work properly in FC mode if svcinfo lsnode returns multipe WWPNs

[Jay Bryant]

We have encountered a case on our system where 'svcinfo lsnode' returns multiple WWPNs:

[root@szmlnx07 ˜]# ssh admin@9.114.62.132 -i /etc/cinder/id_rsa 'svcinfo lsnode 12'
id 12
name node11
UPS_serial_number 1000995057
WWNN 5005076801000FE9
status online
IO_group_id 0
IO_group_name io_grp0
partner_node_id 10
partner_node_name node10
config_node no
UPS_unique_id 2040000249140147
port_id 5005076801400FE9
port_status active
port_speed 4Gb
port_id 5005076801300FE9
port_status active
port_speed 4Gb
port_id 5005076801100FE9
port_status inactive
port_speed N/A
port_id 5005076801200FE9
port_status inactive
port_speed N/A
...

This results in preferred_node_entry['WWPN'] having mutliple WWPNs in it: ['5005076801200FE9', '5005076801300FE9', '5005076801100FE9', '5005076801400FE9'] . This results in line 412 of __init__.py failing its check:

                if not vol_opts['multipath']:
                    if preferred_node_entry['WWPN'] in conn_wwpns:
                        properties['target_wwn'] = preferred_node_entry['WWPN']
                    else:
                        properties['target_wwn'] = conn_wwpns[0]

It appears that an assumption is being made that preferred_node_entry is going to be a single node but looking at the documenation for lsnode it clearly says that it can return multiple WWPNs. So, I think the code above needs to be updated to handle multiple WWPNs.

In the case of our system conn_wwpns is ['500507680140010F', '5005076801406718', '500507680110010F', '5005076801300FE9', '5005076801104EE7', '5005076801404EE7'] . I think the code should check each of the entries in preferred_node_entry['WWPN'] until if finds one that works. In our case 5005076801300FE9. I tried hardcoding the value and it seemed to give the desired result.

I am going to code up a solution to account for this situation.

I think we have a new configuration that may be uncovering this. Our environment has an SVC front ending an XIV and other storage. I think this may be uncovering configurations that hadn't previously been tried.

[Openstack Gerrit (openstack-gerrit)]

Fix proposed to branch: master
Review: https://review.openstack.org/88705

[Openstack Gerrit (openstack-gerrit)]

Reviewed: https://review.openstack.org/88705
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=9bfc5b0bf58ecb7de5904cbd41db6de7330187e3
Submitter: Jenkins
Branch: master

commit 9bfc5b0bf58ecb7de5904cbd41db6de7330187e3
Author: Jay S. Bryant <email address hidden>
Date: Fri Apr 18 17:07:15 2014 -0500

    Fix handling multiple WWPNs on preferred FC node

    There was a bug in the code that checked to see if the
    appropriate WWPN for a preferred node was available on the storage
    host in intialize_connection. It was assumed that there would only
    ever be one preferred WWPN. The code just checked to see if the list of
    preferred WWPNs was in the list of available WWPNs. This would only work
    if all of the WWPNs were available on the storage host. This resulted in
    volumes not being able to be mounted when they should have been
    available.

    This fix changes the check of the preferred WWPNs to check each
    item in the list until one is found. The first one found is used
    as the target WWPN. If no match is found, the old default behavior
    of selecting the first available WWPN is used. I have added a
    warning message for this case as it is quite possible that this
    won't work in the user's environment as was the case that uncovered
    this bug.

    Note that this change is only relevant for systems that are not using
    multipath as systems that use multipath don't choose one WWPN, they
    just send the whole available list back.

    Change-Id: I8d3239da24a8f9fbd8309ba5c90b05b29a068754
    Close-bug: 1308298

[Openstack Gerrit (openstack-gerrit)]

Fix proposed to branch: stable/icehouse
Review: https://review.openstack.org/90240

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/90240
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=c4d3c1c2b044f1e0dfe970c26bcaf744a49eb5da
Submitter: Jenkins
Branch: stable/icehouse

commit c4d3c1c2b044f1e0dfe970c26bcaf744a49eb5da
Author: Jay S. Bryant <email address hidden>
Date: Fri Apr 18 17:07:15 2014 -0500

    Fix handling multiple WWPNs on preferred FC node

    There was a bug in the code that checked to see if the
    appropriate WWPN for a preferred node was available on the storage
    host in intialize_connection. It was assumed that there would only
    ever be one preferred WWPN. The code just checked to see if the list of
    preferred WWPNs was in the list of available WWPNs. This would only work
    if all of the WWPNs were available on the storage host. This resulted in
    volumes not being able to be mounted when they should have been
    available.

    This fix changes the check of the preferred WWPNs to check each
    item in the list until one is found. The first one found is used
    as the target WWPN. If no match is found, the old default behavior
    of selecting the first available WWPN is used. I have added a
    warning message for this case as it is quite possible that this
    won't work in the user's environment as was the case that uncovered
    this bug.

    Note that this change is only relevant for systems that are not using
    multipath as systems that use multipath don't choose one WWPN, they
    just send the whole available list back.

    Change-Id: I8d3239da24a8f9fbd8309ba5c90b05b29a068754
    (cherry picked from commit 9bfc5b0bf58ecb7de5904cbd41db6de7330187e3)
    Close-bug: 1308298

[Jay Bryant (jsbryant)]

It looks like the fix was released. Not sure why the bug wasn't updated.