Description of problem:
OpenStack - Cinder : Cisco Zone driver is failing to create zones for the second switch
Version-Release number of selected components (if applicable)
openstack-cinder trunk
How reproducible:
Always
Steps to reproduce:
Use two cisco mds9k: switchA & switchB
Ethernet links between cinder-volume and mgmt ip of switches are ok
FC links between cinder-volume and switches are ok
FC links between nova-compute and switches are ok
cinder.conf entries
[fc-zone-manager]
zone_driver = cinder.zonemanager.drivers.cisco.cisco_fc_zone_driver.CiscoFCZoneDriver
fc_san_lookup_service = cinder.zonemanager.drivers.cisco.cisco_fc_san_lookup_service.CiscoFCSanLookupService
fc_fabric_names = switchA, switchB
cisco_sb_connector = cinder.zonemanager.drivers.cisco.cisco_fc_zone_client_cli.CiscoFCZoneClientCLI
[switchA]
cisco_fc_fabric_address = mgmt_ip of switchA
cisco_fc_fabric_password = password
cisco_fc_fabric_port = 22
cisco_fc_fabric_user = admin
cisco_zone_activate = True
cisco_zone_name_prefix = openstack
cisco_zoning_policy = initiator-target
cisco_zoning_vsan = None
[switchB]
cisco_fc_fabric_address = mgmt_ip of switchB
cisco_fc_fabric_password = password
cisco_fc_fabric_port = 22
cisco_fc_fabric_user = admin
cisco_zone_activate = True
cisco_zone_name_prefix = openstack
cisco_zoning_policy = initiator-target
cisco_zoning_vsan = None
Create VM from image(create a new volume)
or Create volume from image, Create VM from volume
Check the multi-path between nova-compute and MDS, will find there is only path between compute-node and switchA
Check the configuration of switches, no config on switchB
With cinder logs as below:
2016-10-24 14:54:41.175 24578 DEBUG cinder.zonemanager.drivers.cisco.cisco_fc_san_lookup_service [req-557d2752-fd4f-4cb7-85de-bc3672eaef98 dc5897dd49494fae88bbc3592103e2be f4b11479047f421eac98663a3dee0ce9 - - -] ++++++++++++++++++++++ get_nameserver_info /usr/lib/python2.7/site-packages/cinder/zonemanager/drivers/cisco/cisco_fc_san_lookup_service.py:181
2016-10-24 14:54:41.175 24578 DEBUG cinder.zonemanager.drivers.cisco.cisco_fc_san_lookup_service [req-557d2752-fd4f-4cb7-85de-bc3672eaef98 dc5897dd49494fae88bbc3592103e2be f4b11479047f421eac98663a3dee0ce9 - - -] ['show fcns database vsan ', '10', ' | no-more'] get_nameserver_info /usr/lib/python2.7/site-packages/cinder/zonemanager/drivers/cisco/cisco_fc_san_lookup_service.py:182
2016-10-24 14:54:41.176 24578 DEBUG cinder.zonemanager.drivers.cisco.cisco_fc_san_lookup_service [req-557d2752-fd4f-4cb7-85de-bc3672eaef98 dc5897dd49494fae88bbc3592103e2be f4b11479047f421eac98663a3dee0ce9 - - -] 10 get_nameserver_info /usr/lib/python2.7/site-packages/cinder/zonemanager/drivers/cisco/cisco_fc_san_lookup_service.py:183
2016-10-24 14:54:41.176 24578 DEBUG cinder.zonemanager.drivers.cisco.cisco_fc_san_lookup_service [req-557d2752-fd4f-4cb7-85de-bc3672eaef98 dc5897dd49494fae88bbc3592103e2be f4b11479047f421eac98663a3dee0ce9 - - -] None _run_ssh /usr/lib/python2.7/site-packages/cinder/zonemanager/drivers/cisco/cisco_fc_san_lookup_service.py:234
####at the first time,the value of self.sshpool is None
2016-10-24 14:54:41.176 24578 DEBUG cinder.zonemanager.drivers.cisco.cisco_fc_san_lookup_service [req-557d2752-fd4f-4cb7-85de-bc3672eaef98 dc5897dd49494fae88bbc3592103e2be f4b11479047f421eac98663a3dee0ce9 - - -] 176.0.7.26 _run_ssh /usr/lib/python2.7/site-packages/cinder/zonemanager/drivers/cisco/cisco_fc_san_lookup_service.py:235
2016-10-24 14:54:41.177 24578 DEBUG cinder.zonemanager.drivers.cisco.cisco_fc_san_lookup_service [req-557d2752-fd4f-4cb7-85de-bc3672eaef98 dc5897dd49494fae88bbc3592103e2be f4b11479047f421eac98663a3dee0ce9 - - -] 22 _run_ssh /usr/lib/python2.7/site-packages/cinder/zonemanager/drivers/cisco/cisco_fc_san_lookup_service.py:236
2016-10-24 14:54:41.177 24578 DEBUG cinder.zonemanager.drivers.cisco.cisco_fc_san_lookup_service [req-557d2752-fd4f-4cb7-85de-bc3672eaef98 dc5897dd49494fae88bbc3592103e2be f4b11479047f421eac98663a3dee0ce9 - - -] admin _run_ssh /usr/lib/python2.7/site-packages/cinder/zonemanager/drivers/cisco/cisco_fc_san_lookup_service.py:237
2016-10-24 14:54:41.177 24578 DEBUG cinder.zonemanager.drivers.cisco.cisco_fc_san_lookup_service [req-557d2752-fd4f-4cb7-85de-bc3672eaef98 dc5897dd49494fae88bbc3592103e2be f4b11479047f421eac98663a3dee0ce9 - - -] Tt34upnT _run_ssh /usr/lib/python2.7/site-packages/cinder/zonemanager/drivers/cisco/cisco_fc_san_lookup_service.py:238
####will enter this part of code "/usr/lib/python2.7/site-packages/cinder/zonemanager/drivers/cisco/cisco_fc_san_lookup_service.py"
#if not self.sshpool:
# self.sshpool = ssh_utils.SSHPool(
# ...
# )
####then connect to switchA and make configuration
... some logs of switchA
####after switchA, it will turn to switchB
2016-10-24 14:54:42.952 24578 DEBUG cinder.zonemanager.drivers.cisco.cisco_fc_san_lookup_service [req-557d2752-fd4f-4cb7-85de-bc3672eaef98 dc5897dd49494fae88bbc3592103e2be f4b11479047f421eac98663a3dee0ce9 - - -] ++++++++++++++++++++++ get_nameserver_info /usr/lib/python2.7/site-packages/cinder/zonemanager/drivers/cisco/cisco_fc_san_lookup_service.py:181
2016-10-24 14:54:42.952 24578 DEBUG cinder.zonemanager.drivers.cisco.cisco_fc_san_lookup_service [req-557d2752-fd4f-4cb7-85de-bc3672eaef98 dc5897dd49494fae88bbc3592103e2be f4b11479047f421eac98663a3dee0ce9 - - -] ['show fcns database vsan ', '2', ' | no-more'] get_nameserver_info /usr/lib/python2.7/site-packages/cinder/zonemanager/drivers/cisco/cisco_fc_san_lookup_service.py:182
2016-10-24 14:54:42.953 24578 DEBUG cinder.zonemanager.drivers.cisco.cisco_fc_san_lookup_service [req-557d2752-fd4f-4cb7-85de-bc3672eaef98 dc5897dd49494fae88bbc3592103e2be f4b11479047f421eac98663a3dee0ce9 - - -] 2 get_nameserver_info /usr/lib/python2.7/site-packages/cinder/zonemanager/drivers/cisco/cisco_fc_san_lookup_service.py:183
2016-10-24 14:54:42.953 24578 DEBUG cinder.zonemanager.drivers.cisco.cisco_fc_san_lookup_service [req-557d2752-fd4f-4cb7-85de-bc3672eaef98 dc5897dd49494fae88bbc3592103e2be f4b11479047f421eac98663a3dee0ce9 - - -] <cinder.ssh_utils.SSHPool object at 0x6604410> _run_ssh /usr/lib/python2.7/site-packages/cinder/zonemanager/drivers/cisco/cisco_fc_san_lookup_service.py:234
####foces on the value of SSHPool
2016-10-24 14:54:42.954 24578 DEBUG cinder.zonemanager.drivers.cisco.cisco_fc_san_lookup_service [req-557d2752-fd4f-4cb7-85de-bc3672eaef98 dc5897dd49494fae88bbc3592103e2be f4b11479047f421eac98663a3dee0ce9 - - -] 176.0.7.25 _run_ssh /usr/lib/python2.7/site-packages/cinder/zonemanager/drivers/cisco/cisco_fc_san_lookup_service.py:235
2016-10-24 14:54:42.954 24578 DEBUG cinder.zonemanager.drivers.cisco.cisco_fc_san_lookup_service [req-557d2752-fd4f-4cb7-85de-bc3672eaef98 dc5897dd49494fae88bbc3592103e2be f4b11479047f421eac98663a3dee0ce9 - - -] 22 _run_ssh /usr/lib/python2.7/site-packages/cinder/zonemanager/drivers/cisco/cisco_fc_san_lookup_service.py:236
2016-10-24 14:54:42.954 24578 DEBUG cinder.zonemanager.drivers.cisco.cisco_fc_san_lookup_service [req-557d2752-fd4f-4cb7-85de-bc3672eaef98 dc5897dd49494fae88bbc3592103e2be f4b11479047f421eac98663a3dee0ce9 - - -] admin _run_ssh /usr/lib/python2.7/site-packages/cinder/zonemanager/drivers/cisco/cisco_fc_san_lookup_service.py:237
2016-10-24 14:54:42.955 24578 DEBUG cinder.zonemanager.drivers.cisco.cisco_fc_san_lookup_service [req-557d2752-fd4f-4cb7-85de-bc3672eaef98 dc5897dd49494fae88bbc3592103e2be f4b11479047f421eac98663a3dee0ce9 - - -] Tt34upnT _run_ssh /usr/lib/python2.7/site-packages/cinder/zonemanager/drivers/cisco/cisco_fc_san_lookup_service.py:238
2016-10-24 14:54:42.955 24578 DEBUG cinder.zonemanager.drivers.cisco.cisco_fc_san_lookup_service [req-557d2752-fd4f-4cb7-85de-bc3672eaef98 dc5897dd49494fae88bbc3592103e2be f4b11479047f421eac98663a3dee0ce9 - - -] None _run_ssh /usr/lib/python2.7/site-packages/cinder/zonemanager/drivers/cisco/cisco_fc_san_lookup_service.py:254
2016-10-24 14:54:42.955 24578 DEBUG oslo_concurrency.processutils [req-557d2752-fd4f-4cb7-85de-bc3672eaef98 dc5897dd49494fae88bbc3592103e2be f4b11479047f421eac98663a3dee0ce9 - - -] Running cmd (SSH): show fcns database vsan 2 | no-more ssh_execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:357
2016-10-24 14:54:42.956 24578 DEBUG paramiko.transport [req-557d2752-fd4f-4cb7-85de-bc3672eaef98 dc5897dd49494fae88bbc3592103e2be f4b11479047f421eac98663a3dee0ce9 - - -] [chan 2] Max packet in: 32768 bytes _log /usr/lib/python2.7/site-packages/paramiko/channel.py:1072
2016-10-24 14:54:42.984 24578 DEBUG paramiko.transport [-] [chan 2] Max packet out: 32768 bytes _log /usr/lib/python2.7/site-packages/paramiko/channel.py:1072
2016-10-24 14:54:42.984 24578 DEBUG paramiko.transport [-] Secsh channel 2 opened. _log /usr/lib/python2.7/site-packages/paramiko/transport.py:1428
2016-10-24 14:54:42.994 24578 DEBUG paramiko.transport [-] [chan 2] Sesch channel 2 request ok _log /usr/lib/python2.7/site-packages/paramiko/channel.py:1072
2016-10-24 14:54:43.858 24578 DEBUG paramiko.transport [-] [chan 2] EOF received (2) _log /usr/lib/python2.7/site-packages/paramiko/channel.py:1072
2016-10-24 14:54:43.863 24578 DEBUG paramiko.transport [-] [chan 2] EOF sent (2) _log /usr/lib/python2.7/site-packages/paramiko/channel.py:1072
2016-10-24 14:54:43.864 24578 DEBUG oslo_concurrency.processutils [req-557d2752-fd4f-4cb7-85de-bc3672eaef98 dc5897dd49494fae88bbc3592103e2be f4b11479047f421eac98663a3dee0ce9 - - -] Result was 0 ssh_execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:388
2016-10-24 14:54:43.865 24578 DEBUG cinder.zonemanager.drivers.cisco.cisco_fc_san_lookup_service [req-557d2752-fd4f-4cb7-85de-bc3672eaef98 dc5897dd49494fae88bbc3592103e2be f4b11479047f421eac98663a3dee0ce9 - - -] CLI output from ssh - output: vsan not present
'cause the value of SSHPool is not empty/None, the configuration which should be make on switchB will be configured on switchA and no config made on switchB