os-brick use a wrong device to attach to instance

[XuQi]

Operation Env:
RHOSP 16.2.4

System Env:
1. 2 FC HBAs on compute node
 HBA1 WWN: 21000024ff7f6afa HBA1 port: pci-0000:00:09.0
 HBA2 WWN: 21000024ff7f6afb HBA2 port: pci-0000:00:0a.0
2. 2 FC Storage Ports
 Port1 WWN: 500000E0DA000235
 Port2 WWN: 500000E0DA000224
3. The following device mapped with a lun on compute:
 pci-0000:00:09.0-fc-0x500000e0da000235-lun-1
 pci-0000:00:0a.0-fc-0x500000e0da000224-lun-1

Reproduce steps
1. Create a bootable volume
2. Create a instance from the volume
 The following information in cinder's result:
 'target_wwn': ['500000E0DA000235', '500000E0DA000224']
 'target_lun': 1,
 'initiator_target_map': {'21000024ff7f6afa': ['500000E0DA000224'],
        '21000024ff7f6afb': ['500000E0DA000235']
3. Check the device on the instance and the devices on compute node.

Expected results:
The lun created in step 1 is attached to instance and the following device can be found on the compute node.
 pci-0000:00:09.0-fc-0x500000e0da000224-lun-1
 pci-0000:00:0a.0-fc-0x500000e0da000235-lun-1

Actual result:
No new device was added to compute node.
In xml file of instance, the lun created in step 1 is not be attached.
It is replaced by the lun in the 3 in 'System Env'.

Analysis:
In the log, I saw that the os-brick used the device pci-0000:00:09.0-fc-0x500000e0da000235-lun-1 to attach to the instance which has been mapped with the LUN 600000e00d2800000028000205740000.
2023-02-08 05:28:25.504 8 DEBUG os_brick.initiator.connectors.fibre_channel [-] Looking for Fibre Channel dev /dev/disk/by-path/pci-0000:00:09.0-fc-0x500000e0da000235-lun-1 _wait_for_device_discovery /usr/lib/python3.6/site-packages/os_brick/initiator/connectors/fibre_channel.py:220
2023-02-08 05:28:25.513 8 DEBUG os_brick.initiator.connectors.fibre_channel [req-579a6991-75bc-4c4b-bab6-e5cb62f828fc 881b5bb8ead5457e8d61f7266609bdfe cd6ed70c10d94a9f84fc2d9acdbab698 - default default] Found Fibre Channel volume /dev/sdd (after 0 rescans.) connect_volume /usr/lib/python3.6/site-packages/os_brick/initiator/connectors/fibre_channel.py:248
2023-02-08 05:28:25.525 8 DEBUG os_brick.initiator.connectors.fibre_channel [req-579a6991-75bc-4c4b-bab6-e5cb62f828fc 881b5bb8ead5457e8d61f7266609bdfe cd6ed70c10d94a9f84fc2d9acdbab698 - default default] Device WWN = '3600000e00d2800000028000205740000' connect_volume /usr/lib/python3.6/site-packages/os_brick/initiator/connectors/fibre_channel.py:252

In the function '_get_possible_devices'(os_brick/initiator/connectors/fibre_channel.py), os-brick do a matrix math on hba and target wwn returned by cinder.
So when the device pci-0000:00:09.0-fc-0x500000e0da000235-lun-1 is already on the compute node, it will be used directy to be attached although it is not what we expect.

I think it should figure out the device path by initiator_target_lun_map instead of do a matrix math on hba and target wwn.

[Sofia Enriquez (lsofia-enriquez)]

Hello XuQi, hope this message finds you well.

Would you mind sharing more information regarding this case:
- like mention of the actual devices being used.
- are you using multipathing?
- are you using FC Zoning?
- which driver are you using?

Thanks
Sofia

[XuQi (xu-inori)]

Hi Sofia, Thanks for your reply.

- Actual devices being used are
    pci-0000:00:09.0-fc-0x500000e0da000235-lun-1
    pci-0000:00:0a.0-fc-0x500000e0da000224-lun-1
- Mutipath is used
- "zoning_mode" is set to none in cinder.conf
- I use Fujitsu FC Driver for cinder volume and Libvirt Driver for nova.

In Summary, each of the HBA can access a different lun through two different ports, so in function '_get_possible_devices', os-brick scans a wrong device path although cinder volume driver returned the correct value in "initiator_target_map".

Regards,
Xu Qi.