[IBM Storwize SVC]: use system_id in mkrelationship

[nikeshkm]

IBM_Storwize:Cluster_9.113.56.251:superuser>lspartnership
id name location partnership type cluster_ip event_log_sequence
00000200B28008E8 Cluster_9.113.56.251 local
000000C02A429374 Cluster_9.113.57.140 remote not_present fc
000000C02A629374 Cluster_9.113.57.140 remote fully_configured fc

In above case, creating a GMCV volume giving error :

2021-08-06 03:43:46.403 619812 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2021-08-06 03:43:46.403 619812 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 186, in _run_ssh
2021-08-06 03:43:46.403 619812 ERROR oslo_messaging.rpc.server return self._ssh(ssh_cmd)
2021-08-06 03:43:46.403 619812 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3537, in _run_ssh
2021-08-06 03:43:46.403 619812 ERROR oslo_messaging.rpc.server command)
2021-08-06 03:43:46.403 619812 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in __exit__
2021-08-06 03:43:46.403 619812 ERROR oslo_messaging.rpc.server self.force_reraise()
2021-08-06 03:43:46.403 619812 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in force_reraise
2021-08-06 03:43:46.403 619812 ERROR oslo_messaging.rpc.server raise self.value
2021-08-06 03:43:46.403 619812 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3514, in _run_ssh
2021-08-06 03:43:46.403 619812 ERROR oslo_messaging.rpc.server check_exit_code, attempts)
2021-08-06 03:43:46.403 619812 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/cinder/utils.py", line 640, in _wrapper
2021-08-06 03:43:46.403 619812 ERROR oslo_messaging.rpc.server return r.call(f, *args, **kwargs)
2021-08-06 03:43:46.403 619812 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 411, in call
2021-08-06 03:43:46.403 619812 ERROR oslo_messaging.rpc.server return self.__call__(*args, **kwargs)
2021-08-06 03:43:46.403 619812 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 423, in __call__
2021-08-06 03:43:46.403 619812 ERROR oslo_messaging.rpc.server do = self.iter(retry_state=retry_state)
2021-08-06 03:43:46.403 619812 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 360, in iter
2021-08-06 03:43:46.403 619812 ERROR oslo_messaging.rpc.server return fut.result()
2021-08-06 03:43:46.403 619812 ERROR oslo_messaging.rpc.server File "/usr/lib64/python3.6/concurrent/futures/_base.py", line 425, in result
2021-08-06 03:43:46.403 619812 ERROR oslo_messaging.rpc.server return self.__get_result()
2021-08-06 03:43:46.403 619812 ERROR oslo_messaging.rpc.server File "/usr/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result
2021-08-06 03:43:46.403 619812 ERROR oslo_messaging.rpc.server raise self._exception
2021-08-06 03:43:46.403 619812 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 426, in __call__
2021-08-06 03:43:46.403 619812 ERROR oslo_messaging.rpc.server result = fn(*args, **kwargs)
2021-08-06 03:43:46.403 619812 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/powervc_cinder/volume/drivers/storwize_svc/storwize_svc_common_product.py", line 763, in _ssh_execute
2021-08-06 03:43:46.403 619812 ERROR oslo_messaging.rpc.server self)._ssh_execute(*args, **kwargs)
2021-08-06 03:43:46.403 619812 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3594, in _ssh_execute
2021-08-06 03:43:46.403 619812 ERROR oslo_messaging.rpc.server LOG.error("Error running SSH command: %s", command)
2021-08-06 03:43:46.403 619812 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in __exit__
2021-08-06 03:43:46.403 619812 ERROR oslo_messaging.rpc.server self.force_reraise()
2021-08-06 03:43:46.403 619812 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in force_reraise
2021-08-06 03:43:46.403 619812 ERROR oslo_messaging.rpc.server raise self.value
2021-08-06 03:43:46.403 619812 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3584, in _ssh_execute
2021-08-06 03:43:46.403 619812 ERROR oslo_messaging.rpc.server cmd=last_exception.cmd)
2021-08-06 03:43:46.403 619812 ERROR oslo_messaging.rpc.server oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.
2021-08-06 03:43:46.403 619812 ERROR oslo_messaging.rpc.server Command: svctask mkrcrelationship -master volume-nik-gmcv-1-7b968f81-a12b -aux aux_volume-nik-gmcv-1-7b968f81-a12b -cluster Cluster_9.113.57.140 -global
2021-08-06 03:43:46.403 619812 ERROR oslo_messaging.rpc.server Exit code: 1
2021-08-06 03:43:46.403 619812 ERROR oslo_messaging.rpc.server Stdout: ''
2021-08-06 03:43:46.403 619812 ERROR oslo_messaging.rpc.server Stderr: 'CMMVC6024E The auxiliary volume entered is invalid.\n'
2021-08-06 03:43:46.403 619812 ERROR oslo_messaging.rpc.server

Solution is to use id (000000C02A629374) rather than name (Cluster_9.113.57.140) in mkrcrelationship:
svctask mkrcrelationship -master volume-nik-gmcv-1-7b968f81-a12b -aux aux_volume-nik-gmcv-1-7b968f81-a12b -cluster 000000C02A629374 -global

[Sofia Enriquez (lsofia-enriquez)]

Greetings,

Just to add more information for other cinder members and contributions I'm gonna leave the next IBM document that details changes to supported configurations, and best practice, for the Global Mirror with Change Volumes feature (GMCV)[1]. I think It would be nice to have a more descriptive description next time because it would help to understand the problem faster.

- For the last line of the bug description I assume this gets fixed after running 'svctask mkrcrelationship -master volume-nik-gmcv-1-7b968f81-a12b -aux aux_volume-nik-gmcv-1-7b968f81-a12b -cluster 000000C02A629374 -global', is this correct?
In case it is, can I close this bug as fixed or do we need code fix for this?

- I guess you are facing this problem in master, or are you using any other cinder version?

- I apologize if I'm asking something obvious but I assume from your description that you're facing this every time you're creating a plain volume (not a clone-volume or volume from a snapshot). Is this correct?

Thanks in advance,
Sofia
[1] https://www.ibm.com/support/pages/changes-support-global-mirror-change-volumes

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/cinder/+/809830

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/cinder/+/809830
Committed: https://opendev.org/openstack/cinder/commit/cefba8640842ab9cb479533be24688ae62d7e9bb
Submitter: "Zuul (22348)"
Branch: master

commit cefba8640842ab9cb479533be24688ae62d7e9bb
Author: Venkata Krishna <email address hidden>
Date: Fri Sep 17 16:19:57 2021 +0000

    [SVF] Fix rccg and relationship creation issues

    [Spectrum Virtualize Family] Fix issues while creating a mirror
    volume and creating a group by using the storage backend system_id
    in the place of system_name in create_relationship and create_rccg
    calls to choose the correct partnership based on the system_id.

    This issue occurs when multiple partnerships in different states
    are available for the same storage system_name.

    Closes-Bug: #1939145

    Change-Id: I9391a1a07aac00a3f80bf19cd4887bdfbec78b20

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/cinder 20.0.0.0rc1 release candidate.