This issue is seen with pypowervm 1.1.15
In multi-path flow [0] to [3], the deploys are failing as shown below because in the analysis and conversation with VIOS team found that, for each iqn in the list of iqn(s) passed at [1], the hdisk discovery at VIOS(i.e at [3]) is taking more that 60 secs and below 120 secs. For this reason we need to have a dynamic timeout based on the number of iqn(s) at [3] i.e. minimum time(120 secs) * len(iqn).
[0] https://github.com/openstack/nova-powervm/blob/master/nova_powervm/virt/powervm/volume/iscsi.py#L232
[1] https://github.com/openstack/nova-powervm/blob/master/nova_powervm/virt/powervm/volume/iscsi.py#L194
[2] https://github.com/openstack/nova-powervm/blob/master/nova_powervm/virt/powervm/volume/iscsi.py#L203-L209
[3] https://github.com/powervm/pypowervm/blob/master/pypowervm/tasks/hdisk/_iscsi.py#L158
2018-04-26 15:51:57.710 46801 WARNING pypowervm.wrappers.job [req-587a9011-a017-4e4d-b8de-194a6c0b966a 11bcdc55656c5cb8586f565b852fc261de73ad6a36a91f6e6f1b8d78298eb040 07b0a20ee6634287a82d3e0a1425df16 - 5dccd48e66cc4134b55072487518b9b7 5dccd48e66cc4134b55072487518b9b7] Issuing cancel request for job 1524769669252. Will poll the job indefinitely for termination.
2018-04-26 15:51:57.720 46801 ERROR pypowervm.wrappers.job [req-587a9011-a017-4e4d-b8de-194a6c0b966a 11bcdc55656c5cb8586f565b852fc261de73ad6a36a91f6e6f1b8d78298eb040 07b0a20ee6634287a82d3e0a1425df16 - 5dccd48e66cc4134b55072487518b9b7 5dccd48e66cc4134b55072487518b9b7] The 'ISCSIDiscovery' operation failed. Failed to complete the task in 120 seconds.
2018-04-26 15:51:57.721 46801 WARNING nova_powervm.virt.powervm.volume.iscsi [req-587a9011-a017-4e4d-b8de-194a6c0b966a 11bcdc55656c5cb8586f565b852fc261de73ad6a36a91f6e6f1b8d78298eb040 07b0a20ee6634287a82d3e0a1425df16 - 5dccd48e66cc4134b55072487518b9b7 5dccd48e66cc4134b55072487518b9b7] iSCSI discovery on VIOS 2AFA9DCD-7CD3-49B5-9443-DECC425D0F31 failed with error: The 'ISCSIDiscovery' operation failed. Failed to complete the task in 120 seconds.: JobRequestTimedOut: The 'ISCSIDiscovery' operation failed. Failed to complete the task in 120 seconds.
2018-04-26 15:51:57.725 46801 WARNING pypowervm.wrappers.job [req-587a9011-a017-4e4d-b8de-194a6c0b966a 11bcdc55656c5cb8586f565b852fc261de73ad6a36a91f6e6f1b8d78298eb040 07b0a20ee6634287a82d3e0a1425df16 - 5dccd48e66cc4134b55072487518b9b7 5dccd48e66cc4134b55072487518b9b7] Issuing cancel request for job 1524769669253. Will poll the job indefinitely for termination.
2018-04-26 15:51:57.733 46801 ERROR pypowervm.wrappers.job [req-587a9011-a017-4e4d-b8de-194a6c0b966a 11bcdc55656c5cb8586f565b852fc261de73ad6a36a91f6e6f1b8d78298eb040 07b0a20ee6634287a82d3e0a1425df16 - 5dccd48e66cc4134b55072487518b9b7 5dccd48e66cc4134b55072487518b9b7] The 'ISCSIDiscovery' operation failed. Failed to complete the task in 120 seconds.
2018-04-26 15:51:57.734 46801 WARNING nova_powervm.virt.powervm.volume.iscsi [req-587a9011-a017-4e4d-b8de-194a6c0b966a 11bcdc55656c5cb8586f565b852fc261de73ad6a36a91f6e6f1b8d78298eb040 07b0a20ee6634287a82d3e0a1425df16 - 5dccd48e66cc4134b55072487518b9b7 5dccd48e66cc4134b55072487518b9b7] iSCSI discovery on VIOS 69794FAE-4864-4F83-958E-6FD40EEBA06E failed with error: The 'ISCSIDiscovery' operation failed. Failed to complete the task in 120 seconds.: JobRequestTimedOut: The 'ISCSIDiscovery' operation failed. Failed to complete the task in 120 seconds.
2018-04-26 15:51:57.855 46801 ERROR powervc_nova.virt.ibmpowervm.pvm.volume.iscsi [req-587a9011-a017-4e4d-b8de-194a6c0b966a 11bcdc55656c5cb8586f565b852fc261de73ad6a36a91f6e6f1b8d78298eb040 07b0a20ee6634287a82d3e0a1425df16 - 5dccd48e66cc4134b55072487518b9b7 5dccd48e66cc4134b55072487518b9b7] Failure to connect the volume on required number of vioses
Changes are made to dynamically calculate the timeout value based on the IQN