Activity log for bug #1443977

Date Who What changed Old value New value Message
2015-04-14 14:15:49 Loren Erwin bug added bug
2015-04-14 14:15:49 Loren Erwin attachment added This is a customer's attempt at a patch. https://bugs.launchpad.net/bugs/1443977/+attachment/4375260/+files/volume.py
2015-04-14 18:42:55 Mike Scherbakov fuel: milestone 6.1
2015-04-14 18:43:01 Mike Scherbakov tags customer-found
2015-04-14 18:56:29 Roman Alekseenkov bug task added mos
2015-04-14 18:56:40 Roman Alekseenkov bug task deleted fuel
2015-04-14 18:56:49 Roman Alekseenkov mos: milestone 6.1
2015-04-15 08:52:32 Roman Podoliaka description Customer: When multipath is enabled with Nova, a different codepath is executed for most functions, such as connect_volume and disconnect_volume. A series of loops exist within connect_volume when multipath is enabled An iscsiadm discover is ran against the portal information associated with the instance, in our case a portal on the nimble storage array. For every iqn discovered, _connect_to_iscsi_portal runs and logins to the portal. In our test scenario, with 183 instances - this results in 183*2 IQN targets being discovered, and logged into, every single time an instance is launched on that node. With multiple instance creation this compounds. These calls can take upwards of 1 second to run * the number of times. Additionally there seems to be other code paths with similar loops with iSCSI behaviour. We may have just addressed the connect_volume loop as well. The IQN of the iSCSI device is always the same, although the portal may be different. iscsi_properties['target_iqn'] contains the target_iqn. Instead of looping through every single IQN to login to, the code now only logs into the IQN's it needs. such as for ip, iqn in self._get_target_portals_from_iscsiadm_output(out): if iqn in iscsi_properties['target_iqn']: props = iscsi_properties.copy() props['target_portal'] = ip props['target_iqn'] = iqn self._connect_to_iscsi_portal(props) Upstream bugs: https://bugs.launchpad.net/nova/+bug/1277316 and https://bugs.launchpad.net/nova/+bug/1382440 Customer: When multipath is enabled with Nova, a different codepath is executed for most functions, such as connect_volume and disconnect_volume. A series of loops exist within connect_volume when multipath is enabled An iscsiadm discover is ran against the portal information associated with the instance, in our case a portal on the nimble storage array. For every iqn discovered, _connect_to_iscsi_portal runs and logins to the portal. In our test scenario, with 183 instances - this results in 183*2 IQN targets being discovered, and logged into, every single time an instance is launched on that node. With multiple instance creation this compounds. These calls can take upwards of 1 second to run * the number of times. Additionally there seems to be other code paths with similar loops with iSCSI behaviour. We may have just addressed the connect_volume loop as well. The IQN of the iSCSI device is always the same, although the portal may be different. iscsi_properties['target_iqn'] contains the target_iqn. Instead of looping through every single IQN to login to, the code now only logs into the IQN's it needs. such as for ip, iqn in self._get_target_portals_from_iscsiadm_output(out): if iqn in iscsi_properties['target_iqn']: props = iscsi_properties.copy() props['target_portal'] = ip props['target_iqn'] = iqn self._connect_to_iscsi_portal(props)
2015-04-15 08:52:48 Roman Podoliaka nominated for series mos/7.0.x
2015-04-15 08:52:48 Roman Podoliaka bug task added mos/7.0.x
2015-04-15 08:52:48 Roman Podoliaka nominated for series mos/6.0.x
2015-04-15 08:52:48 Roman Podoliaka bug task added mos/6.0.x
2015-04-15 08:52:48 Roman Podoliaka nominated for series mos/6.1.x
2015-04-15 08:52:48 Roman Podoliaka bug task added mos/6.1.x
2015-04-15 08:52:55 Roman Podoliaka mos/6.0.x: assignee MOS Nova (mos-nova)
2015-04-15 08:52:59 Roman Podoliaka mos/7.0.x: assignee MOS Nova (mos-nova)
2015-04-15 08:53:06 Roman Podoliaka mos/6.1.x: assignee MOS Nova (mos-nova)
2015-04-15 08:53:13 Roman Podoliaka tags customer-found customer-found nova
2015-04-15 08:53:17 Roman Podoliaka mos/6.0.x: milestone 6.0.1
2015-04-15 08:53:20 Roman Podoliaka mos/7.0.x: milestone 7.0
2015-04-15 08:53:26 Roman Podoliaka mos/6.0.x: status New Triaged
2015-04-15 08:53:29 Roman Podoliaka mos/6.1.x: status New Triaged
2015-04-15 08:53:30 Roman Podoliaka mos/7.0.x: status New Triaged
2015-04-15 08:53:34 Roman Podoliaka mos/6.0.x: importance Undecided High
2015-04-15 08:53:35 Roman Podoliaka mos/6.1.x: importance Undecided High
2015-04-15 08:53:37 Roman Podoliaka mos/7.0.x: importance Undecided High
2015-04-16 09:15:05 Roman Podoliaka mos/7.0.x: assignee MOS Nova (mos-nova) Andrey Kurilin (akurilin)
2015-04-16 09:15:10 Roman Podoliaka mos/6.1.x: assignee MOS Nova (mos-nova) Andrey Kurilin (akurilin)
2015-04-16 09:15:16 Roman Podoliaka mos/6.0.x: assignee MOS Nova (mos-nova) Andrey Kurilin (akurilin)
2015-04-24 14:33:58 Vitaly Sedelnik nominated for series mos/6.0-updates
2015-04-24 14:33:58 Vitaly Sedelnik bug task added mos/6.0-updates
2015-04-24 14:34:04 Vitaly Sedelnik mos/6.0-updates: status New In Progress
2015-04-24 14:34:07 Vitaly Sedelnik mos/6.0-updates: importance Undecided High
2015-04-24 14:34:15 Vitaly Sedelnik mos/6.0-updates: assignee Alexander Nevenchannyy (anevenchannyy)
2015-04-24 14:34:20 Vitaly Sedelnik mos/6.0-updates: milestone 6.0-mu-3
2015-04-30 10:29:40 Vitaly Sedelnik mos/6.0-updates: status In Progress Fix Committed
2015-05-08 11:47:58 Andriy Kurilin mos/6.0.x: status Triaged In Progress
2015-05-08 11:48:01 Andriy Kurilin mos/6.1.x: status Triaged In Progress
2015-05-12 08:53:29 Roman Podoliaka mos/6.1.x: status In Progress Fix Committed
2015-05-14 17:13:15 Timur Nurlygayanov summary Mutipath performance issues: issues with connect_volume and similar fucntions Mutipath performance issues: issues with connect_volume and similar functions
2015-05-20 15:53:12 Vitaly Sedelnik mos/6.0-updates: status Fix Committed Fix Released
2015-05-28 08:56:16 Kyrylo Romanenko tags customer-found nova customer-found nova on-verification
2015-07-17 13:07:44 Roman Podoliaka mos/7.0.x: status Triaged Fix Committed
2015-09-26 13:19:53 Vitaly Sedelnik mos/6.0.x: status In Progress Fix Released
2015-09-26 13:20:00 Vitaly Sedelnik mos/6.0.x: assignee Andrey Kurilin (andreykurilin) Alexander Nevenchannyy (anevenchannyy)
2015-09-26 13:20:04 Vitaly Sedelnik mos/6.0.x: milestone 6.0-updates 6.0-mu-3
2015-10-06 15:15:53 Ksenia Svechnikova tags customer-found nova on-verification customer-found nova
2015-11-04 12:09:15 Kyrylo Romanenko tags customer-found nova customer-found nova on-verification
2015-11-05 16:57:05 Kyrylo Romanenko mos/7.0.x: status Fix Committed Confirmed
2015-11-05 16:59:00 Timur Nurlygayanov mos/7.0.x: milestone 7.0 7.0-updates
2015-12-03 14:20:51 Andriy Kurilin mos/7.0.x: status Confirmed Fix Committed
2015-12-10 16:03:30 Alexander Gubanov mos/7.0.x: status Fix Committed Fix Released
2015-12-10 16:03:34 Alexander Gubanov mos: status Fix Committed Fix Released