iscsiadm cmd failure

Bug #832200 reported by Vladimir Popovski
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Medium
Vish Ishaya

Bug Description

iscsiadm call fails in multiple different scenarios possibly due to wrong parameter format passed to _run_iscsiadm:

2011-07-27 21:10:47,141 DEBUG nova.utils [-] Running cmd (subprocess): sudo iscsiadm -m node -T iqn.2011-04.org.zadara:vsa-00000005:volume-00000005 -p 10.0.0.19:3260 ('--op', 'new') from (pid=28764) execute /mnt/share/d-cloud-Vlad/nova/utils.py:143
2011-07-27 21:10:47,153 DEBUG nova.utils [-] Result was 255 from (pid=28764) execute /mnt/share/d-cloud-Vlad/nova/utils.py:161
2011-07-27 21:10:47,153 ERROR nova [-] Exception during message handling
(nova): TRACE: Traceback (most recent call last):
(nova): TRACE: File "/mnt/share/d-cloud-Vlad/nova/rpc.py", line 232, in _process_data
(nova): TRACE: rval = node_func(context=ctxt, **node_args)
(nova): TRACE: File "/mnt/share/d-cloud-Vlad/nova/compute/manager.py", line 113, in decorated_function
(nova): TRACE: function(self, context, instance_id, *args, **kwargs)
(nova): TRACE: File "/mnt/share/d-cloud-Vlad/nova/compute/manager.py", line 1055, in attach_volume
(nova): TRACE: volume_id)
(nova): TRACE: File "/mnt/share/d-cloud-Vlad/nova/volume/manager.py", line 234, in setup_compute_volume
(nova): TRACE: path = self.driver.discover_volume(context, volume_ref)
(nova): TRACE: File "/mnt/share/d-cloud-Vlad/nova/volume/driver.py", line 510, in discover_volume
(nova): TRACE: self._run_iscsiadm(iscsi_properties, ('--op', 'new'))
(nova): TRACE: File "/mnt/share/d-cloud-Vlad/nova/volume/driver.py", line 495, in _run_iscsiadm
(nova): TRACE: iscsi_command)
(nova): TRACE: File "/mnt/share/d-cloud-Vlad/nova/utils.py", line 169, in execute
(nova): TRACE: cmd=' '.join(cmd))
(nova): TRACE: ProcessExecutionError: Unexpected error while running command.
(nova): TRACE: Command: sudo iscsiadm -m node -T iqn.2011-04.org.zadara:vsa-00000005:volume-00000005 -p 10.0.0.19:3260 ('--op', 'new')
(nova): TRACE: Exit code: 255
(nova): TRACE: Stdout: ''
(nova): TRACE: Stderr: 'iscsiadm: no records found!\n'
(nova): TRACE:

(Same code with '--op=new' works good)

or...

2011-08-22 23:38:01,922 DEBUG nova.utils [-] Running cmd (subprocess): sudo iscsiadm -m node -T iqn.2010-10.org.openstack:volume-000000d8 -p 10.0.0.11:3260 ('--op', 'delete') from (pid=2366) execute /mnt/share/d-cloud-Vlad/nova/utils.py:164
2011-08-22 23:38:01,938 DEBUG nova.volume.driver [-] iscsiadm ('--op', 'delete'): stdout=# BEGIN RECORD 2.0-871
node.name = iqn.2010-10.org.openstack:volume-000000d8
node.tpgt = 1
node.startup = manual
iface.hwaddress = <empty>
iface.ipaddress = <empty>
iface.iscsi_ifacename = default
iface.net_ifacename = <empty>
iface.transport_name = tcp
iface.initiatorname = <empty>
node.discovery_address = ubuntu-alex-srv
node.discovery_port = 3260
node.discovery_type = send_targets
node.session.initial_cmdsn = 0
node.session.initial_login_retry_max = 8
node.session.xmit_thread_priority = -20
node.session.cmds_max = 128
node.session.queue_depth = 32
node.session.auth.authmethod = None
node.session.auth.username = <empty>
node.session.auth.password = <empty>
node.session.auth.username_in = <empty>
node.session.auth.password_in = <empty>
node.session.timeo.replacement_timeout = 120
node.session.err_timeo.abort_timeout = 15
node.session.err_timeo.lu_reset_timeout = 20
node.session.err_timeo.host_reset_timeout = 60
node.session.iscsi.FastAbort = Yes
node.session.iscsi.InitialR2T = No
node.session.iscsi.ImmediateData = Yes
node.session.iscsi.FirstBurstLength = 262144
node.session.iscsi.MaxBurstLength = 16776192
node.session.iscsi.DefaultTime2Retain = 0
node.session.iscsi.DefaultTime2Wait = 2
node.session.iscsi.MaxConnections = 1
node.session.iscsi.MaxOutstandingR2T = 1
node.session.iscsi.ERL = 0
node.conn[0].address = 10.0.0.11
node.conn[0].port = 3260
node.conn[0].startup = manual
node.conn[0].tcp.window_size = 524288
node.conn[0].tcp.type_of_service = 0
node.conn[0].timeo.logout_timeout = 15
node.conn[0].timeo.login_timeout = 15
node.conn[0].timeo.auth_timeout = 45
node.conn[0].timeo.noop_out_interval = 5
node.conn[0].timeo.noop_out_timeout = 5
node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144

node.conn[0].iscsi.HeaderDigest = None
node.conn[0].iscsi.DataDigest = None
node.conn[0].iscsi.IFMarker = No
node.conn[0].iscsi.OFMarker = No
# END RECORD
 stderr= from (pid=2366) _run_iscsiadm /mnt/share/d-cloud-Vlad/nova/volume/driver.py:506

As a result iscsiadm doing nothing.

For op new & delete we can probably change the code to pass something like '--op=new' and '--op=delete' instead of ('--op', 'new') & ('--op', 'delete'), but update receives multiple params.

Possible workaround might be also to add something like that within _run_iscsiadm:

iscsi_command=map(str, iscsi_command)

Related branches

description: updated
Revision history for this message
Vish Ishaya (vishvananda) wrote :

I fixed this in my cleanup volume branch by changing the call to execute to unpack the tuple using an asterix:
*iscsi_command

Changed in nova:
assignee: nobody → Vish Ishaya (vishvananda)
status: New → In Progress
importance: Undecided → Medium
Changed in nova:
status: In Progress → Fix Committed
Revision history for this message
Vish Ishaya (vishvananda) wrote :

missed changing all run_iscsiadm commands to use tuples.

Changed in nova:
status: Fix Committed → Triaged
status: Triaged → In Progress
Thierry Carrez (ttx)
Changed in nova:
milestone: none → diablo-4
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in nova:
milestone: diablo-4 → 2011.3
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.