iSCSI target administration utility (ietadm) not detecting inactive sessions

[John Strunk]

[Impact]
Administrators will not be able to delete targets even when all sessions are inactive.

[Test Case]
(script attached to bug report as well)
sudo apt-get install iscsitarget open-iscsi
echo "ISCSITARGET_ENABLE=true" | sudo tee /etc/default/iscsitarget
sudo service iscsitarget restart
sudo dd if=/dev/zero of=/fs.iscsi.disk bs=1M count=512
sudo ietadm --op new --tid=1 --params=Name=mytarget
sudo ietadm --op new --tid=1 --lun=0 --params=Path=/fs.iscsi.disk
cat /proc/net/iet/session
cat /proc/net/iet/volume
sudo iscsiadm -m discovery -t st -p 127.0.0.1:3260
sudo iscsiadm -m node -T mytarget -p 127.0.0.1:3260 --login
cat /proc/net/iet/session
cat /proc/net/iet/volume
sudo iscsiadm -m session --logout
sudo ietadm --op delete --tid=1
cat /proc/net/iet/session
cat /proc/net/iet/volume

Failure is indicated by "Device or resource busy." during the delete operation.

[Regression Potential]
The fix is in the event layer of the DKMS module; it ensures that the session removal event gets to the ietd with kernels >= 3.2.

[Original Bug Report]
Using up to date Ubuntu precise (3/28/12) with:

iscsitarget - 1.4.20.2-5ubuntu2
iscsitarget-dkms - 1.4.20.2-5ubuntu2

iSCSI target administration utility appears to not detect inactive sessions correctly.

Replicating bug:

creating a sparse image:

dd if=/dev/zero of=/fs.iscsi.disk bs=1M count=512

then creating target:

root@ubuntu-precise:~# ietadm --op new --tid=1 --params=Name=mytarget
root@ubuntu-precise:~# ietadm --op new --tid=1 --lun=0 --params=Path=/fs.iscsi.disk

shows available:

root@ubuntu-precise:~# cat /proc/net/iet/session
tid:1 name:mytarget

root@ubuntu-precise:~# cat /proc/net/iet/volume
tid:1 name:mytarget
        lun:0 state:0 iotype:fileio iomode:wt blocks:1048576 blocksize:512 path:/fs.iscsi.disk

login by client is successful:

root@ubuntu-precise:~# iscsiadm -m node -T mytarget -p 172.16.2.133:3260 --login
Logging in to [iface: default, target: mytarget, portal: 172.16.2.133,3260]
Login to [iface: default, target: mytarget, portal: 172.16.2.133,3260]: successful

and shows active session:

root@ubuntu-precise:~# cat /proc/net/iet/session
tid:1 name:mytarget
        sid:281475014263296 initiator:iqn.1993-08.org.debian:01:2f524641a62d
                cid:0 ip:172.16.2.133 state:active hd:none dd:none

ietadm properly shows active and refuses to delete which is expected:

root@ubuntu-precise:~# ietadm --op delete --tid=1
Device or resource busy.

client logs out:

root@ubuntu-precise:~# iscsiadm -m session --logout
Logging out of session [sid: 2, target: mytarget, portal: 172.16.2.133,3260]
Logout of [sid: 2, target: mytarget, portal: 172.16.2.133,3260]: successful

no longer shows session active:

root@ubuntu-precise:~# cat /proc/net/iet/session
tid:1 name:mytarget

but ietadm refuses to acknowledge the disconnect and release the target and a ietadm delete fails saying "in use":

root@ubuntu-precise:~# ietadm --op delete --tid=1
Device or resource busy.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[John Strunk (johndstrunk)]

Testing from iscsitarget trunk with the update for revision 470 (released Fri Apr 6 21:58:31 2012 UTC)

http://iscsitarget.svn.sourceforge.net/viewvc/iscsitarget?revision=478&view=revision

It appears they have addressed this issue with the fix noted above. Obviously, this isn't included in the iscsitarget and iscsitarget-dkms packages in Ubuntu Precise currently. The issue notes there is a problem with kernel versions 3.2+ without the 478 fix in place.

Testing so far seems to indicate this is fixed with that update.

Testing:

creating a sparse image:

dd if=/dev/zero of=/fs.iscsi.disk bs=1M count=512

then creating target:

root@ubuntu-precise-beta2:~# ietadm --op new --tid=1 --params=Name=mytarget
root@ubuntu-precise-beta2:~# ietadm --op new --tid=1 --lun=0 --params=Path=/fs.iscsi.disk

shows available:

root@ubuntu-precise-beta2:~# cat /proc/net/iet/session
tid:1 name:mytarget

root@ubuntu-precise-beta2:~# cat /proc/net/iet/volume
tid:1 name:mytarget
        lun:0 state:0 iotype:fileio iomode:wt blocks:1048576 blocksize:512 path:/fs.iscsi.disk

login by client is successful:

root@ubuntu-precise-beta2:~# iscsiadm -m node -T mytarget -p 172.16.2.133:3260 --login
Logging in to [iface: default, target: mytarget, portal: 172.16.2.133,3260]
Login to [iface: default, target: mytarget, portal: 172.16.2.133,3260]: successful

and active session:

root@ubuntu-precise-beta2:~# cat /proc/net/iet/session
tid:1 name:mytarget
        sid:281474997486080 initiator:iqn.1993-08.org.debian:01:8f33344622fb
                cid:0 ip:172.16.2.133 state:active hd:none dd:none

properly refuses to delete in use target:

root@ubuntu-precise-beta2:~# ietadm --op delete --tid=1
Device or resource busy.

a successful logout:

root@ubuntu-precise-beta2:~# iscsiadm -m session --logout
Logging out of session [sid: 1, target: mytarget, portal: 172.16.2.133,3260]
Logout of [sid: 1, target: mytarget, portal: 172.16.2.133,3260]: successful

shows no active sessions:

root@ubuntu-precise-beta2:~# cat /proc/net/iet/session
tid:1 name:mytarget

root@ubuntu-precise-beta2:~# cat /proc/net/iet/volume
tid:1 name:mytarget
        lun:0 state:0 iotype:fileio iomode:wt blocks:1048576 blocksize:512 path:/fs.iscsi.disk

successfully lets you delete target as device is not in use:

root@ubuntu-precise-beta2:~# ietadm --op delete --tid=1

no longer shows active target or sessions:

root@ubuntu-precise-beta2:~# cat /proc/net/iet/volume
root@ubuntu-precise-beta2:~# cat /proc/net/iet/session

[John Strunk (johndstrunk)]

update above, should state revision 478

[Jan Van Winkel (vanwinkeljan)]

In case somebody need a package with revision 478 applied you can find it on my ppa
https://launchpad.net/~vanwinkeljan/+archive/ppa

[James Page (james-page)]

Fixes uploaded to quantal (currently under beta freeze so will require release team approval) and to precise-proposed for SRU team review.

[Launchpad Janitor (janitor)]

This bug was fixed in the package iscsitarget - 1.4.20.2-10ubuntu1

---------------
iscsitarget (1.4.20.2-10ubuntu1) quantal; urgency=low

  * Merge from Debian testing. Remaining changes:
    - d/iscsitarget.init: If module unloading fails after stopping ietd,
      report it but still exit zero since this doesn't justify causing
      package operations to fail.
    - d/docs: Install README.initiators and README.mcs.
    - d/p/fix-send-data-rsp.patch: Pick change @462 from upstream svn to
      prevent a assertion failure in iscsi.c@492.
    - d/p/linux-3.2-includes.patch: Add additional includes of linux/export.h
      and linux/module.h, resolving dkms build failures with > linux 3.2.
    - d/p/fix-dkms.patch: Add missing patch entries to the dkms.conf file.
  * Fix DKMS compatibility with Linux 3.5 kernel (LP: #1045706):
    - d/p/iscsitarget-svn{444,471}.patch: Cherry picked patches from
      upstream VCS trunk, resolving dkms build failures with > linux 3.5.
    - d/p/fix-linux3.5.patch: Rework usage of ioc_task_link, resolving
      dkms build failures with > linux 3.5.
  * Ensure ietadm can detect inactive sessions (LP: #967984):
    - d/p/iscsitarget-svn478.patch: Cherry picked patch from upstream
      VCS trunk which ensures that events are propagated correctly to
      ietd.

iscsitarget (1.4.20.2-10) unstable; urgency=low

  * [2bebbc2] don't prompt on modified conffiles.
    Thanks to Andreas Beckmann (Closes: #661397)
  * Upload to unstable

iscsitarget (1.4.20.2-9) experimental; urgency=low

  * [7ad9219] Clean up old sysctl knob.
    Thanks to Martin Braure de Calignon (Closes: #645582)
  * [0802fff] Use corrent message type.
    Thanks to Vincent Pelletier (Closes: #647923)
  * [dcab13f] Drop unmaintained iscsitarget-source package
  * [099cc1a] add quilt header to patch
  * [93dc585] Fix LSB Headers

iscsitarget (1.4.20.2-8) experimental; urgency=low

  * [735c872] Add patch to fix build on 3.2 kernels (Closes: #659560)

iscsitarget (1.4.20.2-7) unstable; urgency=low

  * [6ea2c52] Patch clean-up
  * [627d8e4] Enable hardened build flags.
    Thanks to Moritz Muehlenhoff (Closes: #656867)

iscsitarget (1.4.20.2-6) unstable; urgency=low

  * SVN 1.4.20@453 - Co-ordinated with upstream on the stability of the tree
  * [494142b] Drop double installation of upstream changelog.
    Thanks to Christian Pernegger (Closes: #633005)
  * [71de4f8] Add package linux-headers to Recommends.
    Thanks to George Shuklin (Closes: 636157, 628428)
  * [8344c44] Add support to provide daemon options.
    Thanks to Alfredo Sola, Robbert Rijkse (Closes: 609582, 545536)
  * [8208ade] Drop debian/30-iscsitarget.conf. (Closes: 636497)
    We should not play with kernel sysct knobs right from the package.
    Setting those values will depend on a per machine basis
  * [0e96f0e] Update Standards Version to 3.9.2. No changes required
 -- James Page <email address hidden> Tue, 04 Sep 2012 13:43:21 +0100

[Clint Byrum (clint-fewbar)]

Hello John, or anyone else affected,

Accepted iscsitarget into precise-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/iscsitarget/1.4.20.2-5ubuntu3.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please change the bug tag from verification-needed to verification-done. If it does not, change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Colin Watson (cjwatson)]

The verification of this Stable Release Update has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regresssions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package iscsitarget - 1.4.20.2-5ubuntu3.1

---------------
iscsitarget (1.4.20.2-5ubuntu3.1) precise-proposed; urgency=low

  * Ensure ietadm can detect inactive sessions allowing redundant LUN's
    and iSCSI targets to be deleted (LP: #967984):
    - d/p/iscsitarget-svn478.patch: Cherry picked patch from upstream
      VCS trunk to ensure events get propagated correctly to ietd.
 -- James Page <email address hidden> Tue, 04 Sep 2012 13:13:56 +0100