Comment 70 for bug 1569925

Revision history for this message
Rafael David Tinoco (rafaeldtinoco) wrote :

This is what I discovered:

This systemd issue is caused because there are iscsi connections still logged in (not because of multipath, bonding, vlans, etc). One way of reproducing this is by simply editing the open-iscsi.service unit and disabling the logout script. It will, for me in all times, cause the same hang as we are talking to.

inaddy@iscsihang:~$ sudo systemctl edit --full open-iscsi.service

...
ExecStop=/lib/open-iscsi/umountiscsi.sh
ExecStop=/bin/sync
#ExecStop=/lib/open-iscsi/logout-all.sh

But of course no one is doing this. So, reading logout-all you will discover that it logs out all paths but it has some caveats: ISCSI_ROOT_KEEP_ALL_SESSIONS_AT_SHUTDOWN (from /etc/default/open-iscsi) and /run/open-iscsi/shutdown-keep-sessions would cause those iscsi sessions not to be logged out.

Still, I wasn't using neither to reproduce so I found out something else. Disabling both systemd services:

inaddy@iscsihang:~$ systemctl list-unit-files --full | grep iscsi
iscsi.service disabled (alias for iscsid)
iscsid.service disabled
open-iscsi.service disabled

I was still able to reproduce the issue:

http://pastebin.ubuntu.com/25388348/

By issuing a logout when the iscsid daemon is gone. It looks like iscsid is lost after it has died/been killed on the sessions that were previously established. If you try to login again, it will be totally lost when trying to logout, causing leftovers (opened iscsi sessions) causing systemd to hang (even not having daemons running, in this case).

I'm investigating:

- Should systemd hang on those leftovers ? Since there are caveats that explicitly allow you to leave opened sessions, for / for example, I doubt it.

- Does iscsid daemon contain a bug that causes sessions to remain un-mapped and opened if daemon is restarted without a logout of previously opened sessions ?

Now that I'm able to reproduce this at will, I'll be able to answer those questions soon.

Best,
Rafael

PS: Matt, I asked partnership program managers to reach out to understand better your future needs.