open-iscsi should not leave around the stamp file if no connections were made

Bug #1800681 reported by Dimitri John Ledkov
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
open-iscsi (Ubuntu)
Fix Released
Undecided
Unassigned
Bionic
Fix Released
Undecided
Unassigned
Cosmic
Won't Fix
Undecided
Unassigned
Disco
Fix Released
Undecided
Unassigned

Bug Description

Separately, ship finalrd hook which cloud images can switch to using.
By default, it is inert, as finalrd itself is not pulled in.

---

[Impact]

 * Shutdown is delayed by 30s when open-iscsi is configured to auto mode but no iscsi devices were found.

[Test Case]

(1) * Start a PV (non-iscsi) instance in Oracle Cloud (https://www.oracle.com/cloud/)
 * Monitor the instance's console.
 * Shut down the instance
 * Observe the shutdown proceeding in a timely manner without the following printouts:

[ 851.308389] sd-umoun[2880]: Failed to unmount /oldroot: Device or resource busy
[ 851.311743] sd-umoun[2881]: Failed to unmount /oldroot/dev: Device or resource busy
iscsiadm: No active sessions.
iscsiadm: No active sessions.
iscsiadm: No active sessions.
iscsiadm: No active sessions.
iscsiadm: No active sessions.
iscsiadm: No active sessions.
iscsiadm: No active sessions.
iscsiadm: No active sessions.
iscsiadm: No active sessions.
iscsiadm: No active sessions.
iscsiadm: No active sessions.
iscsiadm: No active sessions.
iscsiadm: No active sessions.
iscsiadm: No active sessions.
iscsiadm: No active sessions.
iscsiadm: No active sessions.
iscsiadm: No active sessions.
iscsiadm: No active sessions.
iscsiadm: No active sessions.
iscsiadm: No active sessions.
iscsiadm: No active sessions.
iscsiadm: No active sessions.
iscsiadm: No active sessions.
iscsiadm: No active sessions.
iscsiadm: No active sessions.
iscsiadm: No active sessions.
iscsiadm: No active sessions.
iscsiadm: No active sessions.
iscsiadm: No active sessions.
iscsiadm: No active sessions.
iscsiadm: No active sessions.
Unexpected iSCSI Connection State, forcing iSCSI logout.
iscsiadm: No matching sessions found
[ 881.434458] sd-execu[2882]: /usr/lib/systemd/system-shutdown/open-iscsi.finalrd failed with exit status 21.
[ 881.438313] shutdown[1]: Failed to finalize file systems, ignoring

(2) * Start an iSCSI instance in Oracle Cloud (https://www.oracle.com/cloud/)
 * Monitor the instance's console.
 * Restart the instance
 * Observe the instance rebooting properly

[Regression Potential]

 * The fix is removing /run/initramfs/open-iscsi.interface during running initramfs scripts if not iscsi disks were found. In case not detecting an iscsi device but removing the .interface file the system may reconfigure the network interface later losing connection to the iscsi device.

[Original Bug Text]

open-iscsi should not leave around the stamp file if no connections were made

open-iscsi finalrd hook is sensitive to /run/initramfs/open-iscsi.interface as an indicator that rootfs might have been an iscsi mount, and thus logout needs to happen inside finalrd.

However, in auto-mode, and without any iscsi hard-drives found, that stamp file is still left around. It should be cleaned up, such that such boots do not result in 30s shutdown delay.

Proposed patch is this:

/usr/share/initramfs-tools/scripts/local-top$ diff -u iscsi.orig iscsi
--- iscsi.orig 2018-10-30 15:53:36.314256545 +0000
+++ iscsi 2018-10-30 15:53:02.883686328 +0000
@@ -360,4 +360,19 @@
  done
 fi

+# Remove the interface file if no disks are present
+if [ -f /run/initramfs/open-iscsi.interface ] ; then
+ found=0
+ for disk in /dev/disk/by-path/*-iscsi-*; do
+ if ! "$(readlink -f "$disk")" ; then
+ continue
+ fi
+ found=1
+ break;
+ done
+ if [ $found = 0 ] ; then
+ rm /run/initramfs/open-iscsi.interface
+ fi
+fi
+
 exit 0

tags: added: id-5bbe5cb1a3a04d69ef6d9037
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package open-iscsi - 2.0.874-5ubuntu13

---------------
open-iscsi (2.0.874-5ubuntu13) disco; urgency=medium

  [ Robert C Jennings & Dimitri John Ledkov ]
  * If iscsi root was requested, but no iscsi devices were mounted, remove
    the initramfs interfaces stamp file. Meaning, that on shutdown there
    is no 30s delay, whilst trying to re-establish iscsi login to perform
    a logout. LP: #1800681

 -- Dimitri John Ledkov <email address hidden> Sun, 09 Dec 2018 22:29:20 +0000

Changed in open-iscsi (Ubuntu Disco):
status: New → Fix Released
Revision history for this message
David Krauser (davidkrauser) wrote :

I tested this fix on disco, and it appears to be working as intended.

The issue still exists on bionic, however.

Balint Reczey (rbalint)
Changed in open-iscsi (Ubuntu Cosmic):
status: New → Won't Fix
Balint Reczey (rbalint)
description: updated
Balint Reczey (rbalint)
description: updated
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Please test proposed package

Hello Dimitri, or anyone else affected,

Accepted open-iscsi into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/open-iscsi/2.0.874-5ubuntu2.8 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in open-iscsi (Ubuntu Bionic):
status: New → Fix Committed
tags: added: verification-needed verification-needed-bionic
Balint Reczey (rbalint)
description: updated
Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (open-iscsi/2.0.874-5ubuntu2.8)

All autopkgtests for the newly accepted open-iscsi (2.0.874-5ubuntu2.8) for bionic have finished running.
The following regressions have been reported in tests triggered by the package:

nova/2:17.0.11-0ubuntu1 (armhf)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/bionic/update_excuses.html#open-iscsi

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

description: updated
Revision history for this message
Timo Aaltonen (tjaalton) wrote : Please test proposed package

Hello Dimitri, or anyone else affected,

Accepted open-iscsi into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/open-iscsi/2.0.874-5ubuntu2.9 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Revision history for this message
Brian Murray (brian-murray) wrote : [open-iscsi/bionic] verification still needed

The fix for this bug has been awaiting testing feedback in the -proposed repository for bionic for more than 90 days. Please test this fix and update the bug appropriately with the results. In the event that the fix for this bug is still not verified 15 days from now, the package will be removed from the -proposed repository.

tags: added: removal-candidate
Revision history for this message
David Krauser (davidkrauser) wrote :

I built a test bionic image with open-iscsi from proposed, uploaded to OCI, booted on a PV instance, then performed a reboot. That reboot worked as expected without any scsi-related delays.

tags: added: verification-done
removed: verification-needed verification-needed-bionic
Balint Reczey (rbalint)
tags: added: verification-done-bionic
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package open-iscsi - 2.0.874-5ubuntu2.9

---------------
open-iscsi (2.0.874-5ubuntu2.9) bionic; urgency=medium

  * Ship open-iscsi finalrd hook. LP: #1800681

open-iscsi (2.0.874-5ubuntu2.8) bionic; urgency=medium

  [ Robert C Jennings & Dimitri John Ledkov ]
  * If iscsi root was requested, but no iscsi devices were mounted, remove
    the initramfs interfaces stamp file. Meaning, that on shutdown there
    is no 30s delay, whilst trying to re-establish iscsi login to perform
    a logout. LP: #1800681

 -- Dimitri John Ledkov <email address hidden> Tue, 26 Nov 2019 14:56:20 +0000

Changed in open-iscsi (Ubuntu Bionic):
status: Fix Committed → Fix Released
Revision history for this message
Brian Murray (brian-murray) wrote : Update Released

The verification of the Stable Release Update for open-iscsi has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.