casper toram forgets to disconnect loop device

[komputes]

Binary package hint: ubiquity

Ubiquity cannot install if you are booting LiveCD directly from ISO.

Booting directly from GRUB to the ISO, mounts the ISO to /isodevice.

Ubiquity will not install (even if on separate disk) if this is not unmounted.

The result being that one cannot install Ubuntu in this method.

This method can be used to create a recovery partition or a multi-usb bootable drive. For more explanation on how to do this:

========== START ==========

    * Run the following commands which will create the directory /boot/iso and download the iso file to that directory.

$ sudo -s
# mkdir /boot/iso
# cd /boot/iso
# wget http://releases.ubuntu.com/maverick/ubuntu-10.10-desktop-i386.iso

    * Add a custom menu entry in GRUB2 configuration file. Edit /etc/grub.d/40_custom as root to look like this:

#!/bin/sh
echo "Adding 40_custom." >&2
exec tail -n +3 $0

menuentry "Maverick CD Image ISO" {
set isofile=/boot/iso/ubuntu-10.10-desktop-i386.iso
loopback loop (hd0,1)$isofile
linux (loop)/casper/vmlinuz boot=casper iso-scan/filename=$isofile noprompt noeject
initrd (loop)/casper/initrd.lz
}

    * Afterwards, run update-grub for the changes to be propagated to /boot/grub/grub.cfg

# update-grub
Generating grub.cfg ...
Found linux image: /boot/vmlinuz-...
Found initrd image: /boot/initrd.img-...

========== STOP ==========

================== Workaround =================

After casper copy to ram , it need to disconnect the loop device and unmount

losetup -d /dev/loop0
umount /isodevice

===========================================

The result of booting from the ISO directly from GRUB, is that the CD is that the partition where the ISO is located is mounted to /isodevice.

Ubiquity seems to want to have control of all disks and unmount all disks before it starts installing. In the images here you see the example is sda2 for / and sda for the MBR. I have also tried this with a secondary disk where sdb1 was to be used for / and sdb for the MBR and ubiquity still complained that it had to to unmount /isodevice, even though it is on sda1. Doesn't make much sense to unmount a disk if you will not be editing any part of it.

ProblemType: Bug
DistroRelease: Ubuntu 10.10
Package: ubiquity 2.4.8
ProcVersionSignature: Ubuntu 2.6.35-22.33-generic 2.6.35.4
Uname: Linux 2.6.35-22-generic i686
NonfreeKernelModules: wl
Architecture: i386
Date: Thu Dec 2 16:34:42 2010
LiveMediaBuild: Ubuntu 10.10 "Maverick Meerkat" - Release i386 (20101007)
ProcEnviron:
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: ubiquity

[Evan (ev)]

You need to preseed the following:

d-i partman/filter_mounted boolean false
d-i ubiquity/partman-skip-unmount boolean true

[komputes (komputes)]

How can one preseed d-i values on a LiveCD with ubiquity? Can you please provide instructions and I will test and get back to you.

[komputes (komputes)]

This no longer affects me with the latest version.

[Krainov Lev (lev-krainov)]

reproduced in kubuntu-11.10-desktop-amd64.iso , workaround with exit 0 in /lib/partman/commit.d/01unmount_busy solved the problem

[user (meta1729-deactivatedaccount)]

Present in xubuntu-14.04-desktop-i386.iso. The installer says it needs to unmount /isodevice even if NO partition changes (create/delete/resize) are requested. All that is requested is formatting of other partition as ext4.

Please reopen this bug.

[experimancer (experimancer)]

Still present in ubuntu-14.04-desktop iso, booted from grub2.

No way to continue with installation cause "/isodevice cannot be unmounted" (the operation of which is not even needed).

Not "invalid" or "fixed" not even in any of this bugs duplicates.

Wonder why this has ever been closed, even if it has been around for atleast 4 years in each and every Ubunut installatiion .iso image.?

The quality of Ubuntu distributions is getting worse and worse every year.. And seems that rven this severe installation bugs are not fixed :(

[experimancer (experimancer)]

Fix is actully here: https://help.ubuntu.com/community/Grub2/ISOBoot (i.e add "toram" to relevant Grub2 menuentry or manually umount the /isodevice with "sudo umount -lrf /isodevice" and re-start the installation.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[gregrwm (gregrwm)]

confirmed that the 'umount -lrf /isodevice' workaround works on the bionic iso of 2018/2/8. i didn't even use 'toram'.

everyone expects they need either burn a cd, use a thumb or external drive, or use another installer, eg mini.iso. with this workaround, or better yet if this bug gets fixed, none of those are necessary.

[Phillip Susi (psusi)]

Looks like after casper copies the squashfs to ram, it forgets to disconnect the loop device. The workaround is to run losetup -d /dev/loop0 and umount /isodevice.

[Daniel Richard G. (skunk)]

Philip Susi: Confirmed with the Bionic live CD:

root@xubuntu:~# cat /proc/cmdline
BOOT_IMAGE=(loop)/casper/vmlinuz boot=casper iso-scan/filename=/linux/xubuntu-18.04-desktop-amd64.iso toram

root@xubuntu:~# umount /isodevice
umount: /isodevice: target is busy.

root@xubuntu:~# losetup -d /dev/loop0

root@xubuntu:~# umount /isodevice
(succeeds)

However, I did not observe the issue as originally reported; Ubiquity was able to install the system even with /isodevice mounted and unable to be unmounted. (In my case, the ISO file was on a USB thumbdrive, and I installed to a hard drive.)

[Phillip Susi (psusi)]

Yes; the install only fails if the iso file is on your hard disk.

[Filip (fspacek)]

Here's a patch for casper to delete the loopback after copying:

diff --git a/scripts/casper b/scripts/casper
index 5861ced..fa06c0d 100644
--- a/scripts/casper
+++ b/scripts/casper
@@ -191,7 +191,7 @@ copy_live_to() {
     if [ -e ${copyfrom}/.disk ]; then
         cp -a ${copyfrom}/.disk ${copyto}
     fi
- umount ${copyfrom}
+ umount -d ${copyfrom}
     mount -r -o move ${copyto} ${copyfrom}
     rmdir ${copyto}
     return 0

[Charles Wilkins (cg-chas)]

Regarding comment #20, 2019-10-03,

I tried this patch and umount -d ${copyfrom} seems like it should unmount the loopback device, but it does not.

Yet this works:

root@xubuntu:~# losetup -d /dev/loop0
root@xubuntu:~# umount /isodevice

I am still looking that this, but was curious if anybody else is experiencing the same issue when booting a recent live iso from disk.

[Charles Wilkins (cg-chas)]

I tried explicitly using losetup -d /dev/loop0 right before umount ${copyfrom} in the casper script and it did not work. losetup -d /dev/loop0 and umount /isodevice does work out of rc.local just fyi.

[Michel-Ekimia (michel.ekimia)]

Did you try to force umount at line

https://git.launchpad.net/ubuntu/+source/casper/tree/scripts/casper?h=ubuntu/focal-devel#n200

[Michel-Ekimia (michel.ekimia)]

Even better option of umount : -d ( detach loop devices )

So sudo umount -lfd /isodevice

should be the way.

[Michel-Ekimia (michel.ekimia)]

Any update on this small but big win fix ?

[Michel-Ekimia (michel.ekimia)]

Still on 20.04.1 ...

[Michel-Ekimia (michel.ekimia)]

Just an Update : umount -lfd was not enough

So I'm trying to embed a startup script in a Live custom ISO :

losetup -d /dev/loop0
umount -lfd /isodevice

And it works

[Jerzy Luszawski (dr-agon)]

When booting Kubuntu 20.04.1 (LTS) iso the workaround of detaching /dev/loop0 device and then unmounting /isodevice was required, as in comment #27. Then setup finished.

[Paul W (ponggone)]

Issue is still occurring on ubuntu-20.04.3-live-server-amd64.

Unfortunately having difficulty unmounting the iso file with the following two commands,

losetup -d /dev/loop0
umount /isodevice

UPDATE: Specific issue related to the Dell Optiplex 5080 unit I was using
https://askubuntu.com/questions/1377159/ubuntu-server-20-04-3-setup-stuck-at-block-probing-did-not-discover-any-disks/1377260#1377260

[Martin Laclaustra (martin-laclaustra)]

This issue got worse because casper-lupin functionality was incompletely ported into casper itself.

I wrote a patch that ubuntu developers will need to add in order to get back at the point that this issue describes. The patch is here:
https://bugs.launchpad.net/ubuntu/+source/lupin/+bug/1960457/comments/34

[Martin Laclaustra (martin-laclaustra)]

Disconnecting the loop device may not always be desired after using "toram": See https://bugs.launchpad.net/ubuntu/+source/lupin/+bug/1960457/comments/33

Thus, it may be advisable to ask for that explicitly. I propose using "detach". So the system should be booted with "toram detach".

Solving this problem requires less than 10 lines. I hope that developers can add the patch below.

For this patch to work it is necessary to fix the issue mentioned previously beforehand: https://bugs.launchpad.net/ubuntu/+source/lupin/+bug/1960457/comments/34 first (4 lines)

Here is the patch:

--- scripts/casper_1477
+++ scripts/casper
@@ -75,6 +75,8 @@
                 export TORAM="Yes" ;;
             todisk=*)
                 export TODISK="${x#todisk=}" ;;
+ detach)
+ export DETACH="Yes" ;;
             hostname=*)
                 export CMD_HOST="${x#hostname=}" ;;
             userfullname=*)
@@ -1042,6 +1044,13 @@
     run_scripts /scripts/casper-bottom
     [ "$quiet" != "y" ] && log_end_msg

+ if [ "${live_dest}" ] && [ -z "$PERSISTENT" ] && [ "$DETACH" ]; then
+ if grep -q '^[^ ]* /root/isodevice ' /proc/mounts; then
+ losetup -d /dev/loop0
+ umount /root/isodevice
+ fi
+ fi
+
     # Close the fd's associated with debconf-communicate.
     exec 3>&- 4<&-
     rm -f /tmp/debconf-in.fifo

There are other options for implementing this, for example:
a) detaching as the default behavior and allowing for a "nodetach" argument in cmdline
b) removing "/root" from the patch (in the 2 lines where it appears), will make this work without fixing https://bugs.launchpad.net/ubuntu/+source/lupin/+bug/1960457/comments/34 first.

[Glenn Washburn (crass)]

> Disconnecting the loop device may not always be desired after using "toram": See https://bugs.launchpad.net/ubuntu/+source/lupin/+bug/1960457/comments/33

That comment was very likely not in the context of "toram" usage. That users requirements appear to be iso loopback loading access to /isodevice from the overlay root.

I prefer a nodetach option as I think the expected use case (or maybe just mine) for toram is to remove the usb device on which resides the loopback loaded iso so that the device can be used elsewhere while the live cd is running.

[scmarko (scmarko)]

This should be fixed, because when you boot an ISO file from your hard drive and load Ubuntu to RAM, you are not able to do any repartitioning, installing etc to the hard drive from where the ISO was loaded from when it stays mounted. I have tested this on a Ubuntu installation with LVM hard drive and once I applied the changes to the initrd file of the Ubuntu ISO, then I was able to successfully run the following commands:

losetup -d /dev/loop0
umount /isodevice

After that I was able to repartition the hard drive and install Ubuntu from RAM to hard drive. But this adds too much complexity to the end user if needing to unpack the ISO file and then modify initrd if needing to use Ubuntu like this.

[Mate Kukri (mkukri)]

So I've tried to reproduce this on Oracular 24.10 with the Flutter desktop installer, and I belive this is no longer a problem.

I've create a disk like:
- sda1 - vfat - /boot/efi
- sda2 - ext4 - /boot + Oracular ISO
- sda3 - ext4 - ubuntu partition

Booting the ISO via grub using:

  set isofile=oracular-desktop-amd64.iso
  loopback loop $isofile
  linux (loop)/casper/vmlinuz boot=casper iso-scan/filename=$isofile
  initrd (loop)/casper/initrd

Works just fine for me.

Then after that Ubuntu installation to sda3 works and boots succesfully.

[Michel-Ekimia (michel.ekimia)]

Thanks for your feedback

In my test the ISO would be on sda3 to reproduce

And you would need to try to resize sda3 from the live ISO to see the problem occur

[Mate Kukri (mkukri)]

I am not sure if modifying the same partition the ISO is actively being used from should be supported.

Or are you wanting this to only work when the ISO is used from RAM only?

[Michel-Ekimia (michel.ekimia)]

My initial concern was to install dual boot linux from Booting the live ISO toram from Windows C:

But this could also apply to installing from an existing linux

[Mate Kukri (mkukri)]

But I don't think this is strictly a bug, but I think detaching if toram is in bootargs is probably fine, i'll take another look at this later.

[scmarko (scmarko)]

I'm using the "toram" option to start Ubuntu based installations from already existing installations directly from the hard drive. Reasons for that is that I want to be able to perform a clean installation by completely wiping the hard drive after the ISO has been loaded into RAM.

For example, if there is Ubuntu 16 installed and I want to install Ubuntu 22, then I can set grub to use the boot to RAM option, erase the hard drive and then do the installation.

This way it is possible to start an installation from a remote location without having an USB stick or DVD/CD-ROM drive.

Sometimes I need to help people to upgrade their Linux systems and I usually have remote access to their desktops, so I can upload an installation ISO file to their hard drives. Then I configure grub to boot ISO into RAM. After that I can either instruct the user how to finalize the installation or in some cases, I have used a custom installer that performs the installation automatically.

So there are many cases when it is necessary to boot ISO to RAM and having possibility unmount the hard drive.

[Mate Kukri (mkukri)]

Okay I think we can add a `detach` option to casper to allow for this usecase.

I don't think this a bug strictly speaking, but there is a legitimate usecase for the feature.

[Mate Kukri (mkukri)]

re-opening to test the case when the entire drive is to be formatted instead of just one partition.

[Mate Kukri (mkukri)]

Confirmed that install indeed fails if I repeat comment #34 but erase entire sda instead of just sda3.