missing option to erase and use the whole drive

[sudodus]

After installing in BIOS mode there was an MSDOS partition table with a single partition. When trying to install in UEFI mode onto the same target drive (an SSD), the only option was ‘Manual partitioning’. This can be an obstacle, at least for beginners, who want to use an old HDD or SSD.

This is not bug #1824021, because there is only zram swap.

lubuntu@lubuntu:~$ swapon
NAME TYPE SIZE USED PRIO
/dev/zram0 partition 5,8G 0B 5
lubuntu@lubuntu:~$

Here I interrupt when Calamares is at the partitioning page in order to collect the current status of the live system and Calamares.

-o-

This is a repetition of what I did yesterday, when I exited from Calamares and wiped the first mibibyte on the target drive. Yesterday, after wiping I started Calamares again, and it could use the whole drive and create an installed system.

So there is a workaround, but I think it would be better, if the user would be offered to use the whole drive (and not only manual partitioning). It can be implemented by wiping the first mibibyte, after the user has selected the option to use the whole drive and confirmed an extra question at a final checkpoint.

ProblemType: Bug
DistroRelease: Ubuntu 24.04
Package: calamares 3.3.4-0ubuntu1 [modified: usr/lib/x86_64-linux-gnu/calamares/modules/networkcfg/main.py]
ProcVersionSignature: Ubuntu 6.8.0-11.11-generic 6.8.0-rc4
Uname: Linux 6.8.0-11-generic x86_64
.etc.calamares.modules.after_bootloader_context.conf:
 ---
 dontChroot: false
 timeout: 120
 firmwareType:
     "*": "-rm ${ROOT}/home/${USER}/Desktop/lubuntu-calamares.desktop"
.etc.calamares.modules.finished.conf:
 ---
 restartNowMode: user-checked
 restartNowCommand: "systemctl -i reboot"
.etc.calamares.modules.fstab.conf:
 crypttabOptions: luks,keyscript=/bin/cat
 efiMountOptions: umask=0077
.etc.calamares.modules.shellprocess_logs.conf:
 ---
 dontChroot: true
 timeout: 30
 script:
     - calamares-logs-helper ${ROOT}
.etc.calamares.modules.unpackfs.conf:
 ---
 unpack:
     - source: "/cdrom/casper/filesystem.squashfs"
         sourcefs: "squashfs"
         destination: ""
ApportVersion: 2.28.0-0ubuntu1
Architecture: amd64
CasperMD5CheckResult: pass
CasperVersion: 1.494
CurrentDesktop: LXQt
Date: Mon Apr 1 13:13:01 2024
LiveMediaBuild: Lubuntu 24.04 LTS "Noble Numbat" - Daily amd64 (20240328)
RelatedPackageVersions:
 calamares-settings-ubuntu-common 1:24.04.16
 calamares-settings-lubuntu 1:24.04.16
 xfsprogs 6.6.0-1ubuntu1
 btrfs-progs 6.6.3-1.1
SourcePackage: calamares
UpgradeStatus: No upgrade log present (probably fresh install)

[sudodus (nio-wiklund)]

See also this link

https://discourse.lubuntu.me/t/report-about-iso-testing-and-two-questions-about-reporting-or-not-reporting-bugs/4793

[Ubuntu QA Website (ubuntuqa)]

This bug has been reported on the Ubuntu ISO testing tracker.

A list of all reports related to this bug can be found here:
http://iso.qa.ubuntu.com/qatracker/reports/bugs/2059907

[ԜаӀtеr Ⅼарсһуnѕkі (wxl)]

To be clear, as I understand it, this is the situation caused by running the installer twice in one session. The problem is caused because the drive is still mounted at /tmp. This occurs regardless of whether or not Calamares is interrupted in the middle of installation or if it is allowed to complete the installation and is exited cleanly.

I note that we use the umount module in our penultimate step of the installation (the last being the finished module). calamares-settings-lubuntu doesn't include a config file for it. I would presume it would then use the defaults. There is only one option which, by default, should cause the module to run if one of the previous modules fails.

That said, I see two potential explanations for why the umount doesn't seem to be working:
 1. the missing config file = calamares-settings-ubuntu issue
 2. the module isn't working as expected = calamares issue

With those potential issues resolved, I think the next logical step would be to *also* include running the umount module *before* the partioning step. This will deal with the case where the user has done something else in the live session that mounts the drive.

Obviously, the umount module isn't expected to run if Calamares is forcibly closed. There's really nothing that can be done about that situation.

I think the likelihood of users running the installer twice is slim to none, so I'm calling this low.

[sudodus (nio-wiklund)]

No, I shut down the computer after installing in BIOS mode, cold boot into the installed system, shut down again, boot in UEFI mode into the Lubuntu Noble live. The problem is *not* caused by running the installer twice in one session.

But I agree, that it would be a good idea to run the umount module *automatically before* the partioning step.

[ԜаӀtеr Ⅼарсһуnѕkі (wxl)]

Well, that's weird. I did two installation tests and couldn't reproduce *that* behavior:

 1. With pre-existing BIOS install, fresh boot in UEFI mode
 2. With pre-existing UEFI install, fresh boot in BIOS mode

In both cases, I got the erase disk option.

That said, I wonder if your issue isn't the zram swap after all.

[sudodus (nio-wiklund)]

I can try in another computer (not the Toshiba). But the zram swap should not be linked to any SSD or HDD; it resides in RAM.

When trying to reproduce, did you manually unmount all partitions on the target drive before starting Calamares?

I don't think that beginners will do that.

But when I worked around the problem I did that and I also wiped the first mibibyte in order to 'be sure' the drive is clean with nothing confusing (for example nothing to re-mount automatically).

[ԜаӀtеr Ⅼарсһуnѕkі (wxl)]

I did nothing other than boot and run the installer.

If you can reproduce the behavior again, I would then do this:
 1. A fresh boot into the ISO
 2. Check to see if your drive is mounted (it shouldn't be under any conditions) - `umount` if so and double check
 3. Check to see if your swap is mounted (I would expect it to be) - `swapoff` if so and double check
 4. Run the installer and see if you get the Erase Disk option

Report back about the state during those checks.

[sudodus (nio-wiklund)]

1. Fresh boot into iso file (in UEFI mode).

2. The partition on the target drive is mounted automatically. I unmount it now. This should be done automatically by Calamares.

3. I leave the zram swap on (I don't think it must be swapped off; I want to check that.)

4. Fix for 2054795 and start the installer ... and using the whole drive is allowed and the installation finishes sucessfully.

So this indicates that it is enough to unmount the partition on the target drive. It was 'overkill' when I also wiped the first mibibyte.

-o-

I don't think that average beginners will understand/guess that they should unmount the partitions on the target drive in order to be allowed to install Lubuntu onto the whole drive, so I suggest that we unmount those partitions automatically.

If it is important to have them mounted in order to see what they contain, it could be done if/when the user has selected a drive and selected the option to use the whole drive. (Otherwise it would be easier to unmount the partitions (except on the boot drive) as soon as a drive is selected or maybe even as soon as Calamares is started.)

[ԜаӀtеr Ⅼарсһуnѕkі (wxl)]

That's weird to see the partition mounted. I struggle to understand why.

Try again, but this time don't worry about that and `swapoff`. It certainly is known that swaps are automounted (this has something to do with the core Ubuntu setup). It is also known that, at least when they are related to a drive (e.g. swap file), they prevent an Erase Disk option.

Perhaps these things are related. Swaps get mounted automatically, but if you have a zram swap, perhaps it also mounts the partition where the install associated with that setup lies???

[sudodus (nio-wiklund)]

Why struggle to understand why the partition on the target drive is mounted? Maybe because the target SSD is connected via USB (and not via SATA). During these tests there is no internal drive in my Toshiba, only two SSDs connected via USB, the live drive and the target drive.

In the case of these tests with my Toshiba, I am rather sure, that there is no problem due to any swap.

-o-

I made a quick test in another computer, a Dell Precision M4800 with an internal drive containing Ubuntu Jammy with a swap partition. When booting into Lubuntu Noble live in this case the root partition is not mounted, but the swap partition is connected, so I understand why you want me to swapoff.

I did not want to overwrite Ubuntu Jammy, but I could run Calamares until the partitioning page, and I can verify that in this case it was enough to run

swapoff /dev/sda2

and leave the zram swap on, and get the option to use the whole drive at the partitioning page.

-o-

In order to make installing work with various setups I suggest that Calamares should unmount and/or swapoff whatever is activated on the target drive.

But I don't think everything should be swapped off, there is a reason why Lubuntu live is using zram for swapping: to make it work with a small amount of RAM.

[ԜаӀtеr Ⅼарсһуnѕkі (wxl)]

You're right that the automounting of the partition is irrelevant if we're just going to go ahead an umount whatever, but it's still perplexing not to know *why*.

Anyways, another good point about keeping zram on. We do have that for a reason. Thus, the recommendation of `swapoff -a` is not a really good one.

That said, I see the following action items:

 1. Make sure the umount module is working as intended to ensure closing Calamares unmounts everything
 2. Unmount before running Calamares to ensure no partitions are mounted
 3. `swapoff` swaps except for zram0 (`swapon --show=NAME --raw --noheadings | grep -v zram0` should help here) before running Calamares

[sudodus (nio-wiklund)]

@ԜаӀtеr Ⅼарсһуnѕkі (wxl),

Yes, unmount what can be unmounted (unless when using the boot option 'toram', I think some partition(s) on the live drive should be left mounted (not be forced to be unmounted)). And swapoff everything except the zram.

I think that would squash this bug.

[sudodus (nio-wiklund)]

Well, today I found a case, where unmounting and swapping off does not help. I had to wipe the first mibibyte.

In my Toshiba Satellite Pro c850-19w I connected a target SSD with a current daily live Lubuntu system, and it was not identified as a target device (until I wiped the partition table). This is a corner case, but anyway, I want to report it here.

[ԜаӀtеr Ⅼарсһуnѕkі (wxl)]

Was it mounted?

[sudodus (nio-wiklund)]

Before starting Calamares I unmounted it (of course), and there was no swap partition (only zram).

There was a live Lubuntu system on the target drive, and obviously this was recognized by Calamares.

The running live system was booted from another drive, but Calamares either 'assumed' that Lubuntu was booted from the drive that I intended to use as target for the installation, or generally, is set to avoid installing to a drive with a live system to avoid mistakes.

[ԜаӀtеr Ⅼарсһуnѕkі (wxl)]

Since I've observed a successful install with a cleanly closed Calamares leave the target partition mounted, I'm concerned there's a fundamental bug in the umount module. I've also seen some suggestion upstream (haven't checked the code to be fair) that unmounting is supposed to happen before getting drive information. That said, I think this is a thornier issue than we had previously imagined. Thus, bumped to 24.10+.

[sudodus (nio-wiklund)]

We should at least suggest a workaround in the release notes:

When installing you may want to use the whole target drive, but in some cases (typically if installing into a USB drive) it will not be offered by the installer. In such cases you can exit from the installer and unmount (and/or swapoff) all partitions on the target drive. If that does not help, you can wipe the first mibibyte of the target drive. Please doublecheck that you are wiping the correct target drive,

sudo dd if=/dev/zero of=/dev/sdx bs=1M count=1k

where x is the device letter for the target drive, for example a or b or c. If you want to reduce the risk, you can install mkusb and let it wipe the first mibibyte in a safer way.