Noble daily calamares installer does not honor existing partition table on "Erase All"

[Mike Ferreira]

Noble Daily Lubuntu image: 20231214

General:
If a user choice of a pre-existing partition table was to create and leave a Partition Table Type, that choice is overridden by the installer logic.

We often recommend to new users, the benefits of using a GPT partition table type, when recommending how they install their new installation.

On most Ubuntu Installers that are Ubiquity or Subiquity based, on what is found, if one of the "Erase All" install options are chosen:
If the device probe process does not find an existing partition table, then the logic rules creates a new partition table with a partition type based on the boot mode.
- GPT for UEFI
- MS-DOS for Legacy boot mode.
 (- If Legacy, there are considerations for disks over 2TB.)

If the device probe process does find an existing partition table then
- If leaves the partition table (erases all partitions in it)
 -- If the partition table is type MS-DOS type and Legacy Boot, then normal partitions for that mode.
 -- If the partition table is type GPT type and Legacy Boot, then if adds a bios_boot partition.
 -- If the partition table is type MS-DOS and UEFI boot, then adds ESP partition.
 -- If the partition table is type GPT and UEFI Boot, then adds normal partitions for that mode.

But with Calamares, if you choose "Erase All', then the default behavior is to delete any partition table if it does exist, then add a new partition table based on the Boot mode. It will not honor leaving a pre-existing partition table type.

You can work-around that by using "other", entering the manual partitioner to do all partition table maintenance (creation), partitioning, mounts, etc, then continuing the install, but that is outside of that erase all option.

To much of this, this is choice taken away, and considered a Bug by many of us. To others this seems to be a matter of semantics, where "Erase all" should include the partition table type.

I, and many others seem to agree that this should be filed as a bug, and sent upstream to Calamares. If not considered a bug by them, then maybe they should consider adding that logic as a proposed enhancement.

To reproduce:
1) Start the Lubuntu Calamares Installer in Legacy boot mode. Use "Try" add a GPT type partition table. Start the installer. The installed system will end up be on an MS-DOS type partition table.
2) Start the Lubuntu Calamares Installer in UEFI boot mode. Use "Try" add an MS_DOS type partition table. Start the installer. The installed system will end up be on a GPT type partition table.

ProblemType: Bug
DistroRelease: Ubuntu 24.04
Package: calamares 3.3.0.0-0ubuntu1
ProcVersionSignature: Ubuntu 6.5.0-9.9-generic 6.5.3
Uname: Linux 6.5.0-9-generic x86_64
.etc.calamares.modules.after_bootloader_context.conf:
 ---
 dontChroot: false
 timeout: 120
 firmwareType:
     "*": "-rm ${ROOT}/home/${USER}/Desktop/lubuntu-calamares.desktop"
.etc.calamares.modules.finished.conf:
 ---
 restartNowMode: user-checked
 restartNowCommand: "systemctl -i reboot"
.etc.calamares.modules.fstab.conf:
 crypttabOptions: luks,keyscript=/bin/cat
 efiMountOptions: umask=0077
.etc.calamares.modules.shellprocess_logs.conf:
 ---
 dontChroot: true
 timeout: 30
 script:
     - calamares-logs-helper ${ROOT}
.etc.calamares.modules.unpackfs.conf:
 ---
 unpack:
     - source: "/cdrom/casper/filesystem.squashfs"
         sourcefs: "squashfs"
         destination: ""
ApportVersion: 2.27.0-0ubuntu6
Architecture: amd64
CasperMD5CheckResult: pass
CasperVersion: 1.489
CurrentDesktop: LXQt
Date: Thu Dec 14 23:17:32 2023
LiveMediaBuild: Lubuntu 24.04 LTS "Noble Numbat" - Daily amd64 (20231214)
RelatedPackageVersions:
 calamares-settings-ubuntu-common 1:24.04.9
 calamares-settings-lubuntu 1:24.04.9
 xfsprogs 6.5.0-1ubuntu1
 btrfs-progs 6.3.2-1
SourcePackage: calamares
UpgradeStatus: No upgrade log present (probably fresh install)

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Erich Eickmeyer (eeickmeyer)]

In my opinion, Calamares doesn't need to destroy the partition table, just remove the existing partitions in the "erase and install" scenario. If you're on an EFI system, changing to an MBR partition table could render the system unbootable and vice-verse. For the average user, this would be very unwanted.

[Aaron Rainbolt (arraybolt3)]

Calamares detects what firmware the system is using though. If you're on an EFI system, it's fine if a GPT partition table is left intact, but very bad if an MSDOS one is left (I believe). Calamares, to my awareness, always makes an MBR table on BIOS systems, and a GPT one on EFI systems. That's correct behavior as far as I can tell.

The only way this could be faked out AFAIK is to boot the installer in the wrong mode (booting in BIOS mode on an EFI system by selecting a legacy boot option, or booting in EFI mode on a system you wanted to use in legacy mode). This is certainly possible, but this causes problems with flavors other than Lubuntu too. It's always a bad idea to install like that.

I personally am rather surprised that Ubiquity and Subiquity *don't* erase partition tables. I'm sure it's intentional, and I guess it doesn't appear to cause problems for some people, but the idea of "erase disk" not truly erasing the whole disk is slightly alarming to me.

In any event, if someone can provide a reliable set of steps that allows fooling Calamares into installing Lubuntu into an MBR partition table when booted in EFI mode, or installing into an improperly created GPT partition table (one missing a bios-boot partition) when booted in BIOS mode, then I think this is worth looking into. Until then though, I'm inclined to say this is intentional behavior. If a user truly wants something other than the default, that's what manual partitioning is for. (I'm not the one who makes the final decision on whether or not this is going to be worked on, but right now I just don't see why it needs changed.)

[Simon Quigley (tsimonq2)]

Given our discussions in the Lubuntu Development channel and a wide variety of opinions, as the Lubuntu Release Manager, I am deciding this is a "Won't Fix."

In my own capacity, maybe stop confusing new users with partition table types, in 2023 where it's mostly automatic? If they don't have to worry about it, why burden them with having to learn? This is pretty pedantic, and as you even mentioned, in edge cases they can just use the manual install.

[Mike Ferreira (mafoelffen)]

I respect your decision, though saying
>>>
"maybe stop confusing new users with partition table types"
>>>
LOL. In the forums, we are not the ones "confusing new users"... as they usually come there "pre-confused". Saying that with those words could be take either as funny or offensive to members who have to support users of all skill levels. I currently choose to lean towards taking comment, loosely, as a joke.

To me, with supporting Grub2, 'boot-repair' and 'boot-info'... I know what Grub2 will support as bootable. For the most part, things are automatic, whether that logic is there or not. It is presently not there. Which is different, and not consistent with the (other) Ubiqity and Subuitquity installers that Ubuntu and the other flavors use. Which makes Calamares, sort of stand out as being different in that respect in how it deals with that.

(This is outside of this specific bug report, but something to think about in relation to it;)
Along with some of the other comments... I have recommended that, at least with YannUbuntu with my contributions to 'boot-repair' and 'boot-info', that it already checks the boot-mode and BIOS capabilities, that the "User" should be made aware of at least the boot-mode, so we may possibly prevent Legacy-to-what is supposed to be UEFI, or UEFI-to-BIOS boot mode repairs. That same logic is what we run into on the Forums for Boot Mode errors on new installations. That one prompt, might prevent a lot of problems we face in helping users correct many (current) installation problems. We run into this often, where other OS'es (on other disks) on their system (usually Windows) were upgraded from older legacy versions, and such.

Just some thoughts on that.

Thank you. I have to applaud the quick, timely and thorough response to this report from this team. I am actually very impressed with how you all have handled this one. Thank you all for your dedicated work. It is very appreciated, and not mentioned enough.

[Simon Quigley (tsimonq2)]

> I currently choose to lean towards taking comment, loosely, as a joke.

You're right. I have to be firm, but I mean no fighting words. You all are friends. :)

> [...]

So, we're basing it off of boot-repair, which wasn't in the Ubuntu archive last time I checked (which admittedly, has been a while)? (Open to fixing that, btw.)

I still think the manual install is intuitive enough to help in the weird edge cases.

> Thank you. I have to applaud the quick, timely and thorough response to this report from this team. I am actually very impressed with how you all have handled this one. Thank you all for your dedicated work. It is very appreciated, and not mentioned enough.

The decision was not made unilaterally, to be clear. We opened it up for discussion for quite a few hours in the Development channel, I just recognized we had to come to a conclusion on this one.

We appreciate you too :)

[Chris Guiver (guiverc)]

I'll add a comment, but I have no new information (this maybe a red herring though).

This issue does not worry me (I don't recall encountering it actually).

Aaron said
> The only way this could be faked out AFAIK is to boot the installer in the wrong mode (booting in BIOS mode on an EFI system by selecting a legacy boot option, or booting in EFI mode on a system you wanted to use in legacy mode).

Following support questions, and trying to replicate what a user reported (cycles ago now), I managed to boot my hp dc7700 (a ~2005 box so BIOS only!) and have calamares (older version than we're using now) think it's in a uEFI machine as reported on screen as EFI where you select storage device. To achieve this however the ISO needed to be written in a reformat type operation, and not following the Ubuntu documented procedure(s) for writing ISOs. (The install usually failed if I recall correctly here, as it ended up trying to write the boot detail to the RO thumb-drive given that was where it found the ESP which I suspect may have tricked calamares). (I don't believe this is what is being done here though; point being calamares can/could? be tricked).

[Mike Ferreira (mafoelffen)]

Just for clarification:
>>> we're basing it off of boot-repair(?)...
Not based off 'boot-repair', but more on my experience gained from... of what I know of that GNU Grub2 supports to be bootable, and on how the other two Ubuntu installers behave... Albeit, I admit, my own experience with Lubuntu & Calamares (itself) is not as extensive.

On the later... Yes, a few of us trying to get with Yann to get that updated. I'm sort of embarrassed that is is still on an 18.04 image, and that in itself is starting to cause some issues in the chroot, with some of the patches I gave Yann.

>>> I still think the manual install is intuitive enough to help in the weird edge cases.
For the most part, though I do have some issues filed on Flutter's new partitioner to try to keep it that way, which those aren't worked out yet. (Example: Not Volume Manager aware.) <-- But those are not present in Calamares.

*** Noble testing is going really well so far-- Very little found with Lubuntu. When this came up, the Noble testing group didn't know if this was a really considered a bug or a style choice in Calamares. Frankly, it embarrassingly kept going back and forth, so I handed it off to you all. (Had to keep the peace.) Better decision, (out of my hands) that I can forward back to the other testers.

Sorry. I don't want to dilute this further. Thank you. Great Job.

[Ubuntu QA Website (ubuntuqa)]

This bug has been reported on the Ubuntu ISO testing tracker.

A list of all reports related to this bug can be found here:
http://iso.qa.ubuntu.com/qatracker/reports/bugs/2046500