Disk Setup / Manual installation missing LVM LV as target

[Steffen Neumann]

Hi,

I am trying 24.04 installation from noble-desktop-amd64.iso (2024-03-20 10:14)
with ubuntu-desktop-bootstrap 0+git.1314d484d

My laptop has an existing (non-encrypted) LVM setup, where I created an LV for noble from the existing jammy installation.

I boot into the noble-desktop-amd64.iso, get to screen "Disk Setup", select "Manual installation"
and I could select the /dev/sda* partitions for installation.

I *can not* select the logical volume, it is simply not showing up.

I can manually list LVs and mount /dev/mapper/lvmssd-noble from the terminal.

Expected behaviour:
Next to sd* partitions, I can select the logical volume /dev/mapper/lvmssd-noble as target.

Yours,
Steffen

(noble-desktop-amd64.iso sha256 55f7de9571f2a85ac8c3c9bb553862449f40e1275d41c7646cd83ec26db24bd2)

[john (johnsmith25)]

I've also installed this way for as long as I can remember.

Please can someone confirm ASAP if this is an intentional change or not.

[Juan Andrés Ghigliazza (tizone)]

Same here.

[john (johnsmith25)]

How is this a wishlist item? Its a complete install blocker...

[Juan Andrés Ghigliazza (tizone)]

In the meantime, it would be great if someone could provide a workaround for the problem.

[Mohegan (jack-mohegan)]

Same here. Can't switch to Noble without erase my disks !

Very important bug !!!

[Graeme (graeme-riddick)]

Just sat down to upgrade my system too and same problem for me. Agree with John, it's a blocker.
Cant install do a /dev/mapper/xxx
Unless there is a way to force it in the YAML file?

[Omar AKHAM (akhamomar)]

Same problem for me. It seems that's an old issue : https://answers.launchpad.net/ubuntu/+source/ubiquity/+question/706331

[Adam Dyess (addyess)]

Yup, this one should be higher than a wishlist. At least with Minotaur, there was the legacy installer option.

to be clear, i want to use the installer to select my LVM partition for installation. This has been available for years, and i hate to see it go. Please bring it back

[Mohegan (jack-mohegan)]

I have a bad solution but it works.
I installed the ubuntu 23.10 legacy version (from here : https://cdimage.ubuntu.com/ubuntu/releases/23.10.1/release/). It can detect the LVM volumes.

After that, I've forced the update version with this command :

sudo do-release-upgrade -d

[Thomas Ward (teward)]

This needs to be increased in severity, in my opinion. It is otherwise **very difficult** to set up DISA STIG compliant setups if we can't manipulate LVMs in the manual installer.

[Anderson Luiz Alves (alacn1)]

It should at least show already created lvm partitions, and other dev/mapper partitions, and allow set it as install target.

[DiagonalArg (diagonalarg)]

@teward / That would make this bug a security issue, wouldn't it? This should have /much/ higher priority. For me, it's a deal breaker.

@graeme-riddick / any ideas on the yaml front? I'm wondering if I should try to dig into that.

Edit: Wow, they've got a lot of big issues. We'll be waiting ...
https://bugs.launchpad.net/ubuntu-desktop-provision/+bug/2048473
https://bugs.launchpad.net/ubuntu-desktop-provision/+bug/1881932

Edit2: These two appear to be the same. They are specifically about LUKS, but in both cases we're talking about /dev/mapper/* not being recognized:

Ubuntu 24.04 (20240409) unable to install on preexisting LUKS partions
https://bugs.launchpad.net/ubuntu-desktop-provision/+bug/2060678

Subiquity Manual Installation, 24.04 / Can't find LUKS targets
https://bugs.launchpad.net/ubuntu-desktop-provision/+bug/2064193

[Andrew H (andyvorld)]

I was able to work around the UI not showing LUKS volumes with using the autoinstall.yaml configs,
https://github.com/asayler/ubuntu_autoinstall/blob/master/user-data-encrypted; with everything but the storage section stripped out. This however will format and clear out the entire drive so its not usable for prexisting partitions with data on them.

Playing around with the yaml files more lead to this https://pastebin.com/2HGrWMYF, in a UEFI setup the auto install requires a grub partition and disk fully defined. For the rest of the data partition like / and /boot we can use a prexisting "type: device" (https://curtin.readthedocs.io/en/latest/topics/storage.html) to provide the block device, and using the "preserve: true" should keep the onboard data intact. Only issue with this is that the newly install os does not have cryptsetup in the initramfs so it needs to be manually installed before rebooting.

[Graeme (graeme-riddick)]

Thanks @Andrew H ! Confirm that this yaml code works perfectly for the install.
A apt install cryptsetup-initramfs in the chrooted environment was indeed all it took after the install and before reboot.
Pity that it took me quite a few hours more to install this time around!

[morph027 (morph027)]

Thanks, just tested inside a VM and it looks good. Only thing is i have to add the precise numbers of the existing partitions, which is annoying but manageable.

For the cryptsetup-initramfs, i also need to add the device to crypttab.

I'm using the late-command w/ a shell script like this https://pastebin.com/0CT774ry

  late-commands:
    - 'curtin in-target --target=/target -- sh -c "wget -qO - https://pastebin.com/raw/0CT774ry | bash"'

: https://canonical-subiquity.readthedocs-hosted.com/en/latest/reference/autoinstall-reference.html#late-commands

[morph027 (morph027)]

Sidenote for autoinstall, all partitions not known to curtin within the yaml will get dropped, so make sure to add dummy partitions with 'preserve: true' for each existing partition and their size (e.g. windows dual boot partitions). Ask me how i know 😉 Good thing i'm testing out in a VM.

Example attached one partition not to be used for the installation is 26.5G. While 100G within the yaml works, for 26.5G i needed to specify the proper bytes or the installer will crash and complain about a size mismatch.

[Mike Ferreira (mafoelffen)]

This is sort of the same as 'part of' my bug report: https://bugs.launchpad.net/ubuntu-desktop-provision/+bug/2065236

But my write-up of that is that there is a bigger problem with the new partitioner.

It's not just NOT recognizing LVM2 structures. It's also not recognizing mdadm, and LUKS containers, etc. So there is a bigger problem where it is not recognizing many volume managers, file systems types, and storage containers, that it used to be able to with the older partitioner.

I will link this bug to mine, (as a mention) since this 'IS' older... but there is still a bigger problem that needs to be addressed with this.

The autoinstall example buried deeply within the snap image work, as an alternate method. They are undocumented.

Eevn though they work as a workaround, they don't address the problem with the new partioner not being able to recognize pre-existing storage containers. That was something that Canonical promised that they would improve and fix when they announced choosing the new partitioner, back in the Lunar DEV-Cycle. So far none of that has transpired.

[Piotr Morgwai Kotarbiński (morgwai)]

If this is a part of some joint apple+m$ conspiracy to move ppl away from linux, then they are doing a really good job ;-) Jokes aside, this bug makes it extremely difficult to use custom partitioning in a secure way, which is a very common usecase (reasons include things like a fresh install while preserving /home, encrypted swap partition for secure hibernate etc). Therefore marking this as "whishlist" and "low importance" is a slap in our faces. At this point I'm really done with ubuntu's nonsense: I'm moving to another distro until this is fixed and will be also encouraging all organizations I cooperate with to do the same...

[Vadim Nevorotin (malamut)]

Low importance? Is it a joke? Ubuntu LTS is fully non-installable - how can it be low priority? The main basic feature of installer - install to existing partitions (whatever LUKS, LVM, ZFS or any other block devices supported in Linux) - is completely missing!

Ability to select existing device to install to is a absolutely mandatory feature, without it it's not an installer at all!

[sva42c8 (sva42c8)]

Have been assigned a (yearly) ticket to workout
a migration path again.
But I fail miserably to complete the task again,
but still company allocates every year 3days for research :-).
So here I am:

Encrypted Ubuntu 24.04 side by side to existing
encrypted Windows is not possible.
If it works out( encryption+hibernate), windows partition could be dropped and ubuntu partition can be further enlarged.

Additional encrypted Windows is easy and all requirements below are passed

This bug hasn't hit my research in 22.04,
 so we wait for a working dist-upgrade in August.

Our must have requirements for Desktops and Laptops:
- Bitlocker/Veracrypt/Filevault FDE
- SecureBoot
- hibernation support with enabled encryption.
- Docking support and hotplug and restore window positions
- Resume Hibernation with secure boot like in Windows, Docked and Undocked.

The last point secureboot+hibernate will be a big bumper,
 but that is another problem for next year's reoccuring
ticket(Lockdown patch)

[David DIDIER (ddidier)]

Wishlist and low priority... You fools! I'm seriously considering to install another distribution even if I'm a very long time user of Ubuntu. And this is clearly a security issue.

[Bob McChesney (bmcchesney)]

I was hit with this problem trying to install Ubuntu 24.04 alongside an existing Linux Mint deployment tonight. Annoying but it looks like some people have had success with using autoinstall.yaml.

Perhaps someone could please provide a bit of assistance for me and my existing configuration:

https://pastebin.com/izL3bP1L (Using pastebin as output was illegible here)

What should I be doing to have a safe/sane autoinstall.yaml for my scenario where I want:

* All partitions to be left as is (not reformatted) except the new vgsystem-ubroot
* UEFI bootloader to go in nvme0n1p1
* Grub, initfs, kernel etc to go in nvme0n1p5
* Ubuntu to be installed in vgsystem-ubroot at "/"
* New user to be created in vgsystem-home at "/home/", leaving existing files

I've had a few tries in a VM but I'm getting it wrong every time, so if someone has the required insight, please help.

[Matt Borgerson (mborgerson)]

I also wanted to preserve some existing partitions (namely EFI/Windows) while replacing an older Ubuntu 22.04 LUKS setup. Since I could not find a way to get the installer to recognize the unlocked partition, I just installed to an unencrypted partition and migrated the installed filesystem into a new encrypted partition.

Basically: launch the installer, before continuing in the installer use the Disks utility to replace your encrypted partition with an EXT4 filesystem, select it as the root mount point during manual setup and continue installation normally.

Reboot to confirm installation success.

Reboot into the live installer once again. Mount the install partition and copy the installed filesystem contents out to /tmp/rootfs (via rsync -aXHAS /mnt /tmp/rootfs). Then unmount and use the Disks utility again to create your preferred encrypted partition setup, mount it and copy the filesystem contents back in. Finally, chroot into the filesystem, install cryptsetup, create an /etc/crypttab, update /etc/fstab, then run update-initramfs and update-grub. Once rebooting you should have a new Ubuntu 24.04 install on an encrypted partition along side your other, untouched partitions on the disk.

Hope this helps.

[Bob McChesney (bmcchesney)]

Matt, your suggestion sounds good but unfortunately I have (perfectly reasonably IMO) consumed all of the space after EFI, Windows, and /boot with a LUKS LVM partition, so that's only where I can put new volumes. I think the autoinstall.yaml one-shot is what I need.