[Ubuntu Server 20.04 LTS]: Failed Install (subiquity...install_fail/add_info) during partitioning

[Michael Wesley]

Ubiquity fails when installing to a second partition (located after the first partition) on a drive that has an existing Ubuntu installation on another (previous) partition.

Please see the attachment for a screenshot of the error message.

Taken from: https://ubuntuforums.org/showthread.php?t=2443131

I want to dual boot my current Ubuntu 16 with the new v20. I booted a Live USB and used GParted to shrink my dev/sda1 in size and then directly after it, I created a new partition dev/sda5 in the unallocated space to use for the as Ubuntu 20 install.

During the v20 install I select the custom install method and select the dev/sda5 partition to be formatted to ext4 and mounted a "/". I then select the SWAP partition from my v16 install to be used as the SWAP for the v20 install.

I then proceed to the next screen where I choose my username and password, and after a few moments this error screen pops up.

I downloaded the Ubuntu 20 ISO from the official site and used Rufus to burn it onto a USB, which worked perfectly fine for the Ubuntu 20 Desktop live usb that I ran Gparted off.

I'm not sure what it's doing with sdf1. I have 5 other drives in the machine that are all part of a MDADM RAID array, unless for some reason the disk I'm trying to use is mounted as sdf in this case instead of sda as it was while using the live disk.

[Michael Wesley (electr0)]

Attached: subiquity-info.log

[Michael Wesley (electr0)]

Attached: subiquity-dubug.log

[Michael Hudson-Doyle (mwhudson)]

Hi, thanks for the report. Unfortunately the logs you've provided don't provide much help. Can you upload the crash file from /var/crash? It should be saved to the install media too.

Are you being offered an installer update as part of the install? Do you get the error if you take it?

[Michael Wesley (electr0)]

Attached Crash File

[Michael Wesley (electr0)]

Yes, I am being offered an installer update as part of the install.
Yes, I do get the error if I use the update

I also get the error if I don't update.

I attached the crash file in my comment above.

[Michael Wesley (electr0)]

This seems to be related to:

https://bugs.launchpad.net/ubuntu/+source/subiquity/+bug/1875560

https://bugs.launchpad.net/curtin/+bug/1875903

Unless something else is going on with my installation, this bug hasn't been fixed...

[Michael Hudson-Doyle (mwhudson)]

Congratulations, you have found a new problem :)

Will look in my morning.

[Michael Wesley (electr0)]

No worries. Thanks very much :)

[Ryan Harper (raharper)]

From the crash:

Verifying /dev/sda2 exists
 Running command ['sfdisk', '--json', '/dev/sda'] with allowed return codes [0] (capture=True)
 Verifying /dev/sda2 size, expecting 86564143104 bytes, found 86564143104 bytes
 Device /dev/sda2 ptable entry: {
  "node": "/dev/sda2",
  "size": 169070592,
  "start": 61442048,
  "type": "83"
 }
 Verifying /dev/sda2 partition flag, expecting linux, found None
 An error occured handling 'partition-sda2': RuntimeError - Verifying /dev/sda2 partition flag, expecting linux, found None

Curtin has a logic bug here for msdos ptable and primary partitions; we should lookup the type found in ptable_uuid_to_flag_entry().

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Server Team CI bot (server-team-bot)]

This bug is fixed with commit cb60d8a7 to curtin on branch master.
To view that commit see the following URL:
https://git.launchpad.net/curtin/commit/?id=cb60d8a7

[Michael Wesley (electr0)]

Thanks for fixing this @raharper!
How can I make use of this so I can finally successfully run the installation?
Do I just need to select the "Update Installer" option again?

[Ryan Harper (raharper)]

@Michael

I'll sync with mwhudson, subiquity will need to pull in the fix and release an update to edge, once that's done you can select the Update Installer option.

[Launchpad Janitor (janitor)]

This bug was fixed in the package curtin - 20.1-0ubuntu1

---------------
curtin (20.1-0ubuntu1) groovy; urgency=medium

  * New upstream release.
    - Release 20.1 (LP: #1880741)
    - Handle multiple separators which were found in TestAllindata vmtest
    - verify_ptable_flag: dos primary partitions use ptable_uuid map for flag
      (LP: #1878890)
    - net_meta: add disabled mode to skip writing any network config
      [Lucas Moura]
    - vmtest: trigger guest panic to fail fast
    - Replace grub-shell-helper with install_grub command
    - vmtest-sync-images: update the URL of the maas streams [Paride Legovini]
    - Replace references to old team manager with new team manager
      [James Falcon]
    - tox: pin flake8 to version and add a tip-flake8 environment
      [Paride Legovini]
    - Fix flake8 E741 warning [Lucas Moura]

 -- Ryan Harper <email address hidden> Tue, 26 May 2020 16:31:39 -0500

[Kai Harrekilde-Petersen (kaihp)]

With Ubuntu 20.05.2, the curtin/subiquity installer continues to fail to install onto existing partitions.

With the following partition table, subiquity/curtin refuses to use /dev/sda1 as boot partition and /dev/sda3 as root partition - something it previously accepted as choices (but it failed during the process).

{
   "partitiontable": {
      "label":"dos",
      "id":"0xd9859edb",
      "device":"/dev/sda",
      "unit":"sectors",
      "partitions": [
         {"node":"/dev/sda1", "start":2048, "size":1998047, "type":"83", "bootable":true},
         {"node":"/dev/sda2", "start":2000896, "size":2000896, "type":"83"},
         {"node":"/dev/sda3", "start":4001792, "size":64000000, "type":"83"}
      ]
   }
}

[Kai Harrekilde-Petersen (kaihp)]

@raharper I'd like to have this bug reopened, as it is still not fixed

[Kai Harrekilde-Petersen (kaihp)]

Still not fixed

[Ryan Harper (raharper)]

@Kai

Can you provide the installer crash or logs?

"refuses to use /dev/sda1 as boot partition and /dev/sda3 as root partition"

I wonder if recent changes in subiquity around ESPs may have something to do
with this.

[Kai Harrekilde-Petersen (kaihp)]

@Ryan,

Since the installer refuses to install, there are no crash logs that I can provide you with. If you try to install 20.04 onto a disk with the mentioned layout (the "Custom layout" option), you should find that you are unable to choose sda1 and sda3 as destinations for /boot and /. Don't you have a system where you can emulate this (that would be very handy)?

Is the installer log located in the same folder as the crash logs (/run/casper or something like that)?

A rant for another time is that the 20.04 installer system has never stored the install and crash logs on installation USB stick. In order to retrieve the logs I've had to use a second USB stick and manually copy files over to it.

[Kai Harrekilde-Petersen (kaihp)]

@Ryan

I booted the server into the 20.04 installer and copied off the /var/log directory.
/var/log/installer/subiquity-debug.log.3179 ends with:

2020-06-02 21:22:21,793 INFO root:39 start: subiquity/Filesystem: starting UI
2020-06-02 21:22:21,793 DEBUG root:39 start: subiquity/Filesystem/convert_autoinstall_config:
2020-06-02 21:22:21,794 DEBUG subiquitycore.controller.filesystem:170 self.ai_data = {}
2020-06-02 21:22:21,794 DEBUG root:39 finish: subiquity/Filesystem/convert_autoinstall_config: SUCCESS:
2020-06-02 21:22:44,255 DEBUG subiquity.ui.filesystem.filesystem:314 _action DeviceAction.EDIT partition-sda1
2020-06-02 21:23:05,415 DEBUG subiquity.ui.filesystem.filesystem:314 _action DeviceAction.EDIT partition-sda3

The subiquity-info.log file just reports that it has started the filesystem stuff:
2020-06-02 21:22:21,793 INFO root:39 start: subiquity/Filesystem: starting UI

Maybe I should point out that I cannot select /dev/sda1 at all as /boot, but I can select sda3, but just not as root directory.

If you have some kind of emulation environment to test out disk layout scenarios, then now is the time to pull it out.

[Ryan Harper (raharper)]

> @Ryan,
>
> Since the installer refuses to install, there are no crash logs that I can
> provide you with. If you try to install 20.04 onto a disk with the mentioned
> layout (the "Custom layout" option), you should find that you are unable to
> choose sda1 and sda3 as destinations for /boot and /. Don't you have a system
> where you can emulate this (that would be very handy)?

Thank you for the repro steps.

Booting the curtin live focal iso, I see that the curtin
version included with subiquity is 19.3-67-g145e49393

And this fix for this issue is in curtin 19.3-76-gcb60d8a7.

The fix has landed in *curtin* but we still need subiquity to pull
in this fix (subiquity bundles curtin into the snap).

>
> Is the installer log located in the same folder as the crash logs
> (/run/casper or something like that)?

It should be in /var/log/installer/* and /var/log/curtin/* (if it gets that
far, which in your case it doesnt)

>
> A rant for another time is that the 20.04 installer system has never stored
> the install and crash logs on installation USB stick. In order to retrieve the
> logs I've had to use a second USB stick and manually copy files over to it.
>

Crashes typically are uploaded to the crash service; however in the case
where it doesn't actually crash; there's not a way to "report an issue";
that I know of, so please do file a bug for that so we can track getting a fix
for that.

[Kevin Crowston (crowston)]

Are there any plans to incorporate the fix to curtin? It was released 27 May 2020 but the status of this bug doesn't seem to have changed.

[Michael Hudson-Doyle (mwhudson)]

It is now (as of a few minutes) in the snap currently in edge, so you should be able to test following the instructions in https://discourse.ubuntu.com/t/how-to-test-the-latest-version-of-subiquity/12428

[Kevin Crowston (crowston)]

I followed the instructions to get the new installer from edge. When I try to edit an existing LVM volume to make it root for the install the installer crashes as soon as I hit return on "Edit". There are some lines of backtrace, but they disappear before I can see them. I did upload a bug report, which hopefully is visible somewhere.

[Kevin Crowston (crowston)]

Sorry for the delay in trying this out, I seem to have gotten unsubscribed from the bug report so I didn't get an email saying there was a new comment.

[tlc (tlc)]

I have a similar message trying to install 20.04.1:

An error occured handling 'partions-sda5': RuntimeError - Verifying /dev/sda5 partition flag, expecting swap, found None

[Matthew Hall (mhall-9)]

It appears this is still happening to me many months / years later.

Even when I picked to upgrade the installer snap to the latest version.

I have never managed to get the new Ubuntu Server installer to work, either on a VBox VM like this, or on bare metal. It is very unfortunate since the d-i approach basically always succeeded.

Crash log attached.

[Matthew Hall (mhall-9)]

For the sake of root cause analysis, I bind mounted a manually patched copy of block_meta.py over the top of the existing version and commented out the partition type exception logic, and the install proceeded. So there is definitely something broken about curtin's logic for handling DOS partition tables as opposed to GPT, and the bug is not fully fixed.