Installer fails to remove old partitions

[Diego Reyes]

When installing Ubuntu 18.04-server over an old installation, after selecting/confirming 'whole disk' the installer fails to remove the old partitions and aborts the process.
Tested on two different machines both with previous Ubuntu 16.04 installations.
One workaround is to write zeros to the beginning of the target disk.

[Adam Conrad (adconrad)]

Which installer ISO did you use? Was this with the new one (named "live-server") or the old style (just "server")?

[Michael Hudson-Doyle (mwhudson)]

if this was a live-server install, would it be possible to see the contents of the /var/log/installer directory from the live session?

[Diego Reyes (dmrq70)]

The iso I used is this one:

0b3490de9839c3918e35f01aa8a05c9ae286fc94 *ubuntu-18.04-live-server-amd64.iso

I don't remember seeing the option to boot to a live desktop, so I didn't know how to extract any logs.
I could reproduce it with a VM (Virtualbox on ubuntu), but only when installing 18.04 over a 16.04. If the previous installation was 17.x-desktop, the error didn't show.
I'm attaching a tar with all that's in /var/log, the error I see while installing appears in /var/log/curtin/install.log.
Let me know if I can help.

[Michael Hudson-Doyle (mwhudson)]

Adding curtin to the report as this *looks* like a curtin bug to me on first sight but I might be missing something.

[Scott Moser (smoser)]

Based on Michael's mention of 'extended partitions', this could
possibly be fixed by
 https://code.launchpad.net/~raharper/curtin/+git/curtin/+merge/345361

[Ryan Harper (raharper)]

From the install.log

Current device storage tree:
sda
|-- sda2
|-- sda5
| |-- dm-1
| `-- dm-0
`-- sda1

...

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/curtin/commands/main.py", line 201, in main
    ret = args.func(args)
  File "/usr/lib/python3/dist-packages/curtin/commands/block_meta.py", line 52, in block_meta
    meta_custom(args)
  File "/usr/lib/python3/dist-packages/curtin/commands/block_meta.py", line 1374, in meta_custom
    clear_holders.clear_holders(disk_paths)
  File "/usr/lib/python3/dist-packages/curtin/block/clear_holders.py", line 552, in clear_holders
    shutdown_function(dev_info['device'])
  File "/usr/lib/python3/dist-packages/curtin/block/clear_holders.py", line 247, in wipe_superblock
    if block.is_extended_partition(blockdev):
  File "/usr/lib/python3/dist-packages/curtin/block/__init__.py", line 789, in is_extended_partition
    check_dos_signature(device))
  File "/usr/lib/python3/dist-packages/curtin/block/__init__.py", line 756, in check_dos_signature
    return (is_block_device(device) and util.file_size(device) >= 0x200 and
  File "/usr/lib/python3/dist-packages/curtin/util.py", line 565, in file_size
    with open(path, 'rb') as fp:
OSError: [Errno 6] No such device or address: '/dev/sda2'
[Errno 6] No such device or address: '/dev/sda2'
curtin: Installation failed with exception: Unexpected error while running command.

Yes, https://code.launchpad.net/~raharper/curtin/+git/curtin/+merge/345361 should fix this.

/dev/sda2 is an extended partition with a single logical volume (sda5). I've observed that after wiping the lvm device, there is a period where the kernel removes the /dev/sda2 partition node; this is resolved by asking the kernel to re-read the partition table after each shutdown stage.