Add partitioning example to autoinstall-quickstart

[Dan MacDonald]

At my uni/work we have an LTSP server. The best way to manage update an LTSP server image (for us) is to use a VMDK file as the source of the LTSP image. VB is also cross-platform (unlike Virt-manager) and hence I wanted to write a script to automate the process of creating a new VirtualBox VMDK image from scratch running a bare bones Ubuntu Server 22.04 install (amd64).

I found and successfully followed the cloud-localds method described in this guide:

https://ubuntu.com/server/docs/install/autoinstall-quickstart

I have got my script basically working now (bar having to type 'yes' to begin the otherwise fully automated installation) to automate the installation of a VB Ubuntu VM except for one issue and that is correctly automating the desired partitioning for the VMs disk.

As is, following that guide (curtin?) partitions a disk with a GPT partition label, it uses LVM and creates three partitions. For my LTSP image purposes, I want a mbr disk label and one ext4 partition that fills the full disk. What is the easiest way to achieve this? Is it by using cloud-init or curtin?

I would like to see the autoinstall-quickstart guide updated with instructions on how to automate basic partitioning of a disk or multiple disks.

[Dan MacDonald (allcoms)]

Here's my non-working attempt to do this with cloud-init. I rolled this config into a seed.iso using cloud-localds:

#cloud-config
autoinstall:
  version: 1
  identity:
    hostname: ubuntu-server
    password: "$6$exDY1mhS4KUYCE/2$zmn9ToZwTKLhCw.b4/b.ZRTIZM30JZ4QrOQ2aOXJ8yk96xpcCof0kxKwuX1kqLG/ygbJ1f8wxED22bTL4F46P0"
    username: ubuntu

  disk_setup:
    /dev/sda:
      table_type: mbr
      layout: True
      overwrite: True

  fs_setup:
    - device: /dev/sda
      partition: 1
      filesystem: ext4

This config attempt is based on some examples I saw in the Disk setup section here:

https://cloudinit.readthedocs.io/en/latest/topics/examples.html

Maybe its just formatted wrong? I know how fussy YAML is.

[Dan MacDonald (allcoms)]

Since my previous comment I have learned that Ubuntu server saves its cloud-init install config, including disk partitioning in /var/log/installer/autoinstall-user-data

Great! So that's got me an almost fully working config. Now I just need to genericise it for use in my script which should only amount 3 things:

* Change gpt to mbr

* Remove the disks serial ID and any disk UUID sort of stuff

* Get cloud init to use the full space of the disk

Its that third point I'm not quite sure how to achieve currently. Subiquity inserts a size: field for each partition it creates into autoinstall-user-data. I tried removing the size: field hoping that it would default to using the full remaining space after the bios boot partition but subiquity didn't like this.

Does a disk size need to be specified for each partition? Is there no "use rest of space on disk" cloud-init function I can use when defining a partition?

Here's my updated, non-working attempt at a cloud-init file:

#cloud-config
autoinstall:
  apt:
    disable_components: []
    geoip: true
    preserve_sources_list: false
    primary:
    - arches:
      - amd64
      - i386
      uri: http://gb.archive.ubuntu.com/ubuntu
    - arches:
      - default
      uri: http://ports.ubuntu.com/ubuntu-ports
  drivers:
    install: false
  identity:
    hostname: ltspbase
    password: $6$aPM6zZtkZeD2rgbG$oSAwiMQ9GH/5gs5ZaNPlnIIpp6k7eFQ4ZSw7aTTNZRfnSMoUi7MRI3GXjalcB6ykXYF8SUQE5c6CPsvlXOC1w1
    realname: ubuntu
    username: ubuntu
  kernel:
    package: linux-generic
  keyboard:
    layout: gb
    toggle: null
    variant: ''
  locale: en_GB.UTF-8
  network:
    ethernets:
      enp0s3:
        dhcp4: true
    version: 2
  ssh:
    allow-pw: true
    authorized-keys: []
    install-server: false
  storage:
    config:
    - ptable: mbr
      path: /dev/sda
      wipe: superblock-recursive
      preserve: false
      name: ''
      grub_device: true
      type: disk
      id: disk-sda
    - device: disk-sda
      size: 1048576
      flag: bios_grub
      number: 1
      preserve: false
      grub_device: false
      type: partition
      id: partition-0
    - device: disk-sda
      wipe: superblock
      flag: ''
      number: 2
      preserve: false
      grub_device: false
      type: partition
      id: partition-1
    - fstype: ext4
      volume: partition-1
      preserve: false
      type: format
      id: format-0
    - path: /
      device: format-0
      type: mount
      id: mount-0
  updates: security
  version: 1

[Dan MacDonald (allcoms)]

The subiquity error I got when trying to use my latest cloud-init config

[Dan MacDonald (allcoms)]

Got it working!

I switched mbr back to gpt (I presume mbr isn't a valid option) and I set the disk size to -1 which has used all the space available so we can get away with a generic cloud-init file for my LTSP image creator script.

I would recommend adding the path to autoinstall-user-data to the autoinstall-quickstart guide with a few good tips on configuring it like setting disk size to -1 to use all the space.

#cloud-config
autoinstall:
  apt:
    disable_components: []
    geoip: true
    preserve_sources_list: false
    primary:
    - arches:
      - amd64
      - i386
      uri: http://gb.archive.ubuntu.com/ubuntu
    - arches:
      - default
      uri: http://ports.ubuntu.com/ubuntu-ports
  drivers:
    install: false
  identity:
    hostname: ltspbase
    password: GOESHERE
    realname: ubuntu
    username: ubuntu
  kernel:
    package: linux-generic
  keyboard:
    layout: gb
    toggle: null
    variant: ''
  locale: en_GB.UTF-8
  network:
    ethernets:
      enp0s3:
        dhcp4: true
    version: 2
  ssh:
    allow-pw: true
    authorized-keys: []
    install-server: false
  storage:
    config:
    - ptable: gpt
      path: /dev/sda
      wipe: superblock-recursive
      preserve: false
      name: ''
      grub_device: true
      type: disk
      id: disk-sda
    - device: disk-sda
      size: 1048576
      flag: bios_grub
      number: 1
      preserve: false
      grub_device: false
      type: partition
      id: partition-0
    - device: disk-sda
      size: -1
      wipe: superblock
      flag: ''
      number: 2
      preserve: false
      grub_device: false
      type: partition
      id: partition-1
    - fstype: ext4
      volume: partition-1
      preserve: false
      type: format
      id: format-0
    - path: /
      device: format-0
      type: mount
      id: mount-0
  updates: security
  version: 1

[Dan MacDonald (allcoms)]

It doesn't look like the desktop builds of 22.04 are using subiquity yet?

Is it not possible to get the desktop versions of Ubuntu to save a cloud-init config yet? Will the desktop version be switching to subiquity in the future?

[Dan MacDonald (allcoms)]

It doesn't seem possible to get Ubuntu server to install on an mbr / BIOS machine (VM) with a ext4 root partition but without a GRUB boot partition, which is what I expected but I wanted to try installing without one.

I can get the ltsp image generator script to work with a grub boot partition so its not a prob.

[Dan Bungert (dbungert)]

Thanks for the feedback @allcoms. These are good documentation suggestions.

Subiquity does prefer to use GPT at this time. We're working actively to improve MBR handling.

The GRUB boot partition of size 1 MiB and offset 1 MiB is a requirement when using GPT and BIOS. But it sounds like you're on MBR with BIOS? The GRUB boot partition shouldn't be required there.

> It doesn't look like the desktop builds of 22.04 are using subiquity yet?

Not yet. The "canary" ISO uses Subiquity and is the future architecture for Desktop. If you want to play with that for autoinstall purposes, please pass experimental-autoinstall on the kernel command line.

[Dan MacDonald (allcoms)]

Yes, I was aiming for a MBR BIOS VM. Its what LTSP wants and its also the default of VB VMs.

So subiquity / cloud-init should let me create mbr disks but its got issues? I will have another go at doing a MBR BIOS autoinstall next week.

The experimental-autoinstall option is good to know about. That should also get a mention in the quickstart guide.

Thanks

[Dan MacDonald (allcoms)]

It doesn't seem possible to create a disk with only one ext4 partition and no bios grub partition without having subiquity crash. I have tried using both a gpt and mbr partition label but both fail. Should I open a separate bug report for that?

#cloud-config
autoinstall:
  apt:
    disable_components: []
    geoip: true
    preserve_sources_list: false
    primary:
    - arches:
      - amd64
      - i386
      uri: http://gb.archive.ubuntu.com/ubuntu
    - arches:
      - default
      uri: http://ports.ubuntu.com/ubuntu-ports
  drivers:
    install: false
  identity:
    hostname: ltspbase
    password: "$6$exDY1mhS4KUYCE/2$zmn9ToZwTKLhCw.b4/b.ZRTIZM30JZ4QrOQ2aOXJ8yk96xpcCof0kxKwuX1kqLG/ygbJ1f8wxED22bTL4F46P0"
    realname: ubuntu
    username: ubuntu
  kernel:
    package: linux-generic
  keyboard:
    layout: gb
    toggle: null
    variant: ''
  locale: en_GB.UTF-8
  network:
    ethernets:
      enp0s3:
        dhcp4: true
    version: 2
  ssh:
    allow-pw: true
    authorized-keys: []
    install-server: false
  storage:
    config:
    - ptable: gpt
      path: /dev/sda
      wipe: superblock-recursive
      preserve: false
      name: ''
      grub_device: true
      type: disk
      id: disk-sda
    - device: disk-sda
      size: -1
      wipe: superblock
      flag: ''
      number: 1
      preserve: false
      grub_device: false
      type: partition
      id: partition-0
    - fstype: ext4
      volume: partition-0
      preserve: false
      type: format
      id: format-0
    - path: /
      device: format-0
      type: mount
      id: mount-0
  updates: security
  version: 1

[Dan MacDonald (allcoms)]

The crash log says "Invalid partition label type: mbr" when I tried using mbr.

[Dan MacDonald (allcoms)]

I have tried adding experimental-autoinstall to the kernel parameters for the 22.04 desktop installer but it doesn't create a /var/log/installer/autoinstall-user-data file. I tried with and without leaving the maybe-ubiquity kernel parameter in place too.

I was expecting it would use the subiquity installer with that option but it used the normal one.

[Dan Bungert (dbungert)]

> The crash log says "Invalid partition label type: mbr" when I tried using mbr.

Curtin wants "ptable: msdos" for that case. Supporting an alias for that valid to accept "ptable: mbr" is probably a good idea.

https://curtin.readthedocs.io/en/latest/topics/storage.html

> It doesn't seem possible to create a disk with only one ext4 partition and no bios grub partition without having subiquity crash. I have tried using both a gpt and mbr partition label but both fail. Should I open a separate bug report for that?

I haven't tested it, but that config looks good to me except for the above note. gpt should work also if you give it the 1MiB size & 1 MiB offset compat partition.

While it would be OK to open a new LP for this, it's a known issue that we could improve schema validation and catch errors like this in a more upfront manner. Up to you :)

> I have tried adding experimental-autoinstall to the kernel parameters for the 22.04 desktop installer but it doesn't create a /var/log/installer/autoinstall-user-data file. I tried with and without leaving the maybe-ubiquity kernel parameter in place too.

Don't expect too much of that yet. We know that the autoinstall bits aren't in place yet and have been prioritizing other work first. This is why I disabled autoinstall for the Canary ISO, until it's at least semi-working.

> I was expecting it would use the subiquity installer with that option but it used the normal one.

Ah, interesting. While the Subiquity Terminal UI is present on that ISO, we don't have things set up to actually run it. I'm not sure that UX has been decided for that case. I'm guessing that we'll end up booting to the Desktop and showing the last page of the Desktop Installer UI to show progress.