Installer crashed while trying to manually create a partition

[Frank Heimes]

While trying to manually creating a partition on a DASD FBA disk on an s390x z/VM system the installer crashed and the ssh connection got lost.
In the Terminal I found:

generating crash report
report saved to /var/crash/1680091796.384284496.ui.crash
Traceback (most recent call last):
  File "/snap/subiquity/4569/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/snap/subiquity/4569/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/snap/subiquity/4569/lib/python3.10/site-packages/subiquity/__main__.py", line 5, in <module>
    sys.exit(main())
  File "/snap/subiquity/4569/lib/python3.10/site-packages/subiquity/cmd/tui.py", line 141, in main
    asyncio.run(run_with_loop())
  File "/snap/subiquity/4569/usr/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/snap/subiquity/4569/usr/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
    return future.result()
  File "/snap/subiquity/4569/lib/python3.10/site-packages/subiquity/cmd/tui.py", line 139, in run_with_loop
    await subiquity_interface.run()
  File "/snap/subiquity/4569/lib/python3.10/site-packages/subiquity/client/client.py", line 403, in run
    await super().run()
  File "/snap/subiquity/4569/lib/python3.10/site-packages/subiquitycore/tui.py", line 318, in run
    await super().run()
  File "/snap/subiquity/4569/lib/python3.10/site-packages/subiquitycore/core.py", line 139, in run
    raise exc
  File "/snap/subiquity/4569/usr/lib/python3.10/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "/snap/subiquity/4569/usr/lib/python3/dist-packages/urwid/raw_display.py", line 416, in <lambda>
    wrapper = lambda: self.parse_input(
  File "/snap/subiquity/4569/usr/lib/python3/dist-packages/urwid/raw_display.py", line 515, in parse_input
    callback(processed, processed_codes)
  File "/snap/subiquity/4569/usr/lib/python3/dist-packages/urwid/main_loop.py", line 412, in _update
    self.process_input(keys)
  File "/snap/subiquity/4569/usr/lib/python3/dist-packages/urwid/main_loop.py", line 513, in process_input
    k = self._topmost_widget.keypress(self.screen_size, k)
  File "/snap/subiquity/4569/usr/lib/python3/dist-packages/urwid/wimp.py", line 651, in keypress
    return self._current_widget.keypress(size, key)
  File "/snap/subiquity/4569/lib/python3.10/site-packages/subiquity/ui/frame.py", line 36, in keypress
    return super().keypress(size, key)
  File "/snap/subiquity/4569/lib/python3.10/site-packages/subiquitycore/ui/container.py", line 178, in keypress
    upkey = self.focus.keypress(tsize, downkey)
  File "/snap/subiquity/4569/lib/python3.10/site-packages/subiquitycore/view.py", line 157, in keypress
    key = super().keypress(size, key)
  File "/snap/subiquity/4569/lib/python3.10/site-packages/subiquitycore/ui/stretchy.py", line 159, in keypress
    return self.top_w.keypress(top_size, key)
  File "/snap/subiquity/4569/usr/lib/python3/dist-packages/urwid/decoration.py", line 840, in keypress
    return self._original_widget.keypress((maxcol,maxrow-top-bottom), key)
  File "/snap/subiquity/4569/usr/lib/python3/dist-packages/urwid/decoration.py", line 622, in keypress
    return self._original_widget.keypress(maxvals, key)
  File "/snap/subiquity/4569/usr/lib/python3/dist-packages/urwid/container.py", line 1626, in keypress
    key = self.focus.keypress(tsize, key)
  File "/snap/subiquity/4569/usr/lib/python3/dist-packages/urwid/container.py", line 2316, in keypress
    key = w.keypress((mc,) + size[1:], key)
  File "/snap/subiquity/4569/usr/lib/python3/dist-packages/urwid/decoration.py", line 840, in keypress
    return self._original_widget.keypress((maxcol,maxrow-top-bottom), key)
  File "/snap/subiquity/4569/usr/lib/python3/dist-packages/urwid/decoration.py", line 622, in keypress
    return self._original_widget.keypress(maxvals, key)
  File "/snap/subiquity/4569/lib/python3.10/site-packages/subiquitycore/ui/container.py", line 178, in keypress
    upkey = self.focus.keypress(tsize, downkey)
  File "/snap/subiquity/4569/usr/lib/python3/dist-packages/urwid/decoration.py", line 622, in keypress
    return self._original_widget.keypress(maxvals, key)
  File "/snap/subiquity/4569/lib/python3.10/site-packages/subiquitycore/ui/container.py", line 178, in keypress
    upkey = self.focus.keypress(tsize, downkey)
  File "/snap/subiquity/4569/usr/lib/python3/dist-packages/urwid/wimp.py", line 543, in keypress
    self._emit('click')
  File "/snap/subiquity/4569/usr/lib/python3/dist-packages/urwid/widget.py", line 461, in _emit
    signals.emit_signal(self, name, self, *args)
  File "/snap/subiquity/4569/usr/lib/python3/dist-packages/urwid/signals.py", line 265, in emit
    result |= self._call_callback(callback, user_arg, user_args, args)
  File "/snap/subiquity/4569/usr/lib/python3/dist-packages/urwid/signals.py", line 295, in _call_callback
    return bool(callback(*args_to_pass))
  File "/snap/subiquity/4569/lib/python3.10/site-packages/subiquitycore/ui/form.py", line 512, in _click_done
    emit_signal(self, 'submit', self)
  File "/snap/subiquity/4569/usr/lib/python3/dist-packages/urwid/signals.py", line 265, in emit
    result |= self._call_callback(callback, user_arg, user_args, args)
  File "/snap/subiquity/4569/usr/lib/python3/dist-packages/urwid/signals.py", line 295, in _call_callback
    return bool(callback(*args_to_pass))
  File "/snap/subiquity/4569/lib/python3.10/site-packages/subiquity/ui/views/filesystem/partition.py", line 560, in done
    handler(self.disk, spec, partition=self.partition, gap=self.gap)
  File "/snap/subiquity/4569/lib/python3.10/site-packages/subiquity/common/filesystem/manipulator.py", line 235, in partition_disk_handler
    self.create_partition(disk, gap, spec)
  File "/snap/subiquity/4569/lib/python3.10/site-packages/subiquity/common/filesystem/manipulator.py", line 90, in create_partition
    part = self.model.add_partition(
  File "/snap/subiquity/4569/lib/python3.10/site-packages/subiquity/models/filesystem.py", line 1600, in add_partition
    raise Exception(
Exception: ('size %s or offset %s not aligned to %s', 2028298305, 1048576, 1048576)
Connection to hwe0005 closed.

But I was able to reconnect to the installer via ssh to gather the logs, which I attached here.

[Frank Heimes (fheimes)]

FWIW when I reconnect and proceed the installation, but now keeping the default (means not trying a custom layout), I don't hit the crash again and the installation completes successfully.

[Ubuntu QA Website (ubuntuqa)]

This bug has been reported on the Ubuntu ISO testing tracker.

A list of all reports related to this bug can be found here:
https://iso.qa.ubuntu.com/qatracker/reports/bugs/2013201

[Frank Heimes (fheimes)]

It might well be a problem with the specific disk geometry of the DASD FBA disks,
since trying a very similar installation just with DASD ECKDs worked fine.

Some more background about my test/use case:

Disk storage is usually pretty expensive on mainframes (at least up to the point in time where there was still no NVMe support), hence customers (SAN admins) have often by default relatively small disks, and therefore (OS) admins often use LVM to create bigger disks.
Hence my use case is to have 3 or 4 (smaller) disks, create a small partition on the first one just for /boot (that is where the crash happened) and create an empty/unsued/unformatted partition for the rest of the space of the same disk, as well as for all other disks with empty/unused partitions. Then create a VG across all the empty/unused partitions and on top one big LV, used as root.

[Dan Bungert (dbungert)]

Hi Frank, thanks for the bug report.

The size listed there, 2028298305, is a bit unexpected. Subiquity expects sizes to be aligned to even multiples of 1MiB, and it enforces this with rounding, but it seems that didn't happen this time.

Do you know where that number came from? Was that something you manually chose?

[Frank Heimes (fheimes)]

Hi Dan, many thx for having a look.

Yes, I specified the disk size myself, but on a more raw level.

I did some more tests:

I have these disks:

▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
  Storage configuration [ Help ]
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
  To continue you need to: Mount a filesystem at /

  AVAILABLE DEVICES ▴

    DEVICE TYPE SIZE
  [ /dev/dasda local disk 15.999G ▸ ] █
    free space 15.998G ▸ █
                                                                             █
  [ /dev/dasdb local disk 15.999G ▸ ] █
    free space 15.998G ▸ █
                                                                             █
  [ /dev/dasdc local disk 15.999G ▸ ]
    free space 15.998G ▸

  [ /dev/dasdd local disk 16.000G ▸ ]
    free space 15.999G ▸ ▾

Hence it's kind of a "natural feeling" to specify the size of the right from the point "0.998" as size for a /boot (or if this is not sufficient the rest plus 1Gig: 1.998 see below).

Here with "0.998":

▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
  Storage configuration [ Help ]
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
  To continue you need to: Mount a filesystem at /

   ┌───────────────── Adding MSDOS partition to /dev/dasda ─────────────────┐
   │ │
   │ Size (max 15.998G): 0.998 │
   │ │
   │ │
   │ Format: [ ext4 ▾ ] │
   │ │
   │ │
   │ Mount: [ / ▾ ] │
   │ │
   │ │
   │ [ Create ] │
   │ [ Cancel ] ...

[Dan Bungert (dbungert)]

Hi Chandan,

I'm unclear why this was marked fix committed. Moving back to a new state for now, thanks.

[Frank Heimes (fheimes)]

@nigthshadow07 May I ask why have you changed this to Fix Released?
Have you successfully verified the situation on such an s390x system with FBA disks?

[Frank Heimes (fheimes)]

I'm seeing this issue now also with the candidate for 22.04.3.
On a DASD ECKD with 6.876G size, I manually specified a /boot partition of 1.876G
and I got the message that the size was rounded to 1.876G, that's what I specified).
and after that the installation failed.
See the attached screenshot - I'm attaching the logs in the next comment ...

[Frank Heimes (fheimes)]

Here are the logs ...

Please notice that it this time also happens on an LPAR installation using DASD ECKD disks - so it's more general. I also updated the bug title.

After the installer crash I could relogin (via ssh) and could continue the installation.
I followed the same approach, but chose a slightly different size for /boot this time (I just used "1" for 1G - to make rounding more easy, or ideally to avoid it at all), and the crash did _not_ happen anymore.

So like already discussed before, this is highly likely a rounding issue, that happen in some cases.
But it's not limited to DASD FBA, but can also happen with DASD ECKD.

[Olivier Gayot (ogayot)]

I can reproduce the issue - and I don't think it is specific to s390x.

There is indeed rounding happening but the size value is _stored_ rounded which is the problem.

1.876GiB is not a multiple of 1MiB. Actually, it does not even represent a number of bytes:

> 1.876 * 1024 ** 3 = 2014339661.824

The closest aligned value would be 2015363072 but humanize_size(2015363072) would return 1.876G.

We should store the value as a number of bytes instead.

[Frank Heimes (fheimes)]

Hi Olivier, glad that you could reproduce (and even more glad that it was on non-s390x).

I agree that the 1.876GiB does not nicely translate into a proper byte size.
But the installer screen suggests to specify the size in "G".
And there is a certain 'force' - in case of a disk with for example a size of xxx.876G, and the need for a small /boot - that makes one specifying 1.876GiB or 0.876GiB =)

[Olivier Gayot (ogayot)]

> I agree that the 1.876GiB does not nicely translate into a proper byte size.
> But the installer screen suggests to specify the size in "G".
> And there is a certain 'force' - in case of a disk with for example a size of xxx.876G, and the need for a small /boot - that makes one specifying 1.876GiB or 0.876GiB =)

Sure. I mean it's on us. We're doing the conversion to the nearest aligned value (which is the intended behavior) but then by showing it back to the user nicely, we're replacing that value by something rounded that is not aligned.

Subiquity should internally store the accurate / aligned value - even if it shows a rounded value to the user (in my opinion).

[Olivier Gayot (ogayot)]

The error message was introduced in https://github.com/canonical/subiquity/commit/44fca22175eb92a943130ec097c815297fd36ec0

But the unaligned size was already a thing - which used to be okay because add_partition aligns the value up.

[Frank Heimes (fheimes)]

Yes, that can be the case (~ 23.04.x) - since I bumped into this first time with lunar, but haven't noticed such an issue prior to lunar.

[Frank Heimes (fheimes)]

Retested again with a new subiquity version from the beta channel which worked fine for me - thx !!

[Olivier Gayot (ogayot)]

https://github.com/canonical/subiquity/pull/1759

[Frank Heimes (fheimes)]

Everything seem to be fixed with the 22.04.3 image from Aug 10th.
I wasn't able to provoke this situation anymore.