failure to boot groovy daily

[Chris Guiver]

When reported the groovy daily was failing on most boxes..

Originally occurred if ISO is written via `dd`, `mkusb`, `Startup Disk Creator`, or `gnome-disks` (Restore disk image)

Box still impacted are (owned by sudodus/nio-wiklund)

* Lenovo V130

and owned by Leó Kolbeinsson

* Lenovo V14 IIL,Intel Core i3-1005G!,8GB,256GB SSD

---
Original detail follows
(with minimal edits; these boxes now boot groovy ISOs)

This is very similar to https://bugs.launchpad.net/bugs/1883040

Boxes that have failed to boot it are

dell [optiplex] 755 (c2d-e8300, 8gb, amd/ati radeon rv610/radeon hd2400 pro/xt)
dell [optiplex] 755 (c2d-e6850, 5gb, amd/ati radeon rv516/x1300/x1550)
dell [optiplex] 780 (c2q-q9400, 4gb, amd/ati cedar radeon hd 5000/6000/7350/8350)
hp dc7700 (c2d-e6320, 5gb, nvidia quadro nvs 290)
hp 8200 elite sff (i5-2400, 8gb, nvidia quadro 600)
sony vaio svp112a1cw (i5-9400u, 4gb, intel haswell-ULT)

-- sudodus' boxes
dell Precision M4800
dell Latitude E7240
Toshiba Satellite Pro C850-19w
HP Probook 6450b - works now

-- leok's boxes
Acer [Aspire] E3-111-P60S (Pent.N3530, 4GB, Intel HD Graphics, Realtek RTL8111/81681/8411 GB Ethernet, Qualcomm Atheros AR9462 Wireless, Bluetooth Atheros A315-53, 500 GB hd)

Dell [Optiplex] 7010 ( i5-3470 , 16 GB, Intel Graphics 2500, Intel 82579LM GB Ethernet ,1TB hd) VirtualBox

Dell [Inspiron] 3521, (i3-3217U, 4GB, Intel HD Graphics 4000, Intel HM76 chipset 10/100 Mbps ethernet controller integrated on system board, WiFi 802.11 b/g/N, Bluetooth 4.0, 500 GB hd)
--

The ISO was written twice to two different thumb-drives. Same issue each time on same boxes.

On a number of boxes it’s wanting me to download aka https://bugs.launchpad.net/ubuntu-cdimage/+bug/1883040 however it’s done that on boxes not impacted by that bug, which makes me think thumb-drive/squashfs errs related. Also results of boot appeared different on varying boxes (inconsistent; dc7700 reported no thumb-drive; d755-5 also did that sometimes, sometimes it got to wanting to download - those two boxes were impacted by prior report; the remaining boxes were more consistent in response..; but if trouble reading data on thumb-drive then the slower boxes (dc7700/d755-5) may have more issues & thus be less consistent?)

I'll file this as a bug report so I can close my failed QA-tests, but I'm considering changing the status to 'incomplete', and re-testing tomorrow, OR it needs me to re-write ISO from a different box to a third-thumb-drive as I don't think I've ruled out media issues given Leok's report.

[Chris Guiver (guiverc)]

Decided to mark incomplete, I'll re-test tomorrow, and will revive this report if I have troubles again... Leok's successful QA-test report is reason for the change..

[Ubuntu QA Website (ubuntuqa)]

This bug has been reported on the Ubuntu ISO testing tracker.

A list of all reports related to this bug can be found here:
http://iso.qa.ubuntu.com/qatracker/reports/bugs/1886148

[Chris Guiver (guiverc)]

Next day & updated ISO [2020-06-03 now] and same issue.
have tried 2x thumb-drives, one from yesterday & a new one.
thumb-drives were written firstly by primary d960 (tested on listed boxes), then one was re-written using sonycrap

sony vaio svp112a1cw (i5-9400u, 4gb, intel haswell-ULT)
hp dc7700 (c2d-e6320, 5gb, nvidia quadro nvs 290)
hp 8200 elite sff (i5-2400, 8gb, nvidia quadro 600)
dell [optiplex] 755 (c2d-e6850, 5gb, amd/ati radeon rv516/x1300/x1550)

I've mentioned on #lubuntu-devel my issues, kc2bez will hopefully test..
I've re-ran zsync script & it reports my ISO is 100% complete, so will write a new (fifth) thumb-drive & re-test...

[Chris Guiver (guiverc)]

Another thumb-drive written on sony crapbook has failed too.
stdin: Not a typewriter
/init: line 49: can't open /dev/sr0: No medium failed
/init: line 49: can't open /dev/sr0: No medium failed
Unable to find a medium container a live file system
Attempt interactive netboot from a URL?
yes no (default yes):

which is acting like a squashfs error/bad thumb-drive, but five thumb drives written by two boxes? sha256sum is expected so I don't see the issue

[sudodus (nio-wiklund)]

Things are happening to the boot structure. It seems grub is used also in BIOS mode (which is OK for me).

But it is far from ready to use in this daily iso file.

Cloned drive: both UEFI and BIOS mode: Failed in my Dell Precision M4800. This computer is getting old, but it is not a low end computer.

/init: line 49: can't open /dev/sr0: No medium found
Unable to find a medium container a live file system
Attempt to interactive netboot from a URL?
yes no (default yes):

[sudodus (nio-wiklund)]

Furthermore, I tried to make a persistent live drive with mkusb (mkusb-dus) using the default settings, which has been working for years (until a few days ago also in Lubuntu Groovy).

It failed completely in BIOS mode at the grub rescue prompt :-(

It reached a live-only working desktop in UEFI mode :-/

It does not work at all using mkusb with 'usb-pack-efi', another strong indicator that the boot structure is not what it has been for a long time.

So it seems that either the boot structure has changed a lot by intention, or there is some severe bug. I think we an explanation from a developer at Canonical. I would guess that there are big changed in the package casper. (I don't think that the Lubuntu team is responsible for this bug.)

[sudodus (nio-wiklund)]

typo fix:
I think we an explanation from a developer at Canonical.
-->
I think we need an explanation from a developer at Canonical.

[Leó Kolbeinsson (leok)]

I can confirm this boot failure occurring in Kubuntu as well as Lubuntu (i.e. installers ubiquity and calamares). Failed machines :

Acer [Aspire] E3-111-P60S (Pent.N3530, 4GB, Intel HD Graphics, Realtek RTL8111/81681/8411 GB Ethernet, Qualcomm Atheros AR9462 Wireless, Bluetooth Atheros A315-53, 500 GB hd)

Dell [Optiplex] 7010 ( i5-3470 , 16 GB, Intel Graphics 2500, Intel 82579LM GB Ethernet ,1TB hd) VirtualBox

[sudodus (nio-wiklund)]

I tested in standard Ubuntu Desktop and in Xubuntu. These are affected in the same way, so I think this bug affects all desktop flavours.

[Steve Langasek (vorlon)]

Does the image boot as expected when being copied to the USB stick, instead of being modified by a tool such as mkusb?

[sudodus (nio-wiklund)]

No, when cloned, it does not work at all. This is described in comment #5.

The attempt with mkusb to create a persistent live drive gave me a drive that worked live (only live, no persisence) in UEFI mode.

[sudodus (nio-wiklund)]

I continued testing, this time with the newest computer, that I have access to, a consumer class Lenovo V130 with an nvme drive that boots in UEFI mode with secure boot,

https://dustinweb.azureedge.net/media/494085/v130.pdf

This computer failed with a similar but slightly different output.

This time I tested Ubuntu Desktop Groovy

#perms size date time file-name
-rw------- 2,6G 2020-07-04 08:21 "groovy-desktop-amd64.iso"

and the output on the screen was

[0.460294] Initramfs unpacking failed: Decoding failed
stdin: Invalid argument
cat: read error Input/output error
stdin: Invalid argument [repeated many times, almost filling the whole screen]
Unable to find a medium container a live file system
Attempt to interactive netboot from a URL?
yes no (default yes):

[sudodus (nio-wiklund)]

I should add that I downloaded the iso file via zsync, so the checksum should be OK, and it was cloned to the USB drive. Maybe it helps you to see

'Initramfs unpacking failed: Decoding failed'

[Chris Guiver (guiverc)]

Lubuntu daily [2020-07-04] fails again on
hp 8200 elite sff (i5-2400, 8gb, nvidia quadro 600)

stdin: Not a typewriter
/init: line 49: can't open /dev/sr0: No medium failed
/init: line 49: can't open /dev/sr0: No medium failed
Unable to find a medium container a live file system
Attempt interactive netboot from a URL?
yes no (default yes):

Written to thumb-drive with

`sudo dd bs=4M oflag=sync status=progress of=/dev/sdb if=/de2900/lubuntu_64/groovy-desktop-amd64.iso`

[sudodus (nio-wiklund)]

The current daily Ubuntu Server Groovy iso file fails in the same or very similar way and the desktop iso files, so it is also affected by this bug (tested from a cloned USB pendrive).

#perms size date time file-name
-rw------- 960M 2020-07-05 07:27 "groovy-live-server-amd64.iso"

And there is no improvement in the current daily Ubuntu Desktop Groovy iso file

-rw------- 2,6G 2020-07-05 08:20 "groovy-desktop-amd64.iso"

[sudodus (nio-wiklund)]

I could find a working version of Ubuntu Desktop for PC computers at this link

http://cdimage.ubuntu.com/daily-live/20200629/

So June 29 the daily iso file

#perms size date time file-name
-rw------- 2,6G 2020-06-29 08:27 "groovy-desktop-amd64.iso"

was still good, a cloned USB drive booted both in UEFI and BIOS mode in my Dell Precision M4800 (with the default main menu entry, no tweaks were necessary).

But I noticed, that the automatically created third partition was shown without a label and without file system by

lsblk -fm

which indicates, that something fishy has started a that date. (I expected an ext4 file system with the label 'writable'.)

[Leó Kolbeinsson (leok)]

Lubuntu daily [2020-07-05]

Successful boot and good install on UEFI (no secure boot) :

InWin BL641 i5-10400,16GB,Intel UHD Graphics 630,Ethernet,Intel 660p 512GB M.2 NVMe hd

Successful boot and good install in VirtualBox :

Dell [Optiplex] 7010 ( i5-3470 , 16 GB, Intel Graphics 2500, Intel 82579LM GB Ethernet ,1TB hd)

Failed to boot on older laptop:

Dell [Inspiron] 3521, (i3-3217U, 4GB, Intel HD Graphics 4000, Intel HM76 chipset 10/100 Mbps ethernet controller integrated on system board, WiFi 802.11 b/g/N, Bluetooth 4.0, 500 GB hd)

[sudodus (nio-wiklund)]

@leok,

Please tell us what method/tool that you used when you created the boot drive, that booted in UEFI mode.

[Chris Guiver (guiverc)]

Fails to boot with Lubuntu groovy daily [2020-07-06] on

- uefi
sony vaio svp112a1cw (i5-9400u, 4gb, intel haswell-ULT)
hp 8200 elite sff (i5-2400, 8gb, nvidia quadro 600)
- bios
dell [optiplex] 780 (c2q-q9400, 4gb, amd/ati cedar radeon hd 5000/6000/7350/8350)

written via my usual `dus` & live-only

[sudodus (nio-wiklund)]

@vorlon,

Please explain exactly what you mean by

'Does the image boot as expected when being copied to the USB stick?'

Can *you* boot from a USB drive with an Ubuntu live system created from a current daily groovy amd64 iso file? In that case please explain how you create that USB system, what computer it is and other relevant details.

[sudodus (nio-wiklund)]

The current daily Ubuntu Groovy Desktop file (2020-07-07) works for me in VirtualBox, when booted from the iso file seen as a virtual optical drive.

But it does not work in the same computer (Dell Precision M4800) on real metal, neither in UEFI mode nor BIOS mode, when cloned from the iso file to a USB drive.

[Steve Langasek (vorlon)]

This problem is also perfectly reproducible in a VM, when attaching the image as a disk instead of as a CDROM device. Tested under qemu as both a virtio disk and an ide disk.

[Michael Hudson-Doyle (mwhudson)]

Preeeeetty sure this is a problem with debian-cd not casper.

[sudodus (nio-wiklund)]

I'm glad that there is work in progress, and I am willing to help testing, whenever you have something to test :-)

[sudodus (nio-wiklund)]

I found another mode in which the current groovy iso files make bootable drives: 'grub-n-iso' alias isoboot. I can make a bootable USB drive that boots

- both in UEFI and BIOS mode
- both live-only and persistent live

when booting into a 'grub template' and then into an iso file according to the following link,

https://help.ubuntu.com/community/Installation/iso2usb/isoboot

So this 'grub-n-iso' method works, but a cloned drive from the sane iso file fails (when tested in the same Dell Precision M4800 as used for the tests described above).

[Steve Langasek (vorlon)]

A new image is available now with a partial solution for this issue; please test http://cdimage.ubuntu.com/lubuntu/daily-live/20200708.1/

[Chris Guiver (guiverc)]

:)

20200708.1 boots on sony vaio svp112a1cw (i5-9400u, 4gb, intel haswell-ULT)

I'll test on other boxes as I can.

[Chris Guiver (guiverc)]

Lubuntu groovy daily [20200708.1] boots on

dell [optiplex] 755 (c2d-e6850, 5gb, amd/ati radeon rv516/x1300/x1550)
dell [optiplex] 755 (c2d-e8300, 8gb, amd/ati radeon rv610/radeon hd2400 pro/xt)
dell [optiplex] 780 (c2q-q9400, 8gb, amd/ati cedar radeon hd 5000/6000/7350/8350)
hp 8200 elite sff (i5-2400, 8gb, nvidia quadro 600)
and already mentioned sony vaio svp112a1cw (i5-9400u, 4gb, intel haswell-ULT)

I had issues with

hp dc7700 (c2d-e6320, 5gb, nvidia quadro nvs 290)

That box kept very quickly booting HDD even if I told it to boot thumb-drive. I could however boot an Xubuntu groovy ISO dated 2020-07-01 (picked at random) on it, so maybe this does mean this machine is still impacted !!??

[Chris Guiver (guiverc)]

Of note on 20200708.1 booting on hp dc7700

The thumb-drive used has an orange LED, and it looks like it's read (LED flashing) before it goes to HDD & shows hdd's grub options. It's brief though as mentioned in comment #29

[sudodus (nio-wiklund)]

Cloned Groovy USB drives boot again, at least in some of my computers :-)

+ I tested in my Dell Precision M4800 and it works as usual in UEFI mode (and via grub) in BIOS mode.

- But the Lenovo V130 does not boot in UEFI mode and secure boot - When I select the USB drive from the temporary menu, it skips to the internal drive. This is a different failure mode compared to the previous iso files. (This computer is willing to boot from the grub-n-iso system of comment #26 and from cloned drives made from groovy iso files from June.)

https://dustinweb.azureedge.net/media/494085/v130.pdf

+ I tested in my Toshiba laptop, and it works as usual in UEFI mode with and without secure boot (and via grub) in BIOS.

http://www.toshiba.se/laptops/satellite-pro/c850/satellite-pro-c850-19w/

+ It works in a HP Probook 6450b with an Intel i5 M520 CPU, tested only in BIOS mode.

[Chris Guiver (guiverc)]

I also had Lubuntu groovy daily [20200708.1] boot on

lenovo thinkpad sl510 (c2d-t6570, 2gb ram, i915)
lenovo thinkpad x201 (i5-m520, 4gb, i915)

but it failed to boot on

hp dc7900 (c2d-e8400, 4gb, intel 4 series integrated i915)

(same reaction as hp dc7700; I see the LED flash then it quickly proceeds to boot the hdd-installed OS on selecting to boot USB device)

[Leó Kolbeinsson (leok)]

Further tests on Lubuntu and Xubuntu Groovy daily [20200708.1]

Dell [Inspiron] 3521, (i3-3217U, 4GB, Intel HD Graphics 4000, Intel HM76 chipset 10/100 Mbps ethernet controller integrated on system board, WiFi 802.11 b/g/N, Bluetooth 4.0, 500 GB hd)

Acer [Aspire] E3-111-P60S (Pent.N3530, 4GB, Intel HD Graphics, Realtek RTL8111/81681/8411 GB Ethernet, Qualcomm Atheros AR9462 Wireless, Bluetooth Atheros A315-53, 500 GB hd)

Both machines fail to boot in BIOS mode but can boot successfully in UEFI+secure boot mode.

Also tested :
Dell [Optiplex] 7010 ( i5-3470 , 16 GB, Intel Graphics 2500, Intel 82579LM GB Ethernet ,1TB hd)
in a VirtualBox (BIOS) and this booted successfully

[Michael Hudson-Doyle (mwhudson)]

Hmm, interesting. I don't have any idea myself how to debug a boot failure like this I'm afraid. I do have another change queued up that makes the images more like the ones from June: https://code.launchpad.net/~mwhudson/debian-cd/half-the-grubs/+merge/387085 -- but I don't at all understand why this would affect bootability. I guess we'll see if it helps...

[sudodus (nio-wiklund)]

Let us continue like this: *You* make the images 'more like the ones from June', and *we* (iso-testers) test how it works in our computers.

I have a good experience from booting USB drives via grub also in BIOS mode, so I think and hope that we can keep that feature. Via the link in comment #26 you can see a configuration that works (both in UEFI mode and BIOS mode).

[sudodus (nio-wiklund)]

My Lenovo V130 does not boot from a cloned drive, and did not boot from a persistent live drive by mkusb-dus. But with today's iso file, there is progress. It boots into a persistent live system with the boot option 'upefi', usb-pack-efi, (and there is persistence). So another way to see what works is to look at

- a persistent live drive, and particularly the boot partition,

- the tool that creates it, the bash shellscript 'dus-persistent'

There are details at

https://askubuntu.com/questions/1184161/what-script-is-used-for-creating-usbboot-partition-and-installing-grub-in-mkusb/1184175#1184175

When things have improved further and also the problematic computers boot from cloned iso files, they will probably also boot when mkusb used the default settings, where the system is created instead of extracted from the usb-pack-efi file.

[Leó Kolbeinsson (leok)]

Interesting indeed...and remember we also have users that create USB drives in Windows or macOS.

[sudodus (nio-wiklund)]

The boot performance of the Ubuntu Groovy iso file on my computers are the same with today's daily iso file,

#perms size date time file-name
-rw------- 2,6G 2020-07-10 08:27 "groovy-desktop-amd64.iso"

as with yesterday's iso file. At least, I could not see any difference. Most of my computers boot from a USB drive when cloned, but there are still problems with the Lenovo V130.

This Lenovo computer is set to boot in UEFI mode with secure boot.

- The problem is that it cannot boot from a USB drive *cloned* from the current Ubuntu Groovy iso file dated 2020-07-10 08:27. Instead it skips to the internal drive and its grub menu (even when I select the USB drive from a temporary menu). This is the same problem as with the previous daily iso file.

- It works as intended when mkusb uses usb-pack-efi to create a persistent live drive.

[Chris Guiver (guiverc)]

I booted Lubuntu groovy [20200709; so not the latest!] successfully on

dell [optiplex] 960 (c2q-q9400, 8gb, amd/ati cedar radeon hd 5000/6000/7350/8350)
motion computing j3400 (c2d-u9400, 4gb, intel mobile 4 series)

[Chris Guiver (guiverc)]

Lubuntu groovy [20200710] on dc7700

today's daily was written with `gnome-disks` as I noted a quote by @xnox on community.ubuntu.com stating "restore a disk image" generally works .... (the 'generally' wording covers it though, it's booting on most of my boxes)

[Chris Guiver (guiverc)]

I tried boot Lubuntu groovy [20200710] on dc7900
hp dc7900 (c2d-e8400, 4gb, intel 4 series integrated i915)
It failed with this issue as expected :(

HOWEVER in comment #29 I'd grabbed a random thumb-drive for a test boot, which was (redS) Xubuntu groovy ISO dated 2020-07-01 which booted normally (and is running on dc7700 to validate date from /etc/apt/sources.list), but that thumb-drive (Xubuntu DAILY [20200701]) won't boot on dc7900 either !!

I grabbed another few random thumb-drives (debian buster lxqt & knoppix 8.6 also based on buster, and ubuntu 20.04 [possibly a late RC]) which booted normally

[Leó Kolbeinsson (leok)]

Further tests on Lubuntu Groovy daily [20200710]

For testing purposes I created boot media on Windows 10 box using Rufus 3.11

Tested on the following 2 boxes:

Acer [Aspire] E3-111-P60S (Pent.N3530, 4GB, Intel HD Graphics, Realtek RTL8111/81681/8411 GB Ethernet, Qualcomm Atheros AR9462 Wireless, Bluetooth Atheros A315-53, 500 GB hd)

Dell [Inspiron] 3521, (i3-3217U, 4GB, Intel HD Graphics 4000, Intel HM76 chipset 10/100 Mbps ethernet controller integrated on system board, WiFi 802.11 b/g/N, Bluetooth 4.0, 500 GB hd)

Both machines failed to boot from BIOS mode and booted successfully from UEFI + secure boot

now the good news:

I then created new media on a Lubuntu 20.04 LTS machine using the "Startup Disk Creator"

Retested the above 2 machines and bot booted successfully in BIOS and UEFI+secure boot modes.

[Leó Kolbeinsson (leok)]

Comment #44 last paragraph should read:

Retested the above 2 machines and both booted successfully in BIOS and UEFI+secure boot modes.

[Leó Kolbeinsson (leok)]

Re comment # 22

I can confirm that the current daily Lubuntu Groovy Desktop file (2020-07-107) works in VirtualBox, when booted from the iso file seen as a virtual optical drive.

Tested on 2 machines in VirtualBox
1. InWin BL641 i5-10400,16GB,Intel UHD Graphics 630,Ethernet,Intel 660p 512GB M.2 NVMe hd

2. Dell [Optiplex] 7010 ( i5-3470 , 16 GB, Intel Graphics 2500, Intel 82579LM GB Ethernet ,1TB hd)

However no attempt was made to install on these machines "barebone".

[Leó Kolbeinsson (leok)]

Further testing Lubuntu Groovy daily [20200712]

Same tests and results as in comment # 44

As far as I can see after today's tests - using the "startup disk creator" in Lubuntu 20.04 there are no BIOS mode boot problems on my older machines - running another live media creator i.e. Rufus in Windows results in BIOS mode boot failure.

[sudodus (nio-wiklund)]

The boot performance of the Lubuntu Groovy iso file on my computers are the same with today's current daily iso file,

#perms size date time file-name
-rw------- 1,8G 2020-07-12 16:56 "groovy-desktop-amd64.iso"
as with the previous iso files. At least, I could not see any difference. Most of my computers boot from a USB drive when cloned, but there are still problems with the Lenovo V130.

This Lenovo computer is set to boot in UEFI mode with secure boot.

- The problem is that it cannot boot from a USB drive *cloned* from the current Lubuntu Groovy iso file dated 2020-07-10 08:27. Instead it skips to the internal drive and its grub menu (even when I select the USB drive from a temporary menu). This is the same problem as with the previous daily iso file.

- It works as intended when mkusb uses usb-pack-efi to create a persistent live drive.

[sudodus (nio-wiklund)]

Now that a new Lubuntu Groovy iso file is uploaded I tested it as well as the current Focal iso file,

#perms size date time file-name
-rw------- 1,8G 2020-07-13 16:53 "focal-desktop-amd64.iso"
-rw------- 1,8G 2020-07-13 17:00 "groovy-desktop-amd64.iso"

The boot performance of a cloned USB drive made from this new Lubuntu Groovy iso file on my computers is the same as with the previous one: It works except with the Lenovo V130.

For comparison I tested the current daily Focal iso file and it works well also with the Lenovo V130.

[Chris Guiver (guiverc)]

Lubuntu groovy daily 2020-07-13 (identical file as per #49)

I used (due comment #47) `Startup Disk Creator` on my main box to write this media and it's still failing on

hp dc7700 (c2d-e6320, 5gb, nvidia quadro nvs 290)
hp dc7900 (c2d-e8400, 4gb, intel 4 series integrated i915)

The media is good, booting normally (passing checks) on
dell [optiplex] 755 (c2d-e6850, 5gb, amd/ati radeon rv516/x1300/x1550)

On both old HP boxes I get bright/faster pattern of flashes on the thumb-drive very briefly on selecting USB DEVICE for booting, then I get the hdd's grub menu appear and flashing on thumb-drive returns to normal as if BIOS/box firmware gave up on usb-media.

In my opinion, media written via `dd`, `mkusb` & `Startup Disk Creator` is identical (I don't use persistence or options in QA-testing so simple 'live' media with mkusb/dus)

No change to prior attempts since this bug was discovered on the dc7700 & dc7900, except initially it failed to boot on most if not all my boxes when initially reported.

[Chris Guiver (guiverc)]

Description was edits:

1: current/up-to-date summary @ top
my boxes still impacted
---
2: Original detail

original detail now includes Sudodus' boxes, at least some of them.
quite a bit of my 'thoughts/doubts' were removed.

[sudodus (nio-wiklund)]

@guiverc,

I think you have confused leok's boxes with my boxes. Please modify the summary @top.

I have tested this bug with a USB drive *cloned* from a Groovy daily iso file in a

+ Dell Precision M4800 (and Dell Latitude E7240 - not reported before, but behaves like the other Dell) - work now

+ Toshiba Satellite Pro C850-19w - works now

+ HP Probook 6450b - works now

- Lenovo V130 - *fails* now

[Leó Kolbeinsson (leok)]

@guiverc

Further to @sudodus comment #52

That is correct about the confusion of boxes.Also please note that my
Dell [Optiplex] 7010 ( i5-3470 , 16 GB, Intel Graphics 2500, Intel 82579LM GB Ethernet ,1TB hd) has always booted successfully in VirtualBox.

After further testing Lubuntu groovy daily 2020-07-13

I have two boxes failing in BIOS mode when media is created on a Windows box with Rufus and/or
Universal-USB-Installer

1. Acer [Aspire] E3-111-P60S (Pent.N3530, 4GB, Intel HD Graphics, Realtek RTL8111/81681/8411 GB Ethernet, Qualcomm Atheros AR9462 Wireless, Bluetooth Atheros A315-53, 500 GB hd)

2. Dell [Inspiron] 3521, (i3-3217U, 4GB, Intel HD Graphics 4000, Intel HM76 chipset 10/100 Mbps ethernet controller integrated on system board, WiFi 802.11 b/g/N, Bluetooth 4.0, 500 GB hd)

When the live media is created on Lubuntu Focal box with ""Startup Disk Creator" the boxes above boot successfully in both BIOS and UEFI modes.

I will now file the tests results on the QA testing tracker.

[Chris Guiver (guiverc)]

yeah sorry, I'm easily confused... (avoid brain injuries..)

[Norbert (nrbrtx)]

@guiverc, @sudodus

I need to note that we should use software which writes ISO to the flash-drive on the binary level (this means without any custom boot-loaders on top of included into ISO).

I have used GNOME Disks (`gnome-disks`, *Restore Disk Image*) and `ddrescue` (with command link `sudo ddrescue groovy-desktop-amd64.iso /dev/sdZ --force`) to write latest Ubuntu MATE ISO to the flash-drive.
Below are the current results for todays 20200714 build ( http://cdimage.ubuntu.com/ubuntu-mate/daily-live/20200714/groovy-desktop-amd64.iso , http://iso.qa.ubuntu.com/qatracker/milestones/413/builds/217010/testcases/1303/results ) :

* ThinkPad SL500, legacy only, boot and operate normally;
* VAIO F13, legacy only, boot and operate normally;
* Asustek UX32A, EFI, boot and operate normally;
* Asustek UX32A, legacy, boot and operate normally;
* desktop AsRock G41M-VS2, Core2Duo Q9400 - legacy only, boot and operate normally.

So my systems boot normally with ISOs newer than 20200709.
You can check full history for Ubuntu MATE at http://iso.qa.ubuntu.com/qatracker/milestones/413/history .

---

What is really strange here is that Rufus is a recommended way for Windows - https://ubuntu.com/tutorials/create-a-usb-stick-on-windows .

[Chris Guiver (guiverc)]

@Norbert (nrbrtx), please note comment #42

An image written by GNOME-DISKS failed (2020-07-10 daily) to boot on my dc7700 & dc7900, I just didn't add it to the detail at the top.

I re-wrote the current Lubuntu daily (2020-07-13) to thumb-drive using GNOME-DISKS *Restore-Disk-Image and tried to boot it..

and a GNOME-DISKS Restore-Disk-Image fails to boot on dc7700, or dc7900 via F9 to select thumb-drive, the brief flashing of thumb-drive already mentioned before it boots the grub on the local hdd..

The GNOME-DISKS thumb drive however booted on hp 8200.

[Chris Guiver (guiverc)]

sorry for wording in prior comment...

[Leó Kolbeinsson (leok)]

@Norbert (nrbrtx) , reference comment #55

I also found it strange that Rufus failed so explored th matter abit further.
Here is what I found in the Rufus log file.

"ISO analysis:
  Image is an ISO9660 image
  Detected EFI bootloader(s) (from '/boot/grub/efi.img'):
  ● 'bootx64.efi'
Could not get ISO-9660 file information for file boot/grub/i386-pc/normal.mod
  Could not read Grub version from 'boot/grub/i386-pc/normal.mod'
  Could not detect Grub version
Disk image analysis:
  Image has an unknown Master Boot Record
  Image is a bootable disk image
ISO label: 'Lubuntu 20.10 amd64'
  Size: 1.7 GB (Projected)
  Note: File on disk is larger than reported ISO size by 4.8 MB...
  Uses: EFI (through '/boot/grub/efi.img')
  Note: This ISO uses symbolic links, which will not be replicated due to file system limitations.
  Because of this, some features from this image may not work...
Using image: groovy-desktop-amd64.iso (1.7 GB)"

Note that the Grub version could not be read or detected ..

[Chris Guiver (guiverc)]

Lubuntu groovy amd64 daily (2020-07-17)

was written to DVDR following instructions at https://discourse.ubuntu.com/t/how-to-burn-a-dvd-on-ubuntu/14022, and it booted on both

hp dc7700 (c2d-e6320, 5gb, nvidia quadro nvs 290)
hp dc7900 (c2d-e8400, 4gb, intel 4 series integrated i915)

which have been failing b/c of this issue.

During boot on dc7700 I accidentally pressed SHIFT+INS on the wrong keyboard, which resulted in me seeing messages that i normally see before it asks for permission to download..
"line 49 can't open /dev/sr0: No medium found"
before more expected messages like "fsck:md5sums%"

Today's thumb-drive boots normally too on both boxes :)

[sudodus (nio-wiklund)]

Now one of the failure modes is removed. Congratulations to the Ubuntu developers and to guiverc :-)

I tested the current daily iso files cloned into USB drives

#perms size date time file-name
-rw------- 1,8G 2020-07-17 17:09 "lubuntu/groovy-desktop-amd64.iso"
-rw------- 2,8G 2020-07-18 08:20 "ubuntu/groovy-desktop-amd64.iso"

- and they work well (like the corresponding iso files of yesterday did) in my Dell Precision M4800

- but they fail with the same symptom as the corresponding iso files of yesterday in the Lenovo V130 with a generation 7 Intel i5 CPU. This computer is set to boot in UEFI mode with secure boot:

https://dustinweb.azureedge.net/media/494085/v130.pdf

This computer finds the USB drive in the temporary boot menu (via F12), but skips directly into the grub menu of the internal drive, when I try from a cloned USB drive.

[Leó Kolbeinsson (leok)]

Testing Lubuntu groovy amd64 daily (2020-07-18)

Tested on my 2 older machines and can confirm successful boots

Dell [Inspiron] 3521, (i3-3217U, 4GB, Intel HD Graphics 4000, Intel HM76 chipset 10/100 Mbps ethernet controller integrated on system board, WiFi 802.11 b/g/N, Bluetooth 4.0, 500 GB hd)

Acer [Aspire] E3-111-P60S (Pent.N3530, 4GB, Intel HD Graphics, Realtek RTL8111/81681/8411 GB Ethernet, Qualcomm Atheros AR9462 Wireless, Bluetooth Atheros A315-53, 500 GB hd)

Booting from BIOS and UEFI+secure boot - all successful - media created by "Startup Disk Creator" in box running Lubuntu Focal 20.04 LTS

[sudodus (nio-wiklund)]

@leok,

When cloning works (in all computers and boot modes), we can consider this bug as squashed.

When Rufus fails in its default mode, we can select 'dd-mode' which means cloning, and the result should be identical to that of the Ubuntu Startup Disk Creator and other cloning tools (Disks, mkusb (when cloning), Win32DiskImager, Balena Etcher ...).

It seems like the Lenovo V130 is a corner case, that is left behind. Cloned USB drives are still failing in that computer. But it seems that all other computers tested by us are happy to boot from cloned Groovy iso files.

[Leó Kolbeinsson (leok)]

@sudodus(nio-wiklund),

Thanks for pointing that out...

Rufus retested Lubuntu groovy amd64 daily (2020-07-18) in (dd-mode) and all booted as expected.

The machines tested were:

Dell [Inspiron] 3521, (i3-3217U, 4GB, Intel HD Graphics 4000, Intel HM76 chipset 10/100 Mbps ethernet controller integrated on system board, WiFi 802.11 b/g/N, Bluetooth 4.0, 500 GB hd)

Acer [Aspire] E3-111-P60S (Pent.N3530, 4GB, Intel HD Graphics, Realtek RTL8111/81681/8411 GB Ethernet, Qualcomm Atheros AR9462 Wireless, Bluetooth Atheros A315-53, 500 GB hd)

Looks like the bug is squashed!

[Leó Kolbeinsson (leok)]

This is no longer a problem with any of my machines and can boot with media created in Lubuntu and/or Windows.

[sudodus (nio-wiklund)]

@ Steve Langasek (vorlon),

According to your request I continue the dialogue here about problems to boot a Lenovo V130 in UEFI mode with the current daily Groovy iso files.

This problem was reported in comment #31

https://bugs.launchpad.net/ubuntu-cdimage/+bug/1886148/comments/31

- When I select the USB drive from the temporary menu, it skips to the internal drive. This is a different failure mode compared to the previous iso files. (This computer is willing to boot from the grub-n-iso system of comment #26 and from cloned drives made from groovy iso files from June.)

https://dustinweb.azureedge.net/media/494085/v130.pdf

---

Previously I have not touched the UEFI/BIOS system of the *Lenovo V130* that I have access to, but now, that you ask about it, I had better do it in order to help identify what goes wrong, when trying to boot USB drives cloned from Groovy ISO files.

When testing Groovy iso files, the fix against the boothole bug was entered into this computer, so that USB drives that used to boot would no longer boot.

I turned off secure boot, and then it was possible again to boot for example Ubuntu 20.04.1 LTS (cloned to a USB drive). This matches the observation, that Groovy iso files worked until you modified the boot system (when you removed syslinux boot in BIOS mode, but obviously also modified the boot system in UEFI mode).

But USB drives cloned from the current Ubuntu Groovy as well as Lubuntu Groovy do not boot.

This is when using the boot option in the temporary boot menu (F12)

"Linpus Lite (<name of the USB drive>)"

in the picture attached to the previous comment. This is the only available USB boot option in UEFI mode.

Then I turned off UEFI mode and set teh computer to boot in 'legacy mode'. Now there were more boot options in the temporary boot menu (F12). These options are shown in the picture attached to the previous comment. The option

"USB Hdd: <name of the USB drive>"

*works* also with USB drives cloned from the current Ubuntu Groovy as well as Lubuntu Groovy.

So the problem in this case is not due to secure boot, but the "Linpus Lite" boot option does not work with the current Groovy iso files. It works with older iso files including Ubuntu 20.04.1 LTS and Groovy iso files from before you modified the boot system (when you removed syslinux boot in BIOS mode, but obviously also modified the boot system in UEFI mode).

I attach the output of`efibootmgr -v`.

[sudodus (nio-wiklund)]

I attach the output of mokutil to this comment too, so that you need not skip between the bug reports.

[sudodus (nio-wiklund)]

I attach a photo of the temporary boot menu to this comment too, so that you need not skip between the bug reports.

I am almost 100% sure that no user has tampered with the boot settings until yesterday, when I turned off first secure boot, then UEFI. The boot menu option

"Linpus Lite (<name of the USB drive>)"

was there, when the computer arrived from the vendor (with Windows 10 pre-installed), and it works in UEFI mode with previous versions of Ubuntu, for example Ubuntu 20.04.1 LTS and daily Groovy iso files until June (this bug was detected in the beginning of July).

I switched a RAID setting of the NVMe drive to AHCI in order to be able to install Ubuntu. This is the only modification that I know of in the whole UEFI/BIOS system, and it should not affect the action of "Linpus Lite" for booting from USB.

[Leó Kolbeinsson (leok)]

@sudodus

Have you updated the BIOS on the Lenovo v130? I noticed that there was a BIOS update from Lenovo on 20 July 2020 see thesee links (not sure which model you have )

https://pcsupport.lenovo.com/us/en/products/laptops-and-netbooks/lenovo-v-series-laptops/v130-14ikb/downloads/driver-list/component?name=BIOS%2FUEFI

https://pcsupport.lenovo.com/us/en/products/laptops-and-netbooks/lenovo-v-series-laptops/v130-14igm/downloads/driver-list/component?name=BIOS%2FUEFI

It would be interesting to see what if any effect this would have on the issue at hand.

[sudodus (nio-wiklund)]

@ Leó Kolbeinsson (leok),

Thanks for this heads up :-)

It was likely that an updated version of the BIOS system could fix this issue. I did the BIOS update, and it was successful. The computer works after the update :-)

But it did not fix this bug. The temporary boot option

"Linpus Lite (<name of the USB drive>)"

is still not working with the current daily Groovy iso files :-(

-o-

Also after the upgrade (with AHCI and legacy alias BIOS boot) the option

"USB Hdd: <name of the USB drive>"

works with USB drives cloned from the current Ubuntu Groovy as well as Lubuntu Groovy.

[Steve Langasek (vorlon)]

> I attach the output of `efibootmgr -v`.

This output shows no record of the Linpus Lite boot entry that you were booting. That suggests it is an entry, synthesized by the firmware, which is never exposed in the UEFI nvram variables.

We can't debug or support a boot entry that we can't examine.

> Also after the upgrade (with AHCI and legacy alias BIOS boot) the option

> "USB Hdd: <name of the USB drive>"

>< works with USB drives cloned from the current Ubuntu Groovy as well as Lubuntu Groovy.

When you say AHCI, do you mean UEFI? AHCI has no relevance to USB boot.

Do you still have no way to directly select the USB drive for booting under UEFI after firmware upgrade, instead of using the 'Linpus Lite' boot option?

[sudodus (nio-wiklund)]

> We can't debug or support a boot entry that we can't examine.

I can understand that. I do hope that this "Linpus Lite" is a corner case. Otherwise, if this kind of USB boot systems will get common in UEFI mode, Ubuntu will have severe problems.

> When you say AHCI, do you mean UEFI? AHCI has no relevance to USB boot.

I mean AHCI and I know that it should have no relevance to USB boot. I mentioned it because it was the only thing in the UEFI/BIOS system, that was modified by a user (before this debugging started).

-o-

I have spent hours to search for and test various options in the UEFI/BIOS menu system, but no, I find no way to directly select the USB drive for booting under UEFI after firmware upgrade, instead of using the 'Linpus Lite' boot option.

Personally, I don't mind using BIOS mode alias CSM alias legacy mode. And during 5 years, I can install Ubuntu 20.04.1 LTS via this 'Linpus Lite' boot option and get a system with security updates and that way dual boot with the original Windows 10 system in UEFI mode. So I won't push this issue further. Thanks for trying to help, Steve.

[sudodus (nio-wiklund)]

I can report progress: There are new compressed image files to be extracted and cloned to USB drives for UEFI and BIOS mode, that can boot also with secure boot in a computer updated to squash the boothole bug.

You find these image files at

https://phillw.net/isos/linux-tools/uefi-n-bios/?C=M;O=D

where you also find files with the corresponding md5sums.

These compressed image files are similar to the previous ones made recently, but the boot system is made by installing Ubuntu Focal from the current daily iso files (post 20.04.1 LTS) with up to date program packages, that include secure boot shims for the relevant components.

This works also with a Lenovo V130, that refuses to boot USB boot drives cloned from current daily Groovy iso files in UEFI mode. It is a workaround in order to boot the current daily Groovy operating systems live via a grub-n-iso method.

See details at this link,

https://ubuntuforums.org/showthread.php?t=2447379&p=13985029#post13985029

[Steve Langasek (vorlon)]

Some changes have been committed now to the debian-cd branch which change the options used for the bootloader setup. Can people please retest with http://cdimage.ubuntu.com/lubuntu/daily-live/20200911/ and http://cdimage.ubuntu.com/ubuntu/daily-live/20200911/ ?

[Leó Kolbeinsson (leok)]

Booted Lubuntu image http://cdimage.ubuntu.com/lubuntu/daily-live/20200911/ successfully on 4 machines today 11.09.2020

Tested Boot modes : BIOS,UEFI, and UEFI +secure boot - no problems encountered

see test results on the QA site:
http://iso.qa.ubuntu.com/qatracker/milestones/413/builds/220285/testcases/1701/results

[sudodus (nio-wiklund)]

I can find today's groovy iso files of lubuntu and xubuntu. But there are no

groovy-desktop-amd64.iso
groovy-desktop-amd64.iso.zsync

files (only arm files) for Ubuntu Desktop. How come?

[Leó Kolbeinsson (leok)]

@sudodus
Did you use this link:

http://cdimage.ubuntu.com/lubuntu/daily-live/20200911/

[sudodus (nio-wiklund)]

I am working with these current iso files:

$ md5sum ../[lx]ubuntu/groovy-desktop-amd64.iso
5d4756194e6d917b45584a10109f8e0e ../lubuntu/groovy-desktop-amd64.iso
8ad63bfc247563dae765af648ea62b47 ../xubuntu/groovy-desktop-amd64.iso

and, Leo, I looked at the links provided by Steve.

Now a cloned USB drive with the current daily Lubuntu Groovy can boot a

Lenovo V130

(usnig Linpus Live) in UEFI mode with secure boot :-)

I notice another difference: A third partition is created (during the boot process), but no file system is recognized by lsblk -f, and it is not used for logging (as we are used to from Focal and previous versions of Groovy). Is this intended, or a bug?

-o-

But now there are problems in another computer and BIOS mode. A cloned USB drive with the current daily Lubuntu Groovy fails to boot a

Dell Latitude E7240

in BIOS mode (alias legacy mode) with the error message:

isolinux.bin missing or corrupt.
Selected boot device failed. Press any key to reboot the system.

This computer can boot in UEFI mode.

The same things happened, when I tested the cloned USB drive with the current daily Lubuntu Groovy
in a

Toshiba satellite-pro-c850-19w

'isolinux.bin missing or corrupt' in BIOS mode, and it works in UEFI mode (also with secure boot).

-o-

I tested with a cloned USB drive with the current daily Xubuntu Groovy system. And I had exactly the same boot behaviour as with Lubuntu in the three computers tested.

[Leó Kolbeinsson (leok)]

Just a thought.

It would be interesting to test these ISO´s by creating the USB on another machine than what you have been using.

[sudodus (nio-wiklund)]

@ Leo,

My experience is that cloning is cloning is cloning, whereever it is done, as long as the cloning operation is done correctly, and it is not too difficult. And the test of the files at boot indicates no error, so I am rather confident.

Anyway, I can do the cloning elsewhere, in another computer, and with another operating system and with another tool. Please suggest what I should try, and I will try it, version of Lubuntu, Windows, Rufus in dd-mode or whatever (but I have no access to a Mac computer/operating system).

[Leó Kolbeinsson (leok)]

@sudodus

How about using Lubuntu 20.04 or 20.04.1 and use the "Startup Disk Creator"?
Then try to install/boot on the Dell Latitude E7240 machine.

[Leó Kolbeinsson (leok)]

@sudodus

Sorry forgot- and the Groovy Lubuntu ISO from the the QA testing tracker..

http://iso.qa.ubuntu.com/qatracker/milestones/413/builds/220285/downloads

[sudodus (nio-wiklund)]

@ Leo,

I made an installed Lubuntu 20.04 LTS system in an HP Probook 6450b to be up to date. This computer used to be my daughter's, but it is now degraded to an occasional test computer for this kind of purpose. And I double-checked the Groovy iso file with sha256sum:

$ <<<'43eff66f35b78b933b9834135a33c560d9c1fc9b8340530d6454da0d18232d14 *groovy-desktop-amd64.iso' sha256sum -c
groovy-desktop-amd64.iso: OK

I used the Startup Disk Creator to create a USB boot drive.

When I try to boot my Dell Latitude E7240, I get the same error message as before,

isolinux.bin missing or corrupt.
Selected boot device failed. Press any key to reboot the system.

When I press the Enter key, the computer is rebooted, so no luck :-(

[sudodus (nio-wiklund)]

... The problem occurs only in BIOS mode. As before, it boots in UEFI mode, where I get

"Check finished. No errors found"

and I reach to the desktop as expected.

[Leó Kolbeinsson (leok)]

...I just made an attempt on my Dell Insirion 3521 and got the same failure when booting in BIOS mode.
UEFI and secure boot work fine,Will test another machine and report back.

isolinux.bnin missing or corrupt.

[Leó Kolbeinsson (leok)]

Acer Aspire E3-111 also fails to boot in BIOS mode - same error isolinux.bin missing or corrupt.

Think this confirms the error ..

[Chris Guiver (guiverc)]

Today's Lubuntu daily (2020-09-11) failed to boot on

hp dc7700 (c2d-e6320, 5gb, nvidia quadro nvs 290)
hp dc7900 (c2d-e8400, 4gb, intel 4 series integrated i915)
dell [optiplex] 755 (c2d-e6850, 5gb, amd/ati radeon rv516/x1300/x1550)
dell [optiplex] 755 (c2d-e8300, 8gb, amd/ati radeon rv610/radeon hd2400 pro/xt)
dell [optiplex] 780 (c2q-q9400, 8gb, amd/ati cedar radeon hd 5000/6000/7350/8350)

error was "isolinux.bin missing or corrupt" for the above, it also failed on

dell [optiplex] 990 (i7-2600, 16gb, nvidia geforce gt 6600 gt)

which reported the selected media was not bootable (it likely had the same message, but was quickly erased with a bios message replacing it)

I filed those under a new report https://bugs.launchpad.net/bugs/1886148 now marked duplicate of this.

[sudodus (nio-wiklund)]

I tested also with plain extraction from the current Lubuntu Groovy iso file to a FAT32 partition with what I think are the correct flags

$ LANG=C sudo parted /dev/sdc p
Model: OCZ-AGIL ITY3 (scsi)
Disk /dev/sdc: 60,0GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:

Number Start End Size Type File system Flags
 1 1049kB 60,0GB 60,0GB primary fat32 boot, esp

I tried to boot in UEFI mode, but it did not work :-(

The same method works with Lubuntu 20.04.1 LTS (I tested right now just to be sure).

-o-

I loop mounted the iso file and used rsync to copy to the pendrive's partition mounted at /mnt/sd1:

sudo rsync -r --info=progress2 /mnt/lp1/ /mnt/sd1

Then I flushed the buffers and unmounted /mnt/lp1/ and /mnt/sd1.

[sudodus (nio-wiklund)]

... minor correction: I used a small SSD connected via a USB to SATA adapter instead of a pendrive because it is so much faster.

[Steve Langasek (vorlon)]

On Fri, Sep 11, 2020 at 09:10:33AM -0000, sudodus wrote:
> I notice another difference: A third partition is created (during the
> boot process), but no file system is recognized by lsblk -f, and it is
> not used for logging (as we are used to from Focal and previous versions
> of Groovy). Is this intended, or a bug?

If a partition is really being created during the boot process, but it is
not usable, that is unexpected. Please file a separate bug report against
casper for this.

[sudodus (nio-wiklund)]

@ Steve Langasek (vorlon),

See Bug #1895329:

"when booting cloned live drive 3rd partition is created without file system"

[Steve Langasek (vorlon)]

On Fri, Sep 11, 2020 at 08:21:03AM -0000, sudodus wrote:
> I can find today's groovy iso files of lubuntu and xubuntu. But there
> are no

> groovy-desktop-amd64.iso
> groovy-desktop-amd64.iso.zsync

> files (only arm files) for Ubuntu Desktop. How come?

Sorry, I failed to notice that the amd64 build was unsuccessful - related to
the recent changes. There's an amd64 build now at
http://cdimage.ubuntu.com/ubuntu/daily-live/20200911.2, though it almost
certainly fails in the same way as the Lubuntu one.

[Chris Guiver (guiverc)]

The 20200911.2 won't boot on

- dell [optiplex] 755 (c2d-e6850, 5gb, amd/ati radeon rv516/x1300/x1550)

It feels different though, the "isolinux.bin missing or corrupt" was very quick to appear & get erased by grub being booted off hdd on the d755-5 I listed. I had to boot it 3 times to read the "isolinux.bin missing" message, where I can't recall doing that yesterday.

- hp dc7700 (c2d-e6320, 5gb, nvidia quadro nvs 290)

On the hp dc7700 I never saw any message, it just booted hdd grub, even on selecting thumb-drive as boot device. Yes I could see the thumb-drive flash as it does during reads (flashing rather rapidly) which soon stopped after grub appeared, but no messages on screen - 5 boots attempted & no message.

With yesterday's ISO it was easy to read the "isolinux.bin missing' message & I sure didn't boot this many times to confirm it was booting.

- dell [optiplex] 780 (c2q-q9400, 8gb, amd/ati cedar radeon hd 5000/6000/7350/8350)
I saw a message about the same shape/size as "isolinux.bin missing or corrupt" but it was unreadable in two boots as replaced by hdd's grub.

The failures "felt" quicker than yesterdays.

[Chris Guiver (guiverc)]

Xubuntu groovy daily (2020-09-09) will boot on

hp dc7700 (c2d-e6320, 5gb, nvidia quadro nvs 290)

Xubuntu groovy daily (2020-09-12) however fails

No message seen though just like Lubuntu daily (20200911.1)

Also tried booting (2020-09-12) on
dell [optiplex] 755 (c2d-e6850, 5gb, amd/ati radeon rv516/x1300/x1550) where I could read the "isolinux.bin missing or corrupt" error briefly before hdd's grub menu appeared.

[Chris Guiver (guiverc)]

** Xubuntu & Ubuntu 2020-09-09 ISOs boot normally

Sorry I meant to say in the last comment

I had a Xubuntu 2020-09-09 ISO & Ubuntu 2020-09-09 I downloaded but never (QA-)tested. They do boot in the hp dc7700 box normally. (the Ubuntu was also booted on d755-5)

(ubuntu_amd64)
-rw-rw-r-- 1 guiverc blah 2.8G Sep 9 18:32 /de2900/ubuntu_64/groovy-desktop-amd64.iso
(xubuntu_amd64)
-rw-rw-r-- 1 guiverc blah 1.7G Sep 9 12:08 groovy-desktop-amd64.iso.zs-old

[sudodus (nio-wiklund)]

I must say, that I am surprised :-)

mkusb version 12.6.1 can create persistent live drives, that boot both in BIOS mode and UEFI mode, also with secure boot from the current daily Xubuntu Groovy iso file. I used the setting 'usb-pack-efi'.

$ ls -l xubuntu/groovy-desktop-amd64.iso
-rw------- 1 sudodus sudodus 1803091968 sep 12 02:07 xubuntu/groovy-desktop-amd64.iso

See the attached screenshot.

-o-

When cloned to a USB drive, this file does not boot in BIOS mode.

[Chris Guiver (guiverc)]

Ubuntu groovy desktop amd64 daily of 20200912 fails to boot when written with

sudo dd bs=4M oflag=sync status=progress of=/dev/sdb if=/de2900/ubuntu_64/groovy-desktop-amd64.iso

on

hp dc7700 (c2d-e6320, 5gb, nvidia quadro nvs 290)
dell [optiplex] 755 (c2d-e8300, 8gb, amd/ati radeon rv610/radeon hd2400 pro/xt)
dell [optiplex] 780 (c2q-q9400, 8gb, amd/ati cedar radeon hd 5000/6000/7350/8350)
lenovo thinkpad sl510 (c2d-t6570, 2gb ram, i915)
lenovo thinkpad x201 (i5-m520, 4gb, i915)

but will boot on UEFI enabled boxes (messages differed on various boxes, some I saw none, thinkpad sl510 was "Operating System not found". I didn't test other boxes

[Leó Kolbeinsson (leok)]

Lubuntu Groovy desktop amd64 daily of 20200912 fails to boot with "isolinux.bin missing or corrupt"
error when booting in BIOS mode.

Tested one box (Acer [Aspire] E3-111-P60S) with media created with "Startup Disk Creator",Rufus 3.11 in Windows 10 and the command line "sudo dd bs=4M oflag=sync status=progress of=/dev/sda if=groovy-desktop-amd64.iso"

Will await next daily ISO for further testing.

[Leó Kolbeinsson (leok)]

Lubuntu Groovy desktop amd64 daily of 20200913 fails to boot with "isolinux.bin missing or corrupt"
error when booting in BIOS mode.

Tested 3 laptops for BIOS boot - all failed but booted normally in UEFI and UEFI+secure boot modes. USB media created with "Startup Disk Creator and/or sudo dd bs=4M oflag=sync status=progress of=/dev/sda if=groovy-desktop-amd64.iso

The machines used :
Dell [Latitude] 7280
Dell [Inspiron] 3521
Acer [Aspire] E3-111-P60S

Also tested BIOS boot in VirtualBox on these 2 machines - both successful

Dell [Optiplex] 7010
InWin BL641 i5-10400

So far I have seen no failures when booting BIOS mode in VirtualBox.

[Harry Coin (hcoin)]

S5000PSL - usb creator fails to boot using groovy daily build 9/12, 9/13. Works normally with unetbootin. Daily build did boot perhaps a week or 10 days ago. (Installer failed & crashed when 'something else/ grub bios boot only .. no efi .. but that's another issue. 'Something else' did work when I manually created both a efi and legacy bios partition, even though the efi partition was a waste of space. (gpt partition format).

[Steve Langasek (vorlon)]

Thanks for your forebearance. Can you please test http://cdimage.ubuntu.com/ubuntu/daily-live/20200915.3/ ? I have boot tested this under KVM with both BIOS and UEFI, and both as a CD-ROM and a hard drive, and it has booted successfully for me here.

If there are any problems with this image, I think they are most likely to happen with older or esoteric UEFI implementations; so some targeted testing there would be helpful. (I have no reason to expect this image fixes the Linpus Lite boot issue.)

[Leó Kolbeinsson (leok)]

@vorlon

Just tested 2 barbone machines that boot successfully in BIOS mode.
However when attempting to boot the media in UEFI+secure boot mode the media is not even detected. Hence failure.

The machines tested were:

Dell [Inspiron] 3521
Acer [Aspire] E3-111-P60S

Also please note that on the previous ISO's tested that all passed in VirtualBox BIOS mode.
The failures were all on barebone machines.

Hope this helps.

[Leó Kolbeinsson (leok)]

Further to last post:

Just tested http://cdimage.ubuntu.com/daily-live/20200915.4/groovy-desktop-amd64.iso with the same results. BIOS boot successful and UEFI + secure boot - USB media not found.

[sudodus (nio-wiklund)]

I can confirm that the current daily iso files make cloned USB drives that

- boot in BIOS mode
- fail to boot UEFI mode (not even recognized as a bootable drive).

I tested the current daily Ubuntu iso file dated Sep 15 08:54

$ <<<'920c90aa90b81c48e6ef57c1579dcad97a168fc3e460b3cbdb6a096a0daadbbf *groovy-desktop-amd64.iso' sha256sum -c
groovy-desktop-amd64.iso: OK

in BIOS mode and UEFI mode without secure boot

- in a Dell Precision M4800 with a 4th generation Intel Core i7-4810MQ CPU
- in a Dell Optiplex 7050 with a 6th generation Intel Core i7-6700 CPU

Need I say, that it fails to boot the Lenovo V130 in secure boot? Linpus Lite is not even there in the menu.

-o-

I think that you should make Dells with generation 4+ Intel i7 (like my Precision M4800 and the Optiplex 7050) boot also in UEFI mode. It is way too early to abandon such computers. We must realize that some people want UEFI and even secure boot, I think particularly those who dual boot with Windows.

[Steve Langasek (vorlon)]

Thank you again for all of your testing. I've made some further changes to the xorriso options which should bring the image more in line with standards, following xorriso upstream guidance and I have again boot tested on {BIOS,UEFI} {cdrom,hd} under KVM. Please give http://cdimage.ubuntu.com/ubuntu/daily-live/20200915.5/ a whirl and see if it works any better for you. The changes made are specific to the UEFI boot options, so BIOS boot testing is not required.

> I think that you should make Dells with generation 4+ Intel i7 (like my
> Precision M4800 and the Optiplex 7050) boot also in UEFI mode. It is way
> too early to abandon such computers. We must realize that some people
> want UEFI and even secure boot, I think particularly those who dual boot
> with Windows.

It has always been the intent to get UEFI boot working again on these systems, it was just non-obvious how to get this working while moving BIOS boot to GRUB since the previous hybrid mode depended on isolinux-specific options to xorriso.

The partition table on the current image looks like what we are looking for and should generally fare better with diverse UEFI implementations.

[Leó Kolbeinsson (leok)]

Finally some good news.

Tested 2 machines http://cdimage.ubuntu.com/ubuntu/daily-live/20200915.5/
Both machines booted in BIOS mode and UEFI+secure boot mode with no errors.

The machines tested were:

Dell [Inspiron] 3521
Acer [Aspire] E3-111-P60S

[sudodus (nio-wiklund)]

I started testing with my Dell Precision M4800 with a 4th generation Intel Core i7-4810MQ CPU, and its works

- to clone the current Ubuntu Groovy iso file to a USB drive and make the Dell Precision M4800 boot both in BIOS mode and UEFI mode. See the attached file :-)

- But there is no file system in the third partition (Bug #1895329).

I will continue the testing with other computers ...

[sudodus (nio-wiklund)]

- A Lenovo V130 finds the Linpus Lite boot option, but when selected it skips directly to the internal drive's grub menu (with dual boot Ubuntu and Windows). Back to square one.

- A Toshiba Satellite Pro C850 19w boots both in UEFI mode with *secure boot* and in BIOS mode.

- An old Lenovo X131e with Intel i3 boots both in UEFI mode and in BIOS mode.

- An HP Probook 6450b with Intel i5 boots both in UEFI mode and in BIOS mode.

Summary: There is a big improvement. The various computers tested so far with one exception, can boot both in BIOS mode and UEFI mode (also with secure boot, when that was tested).

[sudodus (nio-wiklund)]

mkusb version 12.6.1 (mkusb-dus) can create persistent live drives, that boot both in BIOS mode and UEFI mode, also with secure boot from the current daily Ubuntu Groovy iso file.

I used the setting 'usb-pack-efi'. This works also with a Lenovo V130 in UEFI with secure boot, so there is a work-around for the problematic computer. The setting 'usb-pack-efi' uses grub from previous LTS releases to create the boot systems for BIOS mode and UEFI mode.

[Leó Kolbeinsson (leok)]

Tested an additional Dell box - this time Dell 7010 i5-3470. http://cdimage.ubuntu.com/daily-live/20200916/groovy-desktop-amd64.iso

Booted successfully BIOS,EFI,EFI +secure boot.

[Chris Guiver (guiverc)]

I downloaded(zsync'd) the [vorlon] requested ISO, but never got time to test it sorry.

I've zsync'd todays (2020-09-16) ISO and it boots on

hp dc7700 (c2d-e6320, 5gb, nvidia quadro nvs 290)
dell [optiplex] 755 (c2d-e6850, 5gb, amd/ati radeon rv516/x1300/x1550)
dell [optiplex] 755 (c2d-e8300, 8gb, amd/ati radeon rv610/radeon hd2400 pro/xt)
dell [optiplex] 780 (c2q-q9400, 8gb, amd/ati cedar radeon hd 5000/6000/7350/8350)
dell [optiplex] 990 (i7-2600, 16gb, nvidia geforce gt 6600 gt)

so I suspect it'll boot fine on other boxes too :)
(written with my usual `dus` (mkusb))

(if I test more boxes later, I'll not mention here unless problems, only logging as QA-tests on iso.qa.ubu)

[Norbert (nrbrtx)]

@all

Could you please systemize all your findings on public Google Spreadsheet?

Reading whole thread everytime is very difficult to track progress or
regress.
I would recommend the following workflow: test ISO, add result to created
table, add concluding comment here.

[Chris Guiver (guiverc)]

I tested Xubuntu groovy daily (2020-09-17) on

hp dc7700 (c2d-e6320, 5gb, nvidia quadro nvs 290)
dell [optiplex] 755 (c2d-e6850, 5gb, amd/ati radeon rv516/x1300/x1550)
lenovo thinkpad x201 (i5-m520, 4gb, i915)
and it booted fine.

[Chris Guiver (guiverc)]

@Steve/vorlon

I've on rare occasion booted the L/X/K/ubuntu amd64 in a i386 box as a quick test and expect a message like

"Kernel requires an x86-64 CPU .. i686 detected"

I booted the current Lubuntu groovy daily on
dell latitude d610 (pentium m, 1.5gb, intel i915)
and the grub menu appeared, but then it's just blinking cursor.

Do you care? or does this matter? want bug report or ignore?
(I suspect that would cause slower boots on amd64? and thus would be won't fix..) ; also mentioned on #ubuntu-devel

[C.S.Cameron (cscameron)]

Booting ISO Files using GRUB 2

I believe another concern of this Bug is the booting of Groovy ISO files directly using GRUB 2.

Programs that use this method include Ventoy. YUMI and MultiBootUSB.

I mostly use a BIOS-UEFI Template image developed by sudodus, see: https://askubuntu.com/questions/1269462/bios-uefi-template-image-for-booting-iso-files

BIOS mode booting of ISO files now appears to work okay with the latest Groovy Daily ISO.

The only way I have been able to get UEFI mode booting to work is by using "nomodeset" which results in a distorted desktop.

This also works for Ventoy, however YUMI BIOS/GRUB and MultiBootUSB are not yet working with 20.10.

[sudodus (nio-wiklund)]

@ C.S.Cameron,

What happens if you try with the same computer, iso file, and USB drive as in comment #115 to make the USB drive persistent live with mkusb-dus using

1. default settings,
2. usb-pack-efi,

and if you

3. *clone* from the iso file to the USB drive?

-o-

@ Steve Langasek (vorlon),

Should we consider the iso-booting problem another symptom of the same bug, or should we create a separate bug report?

[C.S.Cameron (cscameron)]

@sudodus:

I made a persistent 20.10 USB using 12.6.3 unstable.

Persistent boot in BIOS mode worked okay with a distorted desktop.

Plain Persistent boot ended in initramfs unpacking error.

Safe Mode boot menu option worked with resume option.

Persistent boot edited adding nomodeset to the linux line eventually worked okay with a distorted desktop. Boot process took over five minutes.

I will try a clone using mkusb, My guess is that nomodeset will work okay here also.

[C.S.Cameron (cscameron)]

Everything "Plain Persistent boot ended in initramfs unpacking error". and after were in UEFI mode.

[C.S.Cameron (cscameron)]

Creating a Live Only USB using 20.10 20200924 and 12.9.3 ave similar results to the Persistent install.

Worked fine in BIOS mode.

Gave initramfs unpacking error in UEFI mode plain persistent boot.

Gave initramfs unpacking error in UEFI mode persistent boot with nomodeset edit, but quickly continued booting to desktop.

Safe Graphic Mode boot also had initramfs unpacking error in UEFI mode but continued on to Language screen and beyond.

This is similar to my experience trying to boot ISO files directly, but I only recall the unpacking error that continued to boot with the Live only USB.

[sudodus (nio-wiklund)]

@ C.S.Cameron,

It is strange that it works fine in BIOS mode but needs nomodeset in UEFI mode. Something must fail to get activated in UEFI mode.

There are obvious problems with Ubuntu Groovy in this computer. Please specify the details about it: Brand name and model of the computer itself as well as of the graphics card/chip.

[C.S.Cameron (cscameron)]

@Sudodus
Hardinfo follows
Computer : Gigabyte GB-BXi7-5775R
Processor : Intel(R) Core(TM) i7-5775R CPU @ 3.30GHz
Memory : 7959MB (1088MB used)
Machine Type : Desktop
Operating System : Ubuntu 20.04 LTS
User Name : cscameron (cscameron)
Date/Time : Sat 26 Sep 2020 11:20:29 AM
-Display-
Resolution : 1920x1080 pixels
OpenGL Renderer : Mesa Intel(R) Iris(R) Pro Graphics 6200 (BDW GT3)
X11 Vendor : The X.Org Foundation
-Audio Devices-
Audio Adapter : HDA-Intel - HDA Intel HDMI
Audio Adapter : HDA-Intel - HDA Intel PCH
-Input Devices-
 Sleep Button
 Power Button
 Power Button
 Microsoft Microsoft® 2.4GHz Transceiver v8.0
 Microsoft Microsoft® 2.4GHz Transceiver v8.0 Mouse
 Microsoft Microsoft® 2.4GHz Transceiver v8.0 Consumer Control
 Microsoft Microsoft® 2.4GHz Transceiver v8.0 Consumer Control
 Microsoft Microsoft® 2.4GHz Transceiver v8.0 System Control
 SIGMACHIP USB Keyboard
 SIGMACHIP USB Keyboard Consumer Control
 SIGMACHIP USB Keyboard System Control
 Video Bus
 HDA Intel HDMI HDMI/DP,pcm:3
 HDA Intel HDMI HDMI/DP,pcm:7
 HDA Intel HDMI HDMI/DP,pcm:8
 HDA Intel HDMI HDMI/DP,pcm:9
 HDA Intel HDMI HDMI/DP,pcm:10
 HDA Intel PCH Front Headphone
-Printers-
No printers found
-SCSI Disks-
ATA TOSHIBA MQ01ABD1
ATA RD-S350MCN-N1283
SanDisk Ultra Fit

[sudodus (nio-wiklund)]

@ C.S.Cameron,

Thanks for the details about the computer.

Please add the output of the following command to get info about the graphics chip/card:

sudo lshw -C display

[Norbert (nrbrtx)]

@all

Could you please systemize all your findings on public Google Spreadsheet?

Reading whole thread everytime is very difficult to track progress or
regress.
I would recommend the following workflow: test ISO, add result to created
table, add concluding comment here.

[sudodus (nio-wiklund)]

@ Norbert (nrbrtx),

You are welcome to systemize all our findings on public Google Spreadsheet :-)

[Oliver Grawert (ogra)]

this exists already at http://iso.qa.ubuntu.com/qatracker/%20

[Norbert (nrbrtx)]

@ogra, @sudodus

I do not have time to read whole thread 125+ comments.

Maybe I want to help with testing, but in current 125-comment view I can't determine which to test on which hardware, which BIOS/UEFI, which CPU, which motherboard, which vendor, etc.

As all of you are doing this long time, please create spreadsheet somewhere for normal comprehensive reuse and to easy the analysis. Maybe I'll join this party.

[C.S.Cameron (cscameron)]

@sudodus

  *-display
       description: VGA compatible controller
       product: Iris Pro Graphics 6200
       vendor: Intel Corporation
       physical id: 2
       bus info: pci@0000:00:02.0
       version: 0a
       width: 64 bits
       clock: 33MHz
       capabilities: msi pm vga_controller bus_master cap_list rom
       configuration: driver=i915 latency=0
       resources: irq:33 memory:f6000000-f6ffffff memory:e0000000-efffffff ioport:f000(size=64) memory:c0000-dffff

[sudodus (nio-wiklund)]

Thanks for the details about the computer, where Ubuntu Groovy fails in UEFI mode, @C.S.Cameron.

Now I think there is information for the developers as well as for users with similar hardware as yours.

See comments # 115,117-119,121,127.

Summary:

Computer : Gigabyte GB-BXi7-5775R
Processor : Intel(R) Core(TM) i7-5775R CPU @ 3.30GHz

Memory : 7959MB (1088MB used)

Resolution : 1920x1080 pixels
OpenGL Renderer : Mesa Intel(R) Iris(R) Pro Graphics 6200 (BDW GT3)

  *-display
       description: VGA compatible controller
       product: Iris Pro Graphics 6200
       vendor: Intel Corporation
       configuration: driver=i915 latency=0 (used by Ubuntu 20.04 LTS)

[sudodus (nio-wiklund)]

@ Norbert (nrbrtx),

When you have time and energy, please test

- a current daily iso file of a developing version of the flavour that you like

- *cloned* to a USB drive

- in both BIOS and UEFI mode

- in the computers where you want future versions of Ubuntu flavours to work, and that are available for you to test

and share the result here and at the iso testing tracker particularly if you find a computer and test case, where it should work, but does not.

[Leó Kolbeinsson (leok)]

Just tested http://cdimage.ubuntu.com/lubuntu/daily-live/20200927/groovy-desktop-amd64.iso

Lenovo V14 ILL,Intel Core i3-1005G!,8GB,256GB SSD
Testcase:UEFI+secure boot - Live session
Unable to boot Groovy in UEFI mode (this machine brand new right out of the box) no errors encountered just failed to boot
The intenton was to wipe Windows 10 and install Lubuntu
Booting in BIOS (legacy mode) no problem - and installed system - am awaiting the next daily iso (28.09.2020) for further testing on this issue

[Leó Kolbeinsson (leok)]

tested the daily isoś for Xubuntu and Kubuntu 20200928 and also received failures to boot
in UEFI mode on the Lenovo V14 IIL,Intel Core i3-1005G!,8GB,256GB SSD box.

I then tried Lubuntu,Kubuntu and Xubuntu using releases 20.04.1 and had no problems booting in UEFI+secure boot.

[Chris Guiver (guiverc)]

@ Leó Kolbeinsson/leok & @sudodus/nio-wiklund

Boxes still impacted as I understand it are,

owned by sudodus/nio-wiklund
* Lenovo V130

owned by Leó Kolbeinsson
* Lenovo V14 IIL,Intel Core i3-1005G!,8GB,256GB SSD

is there a work-around?, or can one be found should the release occur without this issue being fixed for impacted boxes. With luck this won't be needed, but any work around we can document maybe useful as a fallback. Thanks.

If I'm missing anything, apologies & please correct me (if there are other boxes, edit & add to top description or please let me know).

[sudodus (nio-wiklund)]

@ Chris Guiver (guiverc),

Yes, there is a workaround.

The Lenovo V130 can boot with the current daily Lubuntu Groovy iso file also in UEFI mode, when making a persistent live drive by mkusb-dus and selecting 'usb-pack-efi'.

See also the following link,

https://help.ubuntu.com/community/mkusb/persistent#Installing_-_using_the_Install_icon_on_the_desktop

-o-

You could also ask C.S.Cameron about his problems with a Gigabyte computer. (You can ping him with a message at the Ubuntu Forums, where his user ID is the same as here.)

[Leó Kolbeinsson (leok)]

@ Chris Guiver (guiverc) @ sudodus (nio-wiklund)

Will test Lenovo V14 with persistent live drive and report back asap.
 this is the only box i have that fails to boot UEFI.

[Leó Kolbeinsson (leok)]

@ Chris Guiver (guiverc) @ sudodus (nio-wiklund)

Retested the Lenovo V14 with the workaround with mkusb-dus worked very well and I can now confirm all my boxes now booting sucessfully in UEFI modes with and without secure boot enabled.

[RussianNeuroMancer (russianneuromancer)]

I tested groovy-desktop-arm64.iso (pay attention: arm64) on Lenovo Yoga C630 WOS and it does not boot.

1. When I write iso by using usb-creator-gtk then laptop firmware does not detect flash drive as bootable device and just skip to GRUB2 from UFS.

2, When I write iso manually, by coping content of both partitions from iso to FAT32 partition of flash drive I getting following behavior:

    Flash drive led blink
    Black screen
    Noting happens, it’s just sits here

It’s seems like GRUB2 from iso get started but then it freeze for some reason.

I guess this bugreport is about first behavior rather about the second?

[sudodus (nio-wiklund)]

@ RussianNeuroMancer,

> I guess this bugreport is about first behavior rather about the second?

- Yes, because cloning (e.g. with usb-creator-gtk) is what the developers say should work.

- On the other hand, no, several users (including me) would like the new boot configuration to work (in UEFI mode) also when extracted to a FAT32 partition.

-o-

So far we have been looking at 'IBM compatible PCs' with Intel and AMD processors, but you are welcome with an ARM computer too. I think the bug is the same.

[Launchpad Janitor (janitor)]

This bug was fixed in the package cd-boot-images-amd64 - 7

---------------
cd-boot-images-amd64 (7) groovy; urgency=medium

  * Move images that are input to xorriso, but that don't need to be part of
    the iso9660 tree, into a separate directory.
  * lintian overrides
  * Don't delete tree/efi/boot after copying into the el-torito image: we
    want this included in the iso9660 filesystem for compatibility.
    LP: #1886148.

 -- Steve Langasek <email address hidden> Wed, 30 Sep 2020 09:03:39 -0700

[Steve Langasek (vorlon)]

These changes have unfortunately not landed in time for the 20.10 beta, but will be included in the next daily builds following beta. Once the next daily is available, please re-test those systems that were failing to boot under UEFI - the new cd-boot-images-amd64 now provides the \EFI\BOOT tree on the iso9660 filesystem for compatibility.

[Steve Langasek (vorlon)]

http://cdimage.ubuntu.com/ubuntu/daily-live/20201002/ is now available and includes the EFI/BOOT directory within the iso9660 filesystem. Please test this image on the remaining UEFI systems that were failing to boot the image.

[sudodus (nio-wiklund)]

Lubuntu
-------
I started testing the current daily Lubuntu Groovy iso file (faster to zsync) with the Lenovo V130.

1. When cloned to a USB drive, it does not boot in UEFI mode (the same problem as before).

2. When extracted to a FAT32 partition with boot and esp flags in a USB drive, it boots in UEFI mode :-)

Ubuntu
------
Then I tested the current Ubuntu Groovy iso file, and when extracted to a FAT32 partition with boot and esp flags in a USB drive, the Lenovo V130 boots in UEFI mode :-)

When cloned, it behaved like before: the Lenovo V130 does not boot in UEFI mode, but my other computers (tested in a Dell Precision M4800) are happy to boot (but fail to create an ext file system in partition #3, bug #1886129).

[sudodus (nio-wiklund)]

Wrong reference to the other bug. It should be bug #1895329

http://launchpad.net/bugs/1895329

"when booting cloned live drive 3rd partition is created without file system"

[Marcos Nascimento (wstlmn)]

Similar to the #141, I was able to start normally in efi, creating partitions and copying the files. On the desktop starts normally, so it seems that the issue is restricted to Lenovo notebooks, in my case the ideapad 330-15IKB.

[Steve Langasek (vorlon)]

On Fri, Oct 02, 2020 at 06:56:45PM -0000, sudodus wrote:
> When cloned, it behaved like before: the Lenovo V130 does not boot in
> UEFI mode, but my other computers (tested in a Dell Precision M4800) are
> happy to boot

And is this a regression vs 20.04? I've lost track.

[sudodus (nio-wiklund)]

@ Steve Langasek (vorlon),

Yes, it is a regression vs 20.04. The early Groovy iso files worked too, until you made the big modification of the boot structure near the beginning of July.

[RussianNeuroMancer (russianneuromancer)]

> the new cd-boot-images-amd64 now provides the \EFI\BOOT tree on the iso9660 filesystem for compatibility.

Is there chance to get same fix for arm64 server and desktop images too?
Snapdragon-based laptops with UEFI and also UEFI implementations for SBC (such as https://github.com/pftf/RPi3/releases ) is also affected by this issue.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Marcos Nascimento (wstlmn)]

In previous version images the /isolinux folder existed. Isn't it the case that it's necessary for systems that are experiencing problems?

[Steve Langasek (vorlon)]

On Sun, Oct 04, 2020 at 01:14:43PM -0000, Marcos Nascimento wrote:
> In previous version images the /isolinux folder existed. Isn't it the
> case that it's necessary for systems that are experiencing problems?

No. The isolinux directory is an internal implementation detail of using
the isolinux bootloader, which was only ever used for BIOS booting. We are
not using the isolinux bootloader anymore, and the remaining boot failures
reported are not with BIOS.

[Launchpad Janitor (janitor)]

This bug was fixed in the package cd-boot-images-arm64 - 7

---------------
cd-boot-images-arm64 (7) groovy; urgency=medium

  [ Steve Langasek ]
  * Move images that are input to xorriso, but that don't need to be part of
    the iso9660 tree, into a separate directory.
  * lintian overrides
  * Don't delete tree/efi/boot after copying into the el-torito image: we
    want this included in the iso9660 filesystem for compatibility.
    LP: #1886148.
  * Remove images/ directory on clean so rebuilds work.

  [ Dimitri John Ledkov ]
  * Rebuild against grub2-signed 1.155+2.04-1ubuntu35.

 -- Dimitri John Ledkov <email address hidden> Mon, 05 Oct 2020 11:03:50 +0100

[RussianNeuroMancer (russianneuromancer)]

@xnox, thank you!

Now groovy-desktop-arm64.iso is recognized as bootable by Lenovo Yoga C630 WOS (I will check RPi3B and r3399 uefi implementations later, but that most likely work too).

There is still black screen issue I described earlier: https://discourse.ubuntu.com/t/groovy-to-use-grub2-for-booting-installer-media-in-any-modes-on-all-architectures/16871/29
So I filled separate report about it: Bug 1898823

[Steve Langasek (vorlon)]

I've built http://cdimage.ubuntu.com/ubuntu/daily-live/20201007.1/ now which drops the -partition_offset 16 option to xorriso, which more closely approximates the layout we had in 20.04 (at the expense of partition table correctness). Please check whether this image now boots on the Lenovo V series, and if so, please also check that it works on other systems (in BIOS and EFI modes).

[Leó Kolbeinsson (leok)]

@vorlon just tested the 20201007.1 image on my Lenovo V14 IIL box failed to boot both
BIOS and UEFI modes . Will test on other machines.

[Thomas Schmitt (scdbackup)]

Hi,

so for now -partition_offset 16 is not to blame ?

The only tangible report was back in 2011 about a Macbook Pro
  https://lists.debian.org/debian-cd/2011/04/msg00029.html
with decisive test in
  https://lists.debian.org/debian-cd/2011/04/msg00060.html
and (deplorable) success report in
  https://lists.debian.org/debian-cd/2011/04/msg00061.html

May i propose my personal favorite culprit for the next test here ?

If so, then try what happens if at ISO production time the options

  -part_like_isohybrid -isohybrid-gpt-basdat

are omitted in order to drop the invalid GPT of the ISO.
Lengthy motivation is in:
  https://bugs.launchpad.net/ubuntu/+source/casper/+bug/1895131/comments/22

Have a nice day :)

Thomas

[sudodus (nio-wiklund)]

I tested the current daily Lubuntu Groovy iso file with the Lenovo V130.

When cloned to a USB drive, it does not boot in UEFI mode (the same problem as before) :-(

When extracted to a FAT32 partition with boot and esp flags in a USB drive, it boots in UEFI mode :-)

[sudodus (nio-wiklund)]

The same current daily Lubuntu Groovy iso file, when cloned, boots as it should in a Dell Precision M4800.

But cloned drives are still affected by bug #1895329.

[Leó Kolbeinsson (leok)]

Further tests on the currecnt Ubuntu daily 20201007.1

ACER Aspire E3 boots BIOS,UEFI;UEFI +secure boot without errors
Dell Inspirion 3521 - BIOS - success - UEFI and UEFI+secure boot - fail
Lenovo X230i - boots successfully in all boot modes

Boot media created with "Startup Disk Creator" in Lubuntu Focal

[Leó Kolbeinsson (leok)]

One further test on the V14 ILL box ..this time UEFI+secure boot with media created with a persistent live drive by mkusb-dus and selecting 'usb-pack-efi'.

This time the system booted sucessfully.

[sudodus (nio-wiklund)]

I tested the current daily Ubuntu Desktop Groovy iso file with the Lenovo V130. The results match those of comments #155-156.

After that I made persistent live drives with mkusb (using usb-pack-efi), and they worked as before. So I can confirm the results by Leó Kolbeinsson (leok).

[sudodus (nio-wiklund)]

@ Steve Langasek (vorlon),

Would you be able to ask the manufacturer, either Lenovo or InsydeH2O, what they are looking for in their "Linpus Lite" boot option (in order to boot in UEFI mode)?

In my computer, on the 'title line' of the BIOS information page is written:

InsydeH2O Setup Utility Rev. 5.0

Then I can see the following detailed information:

Product name: Lenovo V130-14IKB
BIOS version: 8YCN31WW(V1.33)
EC version: 8YEC31WW(V1.33)
MTH: 81HQ00KUMX
Lenovo SN: MP1NKHEY

[Steve Langasek (vorlon)]

Hi Thomas,

On Thu, Oct 08, 2020 at 06:56:47AM -0000, Thomas Schmitt wrote:
> so for now -partition_offset 16 is not to blame ?

Appears not.

> The only tangible report was back in 2011 about a Macbook Pro
> https://lists.debian.org/debian-cd/2011/04/msg00029.html
> with decisive test in
> https://lists.debian.org/debian-cd/2011/04/msg00060.html
> and (deplorable) success report in
> https://lists.debian.org/debian-cd/2011/04/msg00061.html

> May i propose my personal favorite culprit for the next test here ?

> If so, then try what happens if at ISO production time the options

> -part_like_isohybrid -isohybrid-gpt-basdat

> are omitted in order to drop the invalid GPT of the ISO.

Yes, thanks, this was the next thing on the list for us to test. http://cdimage.ubuntu.com/ubuntu/daily-live/20201008/
is now built, with these changes (restore -partition_offset, drop
isohybrid/gpt-basdat options).

@sudodus, @leok, could you please retest on Lenovo V-series with this image?
Reports on other hardware are only relevant if it does boot on the Lenovo
V-series; also, we don't need test reports from using tools that modify the
image when writing to disk.

[sudodus (nio-wiklund)]

So you want us to test only USB boot drives with the live system *cloned* from the iso file?

[sudodus (nio-wiklund)]

The Ubuntu Groovy iso file dated Oct 8 13.02

$ md5sum groovy-desktop-amd64.iso
0ca0a6242caffcb7c67497fc659f2b8c groovy-desktop-amd64.iso

is still affected by this bug. Maybe you should ask the manufacturer what they are looking for instead of this trial and error game?

[Steve Langasek (vorlon)]

On Thu, Oct 08, 2020 at 01:10:12PM -0000, sudodus wrote:
> @ Steve Langasek (vorlon),

> Would you be able to ask the manufacturer, either Lenovo or InsydeH2O,
> what they are looking for in their "Linpus Lite" boot option (in order
> to boot in UEFI mode)?

Potentially, but this would take some time to work through channels.
Hopefully the latest image Just Works™ and it will be irrelevant to pursue
this.

[Steve Langasek (vorlon)]

On Thu, Oct 08, 2020 at 01:35:08PM -0000, sudodus wrote:
> So you want us to test only USB boot drives with the live system
> *cloned* from the iso file?

Yes, please.

[sudodus (nio-wiklund)]

If the Ubuntu iso file according to comment #163 is the one you want us to test, I'm sorry, but it is still failing when cloned.

[Leó Kolbeinsson (leok)]

Just tested my Lenovo V14 IIL pn the daily suggested in comment #163 - fails to boot in UEFI with/without secure boot. I am also convinced that the BIOS in the machine is the cause of the failures my USB media is also shown as #Linpus Lite" unless I boot in BIOS mode.

BIOS version DKCN48WW
EC version DKEC48WW
MTM 82C400UUMX
Lenovo SN PF1CY8F4

[Thomas Schmitt (scdbackup)]

Hi,

has it already been tested whether a grub-mkrescue ISO boots to a
GRUB prompt ?

To try it, install

  grub-common grub-pc-bin grub-efi-amd64-bin

run

  mkdir minimal
  touch minimal/empty-file.txt
  grub-mkrescue -o output.iso minimal

and use output.iso like groovy-desktop-amd64.iso.

No menu or so is to be expected. Only "grub> _".
  https://i.stack.imgur.com/DEL0D.jpg

-------------------------------------------------------------------------

I really doubt that it makes a difference, but if a grub-mkrescue ISO
boots, we could try its pure GPT partition table by xorriso -as mkisofs
option

  -appended_part_as_gpt

which produces a "Protective MBR" partition table which indicates a
valid GPT.

-------------------------------------------------------------------------

If GPT does not boot either, then the difference must be in the EFI
partition. Either its storage location or its content.

The location could be tested by a stripped-down groovy-desktop-amd64.iso
with fewer payload but full boot equipment.

About the content of the EFI partition i cannot say much.

Have a nice day :)

Thomas

[sudodus (nio-wiklund)]

Hi Thomas,

I tried to make a minimal grub iso file, but failed. After installing xorriso too, I had the following error:

$ sudo grub-mkrescue -o output.iso minimal
grub-mkrescue: error: `mformat` invocation failed
.

I tried in an Ubuntu Groovy system. What is mformat? What should I add to make it work? Or would it be easy for you to upload a minimal iso somewhere, so that we (Leo and I) can download and test it?

[Steve Langasek (vorlon)]

On Thu, Oct 08, 2020 at 06:13:56PM -0000, sudodus wrote:
> Hi Thomas,

> I tried to make a minimal grub iso file, but failed. After installing
> xorriso too, I had the following error:

> $ sudo grub-mkrescue -o output.iso minimal
> grub-mkrescue: error: `mformat` invocation failed
> .

> I tried in an Ubuntu Groovy system. What is mformat? What should I add
> to make it work? Or would it be easy for you to upload a minimal iso
> somewhere, so that we (Leo and I) can download and test it?

mformat is from the mtools package, not installed by default.

[sudodus (nio-wiklund)]

Thanks Steve :-)

[sudodus (nio-wiklund)]

Booting this minimal device behaves differently in the Lenovo V130 in UEFI mode. Linpus Lite is *not* activated, but in the temporary menu there is an entry

EFI USB device (<name of the USB pendrive>)

1. Secure boot stops further booting, so the Lenovo will not reach to a grub prompt.

2. When I turn off secure boot, the Lenovo boots to the grub prompt :-)

3. The Dell boots to the grub prompt both in BIOS mode and UEFI mode.

Success :-) What's next :-P

[Leó Kolbeinsson (leok)]

@sudodus

Can you send me a link to the iso so I cn test it as well?

[Thomas Schmitt (scdbackup)]

Hi,

> 3. The Dell boots to the grub prompt both in BIOS mode and UEFI mode.
> Success :-) What's next :-P

You could try a GPT partitioned groovy.iso.

In the hope that the automat knows best:

  test -f test.iso && rm test.iso

  xorriso \
    -indev groovy-desktop-amd64.iso \
    -outdev test.iso \
    -boot_image any replay \
    -boot_image any appended_part_as=gpt \
    -boot_image any mbr_force_bootable=off

The "replay" command is supposed to apply the commands to set up the
same boot equipment as the original ISO has. The two following commands
-boot_image switch from MBR partition table to GPT and disable the
forced boot flag, which was inherited from the original's boot flag.

Partition editors should then report GPT in test.iso . Like

  $ /sbin/gdisk -l test.iso
  ...
  Found valid GPT with protective MBR; using GPT.
  ...
  Number Start (sector) End (sector) Size Code Name
     1 64 5714919 2.7 GiB 0700 ISO9660
     2 5714920 5724871 4.9 MiB EF00 Appended2

(The alternative to above replay automat would be what i show in
   https://bugs.launchpad.net/ubuntu/+source/casper/+bug/1895131/comments/22
 with other numbers. For
   0ca0a6242caffcb7c67497fc659f2b8c groovy-desktop-amd64.iso
 they would be 5715916 and 9952.)

-------------------------------------------------------------------------

About the theory that the position of the EFI partition is too far up:

  test -f test.iso && rm test.iso

  xorriso \
    -indev groovy-desktop-amd64.iso.1 \
    -outdev test.iso \
    -boot_image any replay \
    -rm_r /pool /casper/filesystem.squashfs --

This produces a much smaller ISO with MBR partition table (add above
-boot_image commands to get GPT):

  $ /sbin/fdisk -l test.iso
  ...
  Disklabel type: dos
  ...
  Device Boot Start End Sectors Size Id Type
  test.iso1 * 64 238375 238312 116.4M cd unknown
  test.iso2 238376 248327 9952 4.9M ef EFI (FAT-12/16/32)

It will probably not boot successfully beyond GRUB, because the ~2 GB of
filesystem.squashfs will be painfully missed as soon as the kernel is up.

But if it gets to the Ubuntu GRUB menu then this would indicate that
the start address of the EFI partition matters.

(Another test could be to move the EFI partition of a working USB stick
 high up and to check whether it then does not boot any more.)

-------------------------------------------------------------------------

I refrain for now from proposing to try a partition table layout like
with the older ubuntu ISOs (the barely legal one).

Have a nice day :)

Thomas

[Leó Kolbeinsson (leok)]

Booting in minimal device on Lenovo V14 fails in all modes - media not even recognized by BIOS
Tested on Acer machine - media recognized and boots to grub prompt in all modes/BIOS/UEFI/UEFI+secure boot.

[Leó Kolbeinsson (leok)]

Did some more testing on the Lenovo V14 IIL box and thought it might prove interesting to try other
Linux distro´s - Manjaro KDE Minimal and openSUSE Leap KDE - both fail to boot in all boot modes.
Also in these tests the media is also called "Linpus Lite".

[sudodus (nio-wiklund)]

@ Leó Kolbeinsson (leok),

I am not sure what test you reported in comment #175. Maybe I am too late, but this morning I uploaded my minimal iso file and the corresponding md5sum to

https://phillw.net/isos/linux-tools/alpha-beta/groovy/

$ md5sum -c bug-1886148_comment-172-173.iso.md5
bug-1886148_comment-172-173.iso: OK

[Leó Kolbeinsson (leok)]

@ sudodus

Re comment # 175

Sorry for not being more specific (I made and tested a minimal iso file). that is what I tested - should have let you know -thanks for sending the link.

[sudodus (nio-wiklund)]

@ Leó Kolbeinsson (leok),

I am surprised, that what works in my Lenovo does not work in yours. But the computers are different, and we can only guess what works before we have tested.

[Thomas Schmitt (scdbackup)]

Hi,

Leó Kolbeinsson wrote:
> Booting in minimal device on Lenovo V14 fails in all modes [...]
> Tested on Acer machine - media recognized and boots to grub prompt in all
> modes/BIOS/UEFI/UEFI+secure boot.

I assume that this was the grub-mkrescue ISO. Right ?

It is a bit a surprise that Secure Boot works on the Acer.
But that it absolutely does not work on the Lenovo V14 indicates a general
problem with GRUB.
If my assumption about the ISO being an unaltered grub-mkrescue ISO, then
we should consider to forward this info to GRUB's Ubuntu maintainers and
in the end to grub-devel mailing list.

-----------------------------------------------------------------------

> [...] on the Lenovo V14 IIL box [...]
> Manjaro KDE Minimal and openSUSE Leap KDE - both fail to
> boot in all boot modes.

And what about
  https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-10.6.0-amd64-netinst.iso
(~ 350 MB, ISOLINUX for PC-BIOS, GRUB2 for EFI, barely legal partition
 table with low address of EFI partition.)

Inspection results:

manjaro-kde-20.1.1-201001-linux58.iso :
Uses GRUB2 for PC-BIOS and EFI. The EFI partition is appended after ~ 3 GB.
Much like what 20201008/groovy-desktop-amd64.iso has.

openSUSE-Leap-15.0-KDE-Live-x86_64-Current.iso:
ISOLINUX for PC-BIOS, GRUB2 for EFI, The EFI partition is appended after
~ 900 MB. Except the ISOLINUX stuff it is like 20201008/groovy.

openSUSE-Leap-15.2-KDE-Live-x86_64-Media.iso:
GRUB2 for PC-BIOS and EFI. Elsewise like 15.0 and 20201008/groovy.

openSUSE-13.1-NET-x86_64.iso would be interesting. It has an EFI partition
with very low start address. (Kiwi's swallow-your-brother stunt.)

Have a nice day :)

Thomas

[sudodus (nio-wiklund)]

Hi Thomas,

I failed according to your instructions at comment #174.

$ xorriso \
> -indev groovy-desktop-amd64.iso \
> -outdev test.iso \
> -boot_image any replay \
> -boot_image any appended_part_as=gpt \
> -boot_image any mbr_force_bootable=off
xorriso 1.5.2 : RockRidge filesystem manipulator, libburnia project.

Drive current: -indev 'groovy-desktop-amd64.iso'
Media current: stdio file, overwriteable
Media status : is blank
Media summary: 0 sessions, 0 data blocks, 0 data, 46.5g free
Drive current: -outdev 'test.iso'
Media current: stdio file, overwriteable
Media status : is blank
Media summary: 0 sessions, 0 data blocks, 0 data, 46.5g free
xorriso : NOTE : No proposals available for boot related commands

Nothing was processed, the window returned to prompt at once after printing the above list.

[sudodus (nio-wiklund)]

@ Thomas,

There was a problem with permissions. I fixed it ...

@ everybody,

This test.iso file modified according to comment #181, when cloned, makes USB drive that can boot the Lenovo V130 in UEFI mode with secure boot.

Now there is a temporary boot option 'Linpus Lite' again (and it works).

The Dell boots both in BIOS mode and UEFI mode.

Success :-)

[Thomas Schmitt (scdbackup)]

Hi,

> $ xorriso \
> -indev groovy-desktop-amd64.iso \
> ...
> -boot_image any replay \
> ...
> xorriso : NOTE : No proposals available for boot related commands

This means that xorriso did not recognize any boot equipment in
groovy-desktop-amd64.iso .

> Drive current: -indev 'groovy-desktop-amd64.iso'
> Media current: stdio file, overwriteable
> Media status : is blank
> Media summary: 0 sessions, 0 data blocks, 0 data, 46.5g free

This looks like the input ISO is not accessible by path
groovy-desktop-amd64.iso, or is an empty file, or a file which begins by
64 KiB of zeros, or a file which has a specially devalued PVD in block 16.
Such files count as "blank medium in the drive".
(xorriso is a CD/DVD/BD burn program with the second job of making images.)

I just tested with xorriso-1.5.2 (newest release, i normally use the
development version) and got test.iso with GPT. So i expect that it will
work for you if you give it the ISO file.
The first "Media summary:" line of the run is for -outdev and should say

  Media summary: 1 session, 1431617 data blocks, 2796m data, ... free

The second line is for the -indev and should really say

  Media summary: 0 sessions, 0 data blocks, 0 data, ... free

because test.iso must not exist before this run.

Have a nice day :)

Thomas

[Leó Kolbeinsson (leok)]

@sudodus

Can you send me a link to test.iso so I can test exactly the same iso om V14?

[Thomas Schmitt (scdbackup)]

Hi,

> There was a problem with permissions.

Hm. I should make xorriso report this. No read permission should not
count as blank pseudo-medium.
What kind of permission was missing and which change fixed it ?

> This test.iso file modified according to comment #181, when cloned,
> makes USB drive that can boot the Lenovo V130 in UEFI mode with secure
> boot.
> > -boot_image any replay \
> > -boot_image any appended_part_as=gpt \
> > -boot_image any mbr_force_bootable=off

So it wants to see a valid GPT.
grub-mkrescue produces this by default.
Does installation to the stick by usb-pack-efi produce GPT ?

There remains the riddle why the V130 booted with older Ubuntus which
have an invalid GPT. (I assume that it does. Does it ?)
Even the 20201007.1 ISO had an invalid GPT, much like older Ubuntus.

Whatever, this success will probably not happen with Leó Kolbeinsson's
Lenovo V14 IIL, as it does not boot by grub-mkreascue.
Nevertheless, it would be worth a try. Just to be sure.

------------------------------------------------------------------------

I am running out of ideas. Open questions are:

- does Lenovo V14 IIL boot with
    https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-10.6.0-amd64-netinst.iso
  which has an old Fedora/Debian/Ubunto partition layout ?

- with which publicly downloadable ISO does V14 IIL boot at all ?

Do i get it right that the problems currently only affect sudodus'
Lenovo V130 and Leó Kolbeinsson's Lenovo V14 IIL ?
Any other machines which do not boot by the newest Ubuntu ISOs ?
(E.g. 20201008, MD5 0ca0a6242caffcb7c67497fc659f2b8c).

Have a nice day :)

Thomas

[Leó Kolbeinsson (leok)]

@scdbackup - Thomas Schmitt

I can confirm that the only box not booting for me is the V14 IIL .

all of my other Dell,Lenovo and MacMini machines boot as normal

[sudodus (nio-wiklund)]

Hi Thomas,

1. There was a name error, wrong name of the input iso file - easy to understand and fix, but I asked before thinking twice. There was also an ownership/permission error, the file was owned by root and there were not enough permissions for the standard user (because I had copied it from my main computer to the test drive with Groovy persistent live, where I installed programs and ran commands according to your advice).

2. mkusb creates a GPT by default, when making a persistent live drive. This can be modified in the settings menu. The default is not to use usb-pack-efi, and it has not been necessary with previous versions of Ubuntu (including 20.04.x LTS), it has worked to use 'grub-install command lines' (in the script dus-persistent).

After the big modification of the boot structure is has been necessary to use usb-pack-efi (selected in the settings menu) to make persistent live drives from Groovy iso files. I have not yet tested how mkusb can create persistent live drives with your modifications with the xorriso command line.

3. And yes, previous Ubuntu versions from when UEFI mode and cloning were established (I think around 12.10 or 13.04) it has worked to clone to USB drives, and these USB drives can boot both in BIOS and UEFI mode (including 20.04.x LTS and Groovy until the beginning of June and the big modification of the boot structure.

4. I am afraid, that there are several other new and fairly new Lenovo computers that are affected by this bug.

4.1 One of them is reported here by @RussianNeuroMancer in comment #151:

Lenovo Yoga C630 WOS with arm architecture

If I understand correctly, it can boot (in UEFI mode) from a USB drive with the content of the current Groovy iso file *extracted* to a FAT32 partition.

[sudodus (nio-wiklund)]

@ Leó Kolbeinsson (leok),

Now I have uploaded my modified Groovy iso file and the corresponding md5sum to

https://phillw.net/isos/linux-tools/alpha-beta/groovy/

$md5sum -c groovy-desktop-amd64_2020-10-08_fixed-by-xorriso.iso.md5
groovy-desktop-amd64_2020-10-08_fixed-by-xorriso.iso: OK

Good luck testing it :-)

[sudodus (nio-wiklund)]

Now I have tested how mkusb can create persistent live drives with Thomas Schmitt's modifications with the xorriso command line.

It is still working when using usb-pack-efi (and the default GPT)

- in the Lenovo V130 in UEFI mode with secure boot and

- in the Dell Precision M4800 both in BIOS mode and UEFI mode

-o-

With default settings (not using usb-pack-efi), a persistent live drive by mkusb works in UEFI mode (both in the Lenovo V130 and the Dell Precision M4800). But it does not work in BIOS mode. (This may be a headache for me, but not for the Ubuntu developers, and I mention it only for information.)

[Leó Kolbeinsson (leok)]

Tested the iso from sudodus (see comment #189) on the V14IIL Lenovo in all 3 modes- Legacy BIOS,UEFI,and UEFI+secure boot- success rate 100%. All 3 modes boot as normal.

[sudodus (nio-wiklund)]

Hi Leo,

This sounds good, but please tell us if you tested a system

- cloned or

- persistent live made by mkusb or

- made some other way.

[Leó Kolbeinsson (leok)]

@sudodus

re: comment 190/191

I used and tested the iso with 3 different methods none of them with persistent - all with the same results.

1. mkusb (live only or linux installer from iso file)
2. mkusb cloning iso file
3. rufus in Windows 10 (dd mode)

[Thomas Schmitt (scdbackup)]

Hi,

> on the V14IIL Lenovo in
> all 3 modes- Legacy BIOS,UEFI,and UEFI+secure boot- success rate 100%.
> 2. mkusb cloning iso file
> 3. rufus in Windows 10 (dd mode)

So now it looks like valid GPT does the trick for Ubuntu ISOs on Lenovo.

There remain riddles:

Why does the grub-mkrescue ISO not boot on V14IIL ?
(I still wonder which other ISOs boot on that machine.)

Why did the old Ubuntu ISOs with their invalid GPT boot on sudodus' V130
whereas a similarly invalid GPT did not help with the 20201007.1 ISO ?

Maybe Canonical Ltd. should ask Lenovo Group Limited for enlightenment.

-------------------------------------------------------------------------

The xorrisofs option to cause GPT during the production of an Ubuntu ISO
would be

  -appended_part_as_gpt

With GPT it is crucial for mountability of partition 1 (the ISO) to use
option

  -partition_offset 16

GPT partition 1 may start only after the GPT array. So it cannot start at
LBA 0. The best non-0 value for the partition start is ISO LBA 16 (GPT
LBA 64). Lower is not possible, higher is waste.

-------------------------------------------------------------------------

Valid GPT is fewly tested with popular ISOs.

If Ubuntu ISOs switch from MBR partition table to GPT, then it will become
interesting to see whether there are any EFIs which hate it (and why they
do).

Have a nice day :)

Thomas

[sudodus (nio-wiklund)]

Thanks Thomas for your insight in booting from a cloned image of an iso file and the efficient method to find how to squash this bug :-)

[Steve Langasek (vorlon)]

Thanks very much for the analysis, Thomas!

> There remain riddles:

> Why does the grub-mkrescue ISO not boot on V14IIL ?
> (I still wonder which other ISOs boot on that machine.)

> Why did the old Ubuntu ISOs with their invalid GPT boot on sudodus' V130
> whereas a similarly invalid GPT did not help with the 20201007.1 ISO ?

Yes, that is puzzling.

> Maybe Canonical Ltd. should ask Lenovo Group Limited for enlightenment.

I've escalated it internally to the right team, and they will follow up, but
no guarantees we'll get a meaningful answer :)

> -------------------------------------------------------------------------

> The xorrisofs option to cause GPT during the production of an Ubuntu ISO
> would be

> -appended_part_as_gpt

Thinking it through, I am happy for us to make this change from MBR to GPT.

- optical media don't care about the partition table, only the El-Torito
  boot image.
- BIOS boot of USB doesn't care about the partition table, only the MBR
- UEFI 2.0 requires systems to understand GPT
- pre-2.0 UEFI systems are mostly not guaranteed to work anyway.

I've made this switch now and confirmed the result at
<http://cdimage.ubuntu.com/ubuntu/daily-live/20201009.1>.

I do see that the GPT has an extra partition at the end; is this required
for alignment? I haven't seen such partition entries when using MBR.

$ gdisk -l ~/devel/iso/groovy-desktop-amd64.iso
GPT fdisk (gdisk) version 1.0.5

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.
Disk /home/vorlon/devel/iso/groovy-desktop-amd64.iso: 5736268 sectors, 2.7 GiB
Sector size (logical): 512 bytes
Disk identifier (GUID): 8897A8AA-43A3-4BD4-A4AA-C472E4376703
Partition table holds up to 248 entries
Main partition table begins at sector 2 and ends at sector 63
First usable sector is 64, last usable sector is 5736204
Partitions will be aligned on 4-sector boundaries
Total free space is 1 sectors (512 bytes)

Number Start (sector) End (sector) Size Code Name
   1 64 5725651 2.7 GiB 0700 ISO9660
   2 5725652 5735603 4.9 MiB EF00 Appended2
   3 5735604 5736203 300.0 KiB 0700 Gap1
$

Here is the full commandline used to construct this image.

xorriso -as mkisofs -r -checksum_algorithm_iso md5,sha1 -V Ubuntu\ 20.10\ amd64 -o /srv/cdimage.ubuntu.com/scratch/ubuntu/groovy/daily-live/debian-cd/amd64/groovy-desktop-amd64.raw -J -joliet-long -l -b boot/grub/i386-pc/eltorito.img -no-emul-boot -boot-load-size 4 -boot-info-table --grub2-boot-info --grub2-mbr cd-boot-images/usr/share/cd-boot-images-amd64/images/boot/grub/i386-pc/boot_hybrid.img -append_partition 2 0xef cd-boot-images/usr/share/cd-boot-images-amd64/images/boot/grub/efi.img -appended_part_as_gpt -eltorito-alt-boot -e --interval\:appended_partition_2\:all\:\: -no-emul-boot -partition_offset 16 cd-boot-images/usr/share/cd-boot-images-amd64/tree CD1

[Marcos Nascimento (wstlmn)]

With the 2020-10-09 - 9:15 p.m. update, it started in efi mode on lenovo ideapad 330-15IKB. :)

[sudodus (nio-wiklund)]

I can report *success* from testing with the current daily Ubuntu Groovy iso file

$ ls -l groovy-desktop-amd64.iso
-rw------- 1 sudodus sudodus 2936969216 okt 9 21:15 groovy-desktop-amd64.iso

$ md5sum groovy-desktop-amd64.iso
00cbcce8855c00dd2231077e3a01aebc groovy-desktop-amd64.iso

After cloning to a USB drive, it can boot the problematic Lenovo V130 in UEFI mode with secure boot :-)

I checked also with a Dell Precision M4800, and this computer boots nicely both in BIOS mode and UEFI mode. In the near future I will test also in other computers, that are available, and I will blow the whistle if necessary.

See also the attached text file.

[Thomas Schmitt (scdbackup)]

Hi,

> Thinking it through, I am happy for us to make this change from MBR to
> GPT.

Me not so much. GPT has that backup at the end of the device. But when
an image is made, the size of the storage device is not known. So after
putting (cloning) the image onto the USB stick, the backup GPT is
misplaced.

So we'd actually need an adjustment step after cloning, although the boot
firmwares don't care for that flaw. But partition editors do.
(Modern versions of program sfdisk fix the problem silently when any
 manipulation of GPT is done. E.g.:
   echo 'name=DATA' | sudo sfdisk -a /dev/sdc
)

But if more machines boot by GPT than by MBR partition table, i can hardly
be against it. Let's see what problems this causes ...

> I do see that the GPT has an extra partition at the end; is this required
> for alignment?

It's the padding against the TAO CD Read-Ahead Bug. This bug is caused by
an ambiguity in SCSI MMC specs about command READ CAPACITY when the last
track ends by two TAO Run-out blocks. About 80 percent of CD capable drives
count them as readable, which they are not for data read command READ(10).

By a cargo cult theory of last century, ISO 9660 producers and most burn
programs add 300 KiB of padding to keep legitimate read operations away
from the dangerous end of the device.
Actually it would have to be the buffer size of Linux when reading ahead
of the block range that is requested by the ISO 9660 filesystem driver.

(If i had a better standing at linux-scsi it would already be fixed for
 single session by testing the last two blocks for readability and
 adjusting the device size perception of the kernel if they are not
 readable by READ(10). I have made me a kernel which does this.)

With ISO images which never end up on a CD you can safely suppress this
padding by xorriso -as mkisofs option

  -no-pad

Even then most CD burn programs will add their own padding by default or
use write type SAO which produces no Run-out blocks.

> I haven't seen such partition entries when using MBR.

With MBR partition table the padding is either counted as part of
partition 1 (old Fedora/Debian/Ubuntu layout) or not marked as partition
(with appended partition).

The reason for that GPT partition is probably in my discussions with
Vladimir Serbinenko when i made xorriso ready for serving under
grub-mkrescue. If we find compelling reasons not to create it, i could
try to suppress this.
(The boot preparation code in libisofs is quite entangled by the many
 variations and little stunts which it accumulated over time.)

Have a nice day :)

Thomas

[Leó Kolbeinsson (leok)]

I can also confirm support testing current daily Ubuntu Groovy iso file
http://cdimage.ubuntu.com/daily-live/20201009.1/groovy-desktop-amd64.iso

Tested 3 machines with USB media created with "startup disk creator"

1. Dell 3521 Inspirion
2. Lenovo V14 IIL
3. Leonovo x230i

All booted in BIOS and UEFI + secure boot.

[Leó Kolbeinsson (leok)]

Correction to comment #199

I can also confirm success testing current daily Ubuntu Groovy iso file
http://cdimage.ubuntu.com/daily-live/20201009.1/groovy-desktop-amd64.iso

Tested 3 machines with USB media created with "startup disk creator"

1. Dell 3521 Inspirion
2. Lenovo V14 IIL
3. Leonovo x230i

All booted successfully in BIOS and UEFI + secure boot modes.

[sudodus (nio-wiklund)]

More tests - here is a list of all computers tested by me with the Ubuntu Groovy system

$ ls -l groovy-desktop-amd64.iso
-rw------- 1 sudodus sudodus 2936969216 okt 9 21:15 groovy-desktop-amd64.iso

$ md5sum groovy-desktop-amd64.iso
00cbcce8855c00dd2231077e3a01aebc groovy-desktop-amd64.iso

cloned from the iso file to a USB drive (except for VirtualBox). Please note that tested mode shows what is tested, and a missing mode does not indicate failure, only that is was not tested. Booting was successful in all these tests performed today, 2020-10-10.

Computer spec - tested mode
----------------------------------------
Lenovo V130 - UEFI mode secure boot
Lenovo X131e - UEFI mode
Dell Precision M4800 - BIOS & UEFI modes
Dell Latitude E7240 - BIOS & UEFI modes
HP Probook 6450b - BIOS mode
Toshiba Satellite Pro C850-19W - BIOS & UEFI modes (also with secure boot)
VirtualBox 6.1.10_Ubuntu r138449 - iso file emulating optical disk - BIOS mode

[Leó Kolbeinsson (leok)]

Updated test list current daily Ubuntu Groovy iso file
http://cdimage.ubuntu.com/daily-live/20201009.1/groovy-desktop-amd64.iso

Tested machines with USB media created with "startup disk creator"

Booting successful in all tests

Computer spec - tested mode
----------------------------------------
Dell 3521 Inspirion - BIOS + UEFI & UEFI + secure boot
Lenovo V14 IIL - BIOS + UEFI & UEFI + secure boot
Leonovo x230i - BIOS + UEFI & UEFI + secure boot
Lenovo L450 - BIOS + UEFI & UEFI + secure boot
Lenovo T430i - BIOS + UEFI (secure boot not supported by BIOS)
Acer E3-111-P60S - BIOS + UEFI+ secure boot
Dell Latitude 7280 - BIOS + UEFI+ secure boot
Lenovo M72e - BIOS + UEFI+ secure boot
QEMU virtual machine running in Dell 7040 - BIOS mode

[sudodus (nio-wiklund)]

I have tested the current daily (2020-10-10) Xubuntu and Lubuntu iso files too. When cloned to a USB drive, they can also boot the Lenovo V130 in UEFI mode (and secure boot). It seems that bug #1895329 is squashed too.