lubuntu/xubuntu reinstall (& install) on dual boot system, grub does appear & offer OS choice

[Chris Guiver]

Warning: os-prober will not be executed to detect other bootable partitions.
----

Lubuntu noble install to
- hp dc7700 (c2d-e6320, 8gb, amd/ati rv610/radeon hd2400 pro/xt)

System had two Lubuntu noble installs on box; original full disk install (sda1) + install alongside (sda2)
This box is legacy (BIOS) or pre-uEFI (2005 HP Compaq) so no ESP was created at install time

guiverc@dc7700-3:~$ blkid
/dev/sda2: LABEL="lubuntu_2404" UUID="4b96ae36-ff60-46ff-888d-bab52f0664fc" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="b22c54df-02"
/dev/sda1: LABEL="lubuntu_2404" UUID="24611116-48e0-47e3-af47-4cd692dcdbd6" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="b22c54df-01"

** expected result

On reboot post-install (Lubuntu so calamares install) I grub did NOT appear, so I could boot into the untouched install & checked it worked (grub offered me two options correctly prior to install).

** actual result

No grub menu appeared, so no dual boot option was offered??

No problem, I'll just `update-grub` and that should fix it won't it?

guiverc@dc7700-3:~$ sudo update-grub
[sudo] password for guiverc:
Sourcing file `/etc/default/grub'
Sourcing file `/etc/default/grub.d/lubuntu-grub-theme.cfg'
Generating grub configuration file ...
Found theme: /usr/share/grub/themes/lubuntu-grub-theme/theme.txt
Found linux image: /boot/vmlinuz-6.8.0-22-generic
Found initrd image: /boot/initrd.img-6.8.0-22-generic
Found memtest86+x64 image: /boot/memtest86+x64.bin
Warning: os-prober will not be executed to detect other bootable partitions.
Systems on them will not be added to the GRUB boot configuration.
Check GRUB_DISABLE_OS_PROBER documentation entry.
Adding boot menu entry for UEFI Firmware Settings ...
done

The issue is
Warning: os-prober will not be executed to detect other bootable partitions.

Prior releases of Ubuntu haven't done this (new grub 2.06 change) with a patch (ubuntu-os-prober-auto.patch) carried by Ubuntu that reverses this upstream change..

If this is new behavior (for noble) we (Lubuntu at least) just need to document it, as its a change to mantic, lunar & prior releases (inc. kinetic, jammy which were 2.06 or later, it was default prior to jammy anyway).

Maybe I'm missing something... I'll correct/add more as I can

ProblemType: Bug
DistroRelease: Ubuntu 24.04
Package: grub2-common 2.12-1ubuntu7
ProcVersionSignature: Ubuntu 6.8.0-22.22-generic 6.8.1
Uname: Linux 6.8.0-22-generic x86_64
ApportVersion: 2.28.0-0ubuntu1
Architecture: amd64
CasperMD5CheckResult: unknown
CurrentDesktop: LXQt
Date: Tue Apr 9 16:21:14 2024
InstallationDate: Installed on 2024-04-09 (0 days ago)
InstallationMedia: Lubuntu 24.04 LTS "Noble Numbat" - Daily amd64 (20240408)
SourcePackage: grub2
UpgradeStatus: No upgrade log present (probably fresh install)

[Chris Guiver (guiverc)]

I vaguely recall an indirect discussion with AndreaRighi (or Julian Andres Klode maybe) who mentioned a (possible?) change coming thru for noble.. I'm wondering if this is my discovery of what was mentioned.. alas I can't recall the 'comment' this reminds me of

My testing today can be read here - https://iso.qa.ubuntu.com/qatracker/milestones/450/builds/299105/testcases/1701/results/

Either way my install is perfect, except the dual-booting option offered by GRUB got lost.. This is behavior I get on Debian, so I know how to fix it, just don't expect it with a Ubuntu re-install.

[Ubuntu QA Website (ubuntuqa)]

This bug has been reported on the Ubuntu ISO testing tracker.

A list of all reports related to this bug can be found here:
http://iso.qa.ubuntu.com/qatracker/reports/bugs/2060624

[Chris Guiver (guiverc)]

The Xubuntu install was an install into one partition (SDA2) where two partitions exist both with OSes (SDA1 & SDA2).

The resulting Xubuntu install booted without grub appearing on boot, and no way for a user to boot into prior SDA1 install of Lubuntu noble.

Attempt as in original post of `sudo update-grub` reports

Warning: os-prober will not be executed to detect other bootable partitions.

thus my prior 24.04 system is not offered as boot option.

[Chris Guiver (guiverc)]

The Xubuntu install was an install into one partition (SDA2) where two partitions exist both with OSes (SDA1 & SDA2).

The resulting Xubuntu install booted without grub appearing on boot, and no way for a user to boot into prior SDA1 install of Lubuntu noble.

Attempt as in original post of `sudo update-grub` reports

Warning: os-prober will not be executed to detect other bootable partitions.

thus my prior 24.04 system is not offered as boot option.

I've NOT encountered this issue with an auto(resize)/install-alongside install, ONLY with something-else/manual-partitioning type of installs.. so its not all installs.

[Julian Andres Klode (juliank)]

I cannot comment on what is happening here, but the patch is still there and hasn't changed. If you delete /boot/grub/grub.cfg it would run os-prober again or give a sensible error for what's happening.

[Chris Guiver (guiverc)]

I've NOT encountered this issue with a install.alongside (calamares/lubuntu) or the ubuntu-desktop-installer equivalent (auto(resize) testcase)... as both installs (prior + new) appear in GRUB on second install when that is run. This is where it matters most anyway, and it works perfectly.

I've only encountered this on manual.partitioning (something-else equivalent) and manually selecting a pre-existing partition for re-use (ubuntu-desktop-install now marks that as with format compulsory)...

This will impact very few users.. and can easily be covered by a RELEASE NOTES message if required anyway, as very few users will be impacted by it (most will have no issue dealing with the PROBER edit)..

--

Alas the Xubuntu install no longer exists; that install being overwritten by subsequent QA test installs. I'd have to re-create the issue, which won't be tonight...

[Mate Kukri (mkukri)]

I haven't managed to reproduce this issue yet, I've installing various dual boot combinations of Xubuntu/Lubuntu into a BIOS qemu VM.

@guiverc If you could provide exact steps to reproduce this starting from an empty virtual machine, that would be extremely helpful.

[Chris Guiver (guiverc)]

I have made many installs of various boxes, and exploring logs of the iso.qa.ubu link in comment #3 i see only two references to this issue (within a few days of each other), which include the following

---
hp dc7700 (c2d-e6320, 8gb, amd/ati rv610/radeon hd2400 pro/xt)
testcase: install using existing partition
---

I've made four installs on that box today (https://iso.qa.ubuntu.com/qatracker/milestones/453/builds/300140/testcases/1701/results/) yet did NOT experience the issue. Currently the box has two installs (first was full disk, then auto-resize equiv.. ) so I'll try and repeat the install I made in the next hour+ and see if it occurs.

Lubuntu testcases are described here - https://discourse.lubuntu.me/t/testing-checklist-understanding-the-testcases/2743

but an INSTALL USING EXISTING PARTITION means

- boot install media
- using manual partitioning, selecting an existing partition (for /) BUT DO NOT FORMAT (ie. re-use it...)

This is NO longer possible with Xubuntu as FORMAT is now mandatory (due to another bug, scheduled for fix post-release of noble..)

I'll add some non-standard packages to one of my installed systems on dc7700 (per testcase; something for installer to re-install), then attempt to re-create this issue & reply back in an hour (or three) and see if it re-occurs... then provide detailed description of what I did (if issue re-occurs).

[Chris Guiver (guiverc)]

I completed an install on dc7700

- two systems (noble) exist on drive
- boot install media (lubuntu noble RC)
- select partition (I selected SDA2, ignoring SDA1) for `/` WITHOUT FORMAT
- let it install

(this system is OLD, legacy-BIOS so no ESP; Lubuntu's calamares doesn't create one by default for BIOS machine; iso.qa report at https://iso.qa.ubuntu.com/qatracker/milestones/453/builds/300140/testcases/1701/results/)

on reboot; two options appeared in grub.

- SDA1 was in grub & when booted looked untouched
- SDA2 contained new install, file-system system directories all time-stamped mins ago (not hours ago) as expected for new install.. packages I added are present (fortune, clementine etc)..

I COULD NOT RE-CREATE THIS ISSUE..

Only difference between this and prior install could have been I selected second partition (SDA2) where last time I selected SDA1?? but I can't see that as making an difference; could it??

Given I've only experienced this issue twice, and could not re-create.. I'd be happy with marking bug as INCOMPLETE..

I could re-try by installing to SDA1 (same type of install) if you think there is any chance of difference.. but I couldn't re-create issue today (me attempting same type of install on same box)

[Mate Kukri (mkukri)]

Hi Chris, i'll try the same thing:
- two preinstalled noble system on MBR, with sda1 and sda2
- install lubuntu without reformat on sda1 first
- install lubuntu without reformat on sda2 second

If none of that reproduces, i'll mark it incomplete.

[Mate Kukri (mkukri)]

@guiverc, it seemingly reproduced with an install to sda1, but only _without_ the format

[Mate Kukri (mkukri)]

Hmm it seemingly reproduced once, but then i fixed it with dpkg-reconfigure grub-pc, did another install to overwrite it, and it was working there.

[Mate Kukri (mkukri)]

Reproduced again. I think the steps are, and they have to be repeated from step 1 each time:

Start with clean disk:
1. Partition disk as MBR with two partitions, install lubuntu on sda1 _with_ format
2. Install lubuntu on sda2 _with_ format
3. Install lubuntu on sda1 _without_ format

I'll spend a bit more time later trying to figure out the exact root cause, but this seems like a rather specific issue that shouldn't be a release blocker for sure.

[Chris Guiver (guiverc)]

Just an FYI... but on the two cases I've encountered this issue I didn't install as per #14. I'll have always performed a

1. full disk install (any prior partitioning erased); noble (may not have been lubuntu) but single partition only
2. install alongside (calamares wording; auto-resize type creating a second install)

(ie. I did two install types & achieved same result as steps 1 & 2 as Mate (mkukri) in comment #14)

At step 3 my usual thing is a 'replace partition' (calamares wording) install using one of the partitions (this is a FORMAT install).. where which partition I use is ~random (usually opting to keep the partition I'll most recognize as untouched; the install where I changed the wallpaper usually so easily recognized)..

Any testing of 'install using existing partition' for me would be 4th (or 5th) install almost guaranteed I believe..

My point is it can be triggered in other ways I believe, but again very few users will encounter it... I rarely encountered it!

(then again what I listed ^ is what I did today, do most days probably too.. maybe why I've not experienced it much as I suspect i re-use SDA2 most days (& not sda1); ie. changing wallpaper on that first install (of the day) so its recognized on the many installs that usually follow...)

I concur with view not a release blocker.

-- said by me on #lubuntu-devel (matrix)
> It may worthwhile (if not fixed & I doubt it will be) for me to create a page on [lubuntu]discourse(doco) for users should they encounter this issue ... to me it's not a big issue; I've only encountered it rarely, it requires specific conditions I doubt many will encounter.. but could be mentioned in release notes as well with somewhere we can point users too...

[Mate Kukri (mkukri)]

Yes, I agree there might be other ways to trigger it, I'm just leaving a reproducible one I found above.

And not being a release blocker doesn't mean it won't be fixed btw, just that I am not going to rush this.

[Chris Guiver (guiverc)]

(
another FYI.... I mentioned it's no longer re-creatable with Xubuntu due to forced FORMAT currently with `ubuntu-desktop-installer` ...

the issue i'm referring to is/was https://bugs.launchpad.net/ubuntu-desktop-provision/+bug/2058638 .. current 'fix' being to force format , but my understanding is that will be reverted once time is available to correct the current 'regression' format triggers with 24.04 over 23.10 & earlier)

This bug (2060624) thus can only be triggered (I believe) by the calamares installer used Lubuntu, Kubuntu & Ubuntu.Unity; as ubuntu-desktop-installer is forcing format (at least for now).
)

[ԜаӀtеr Ⅼарсһуnѕkі (wxl)]

I *think* this is the same issue as https://bugs.launchpad.net/ubuntu/+source/calamares/+bug/2063354 which calamares 3.3.5-0ubuntu4 seeks to fix. That should be migrating shortly and we'll respin Lubuntu.

[Chris Guiver (guiverc)]

install using Lubuntu 20240425 ISO (latest RC as of now) and issue reoccurred. on ISO is

calamares 3.3.5-0ubuntu4

as before

- erase disk & install to dc7700
- install alongside (slider moved random amount as usual, so size [of partitions] aren't critical
- install & re-use SDA1 (first installed partition; what was shrunk) and critically NO-FORMAT is used

on reboot, no grub menu appears.
if I go to terminal and `sudo update-grub`... no other OS is shown, with 'OS_PROBER will not be executed' message appears..

I can't boot into the other OS that was present before this QA-install test.

-- as stated on matrix

`sudo vim /etc/default/grub` .. uncomment the OS_PROBER line, save & exit .. then `sudo update-grub` and now on reboot the install is exactly as expected/wanted (ie. other OS(es) are now offered at grub ...)

After that change.. I can boot the other OS, and everything looks as expected... This procedure to workaround issue is easy to document

[Mate Kukri (mkukri)]

This is not the same issue unfortunately, and will be investigated further.

[Mate Kukri (mkukri)]

I finally found the cause, the two big hints were:
- 2 fresh installs were in this order sda1, sda2
- only happens with install *without a reformat to sda1*

After the first install to sda1:
- MBR (and MBR gap) contains GRUB core pointing to sda1, sda1 contains modules and config, this config on sda1 contains no os-prober entries as this was a fresh install

After the second install to sda2:
- MBR (and MBR gap) contains GRUB core pointing to sda2, sda2 contains modules and config, this config on sda2 now has os-prober entries for the install on sda1
- everything seems normal
- sda1's grub config still doesnt contain os-prober entries, but it is dormant now

After the third install *without reformat to sda1*:
- MBR contains GRUB core again pointing to sda1, sda1 contains modules and config
- while the installer regenerates the GRUB config here, during this re-generation the old config *on sda1* (due to no format) is checked for the presence of os-prober entries, and these won't appear in the dormant config from step 1, thus no new os-prober entries for the install on sda2 will be generated

As a result sda1 was just a symptom and not the true cause, the true cause is overwriting an install with a dormant grub config containing no os-prober entries.

The reason a no reformat re-install to sda2 didn't trigger is due to the existing grub config containing os-prober entries from step2. If the initial installs were in a different order, e.g. sda2 then after it sda1, then installing without reformat to sda2 will also trigger it.

[Chris Guiver (guiverc)]

Thank you Mate for working out issue & great explanation..

[Mate Kukri (mkukri)]

Tagging calameres fix.

Desktop/server installer also potentially will need a fix for this if no reformat installs will be re-eanbled.

[ԜаӀtеr Ⅼарсһуnѕkі (wxl)]

Mate, what exactly is the right solution here, then, if we don't force reformats?

[Mate Kukri (mkukri)]

@wxl

There are two solutions:
- Always remove the existing GRUB configuration before installing GRUB
- If you know that os-prober entries are required in the new install, use, GRUB_DISABLE_OS_PROBER=false grub-install ...

[Neil Nelson (nnelson)]

I have Xubuntu 23.04 on one drive and 22.04 on another drive as a backup. I have upgraded to 24.04 twice, once when it became available and some time around the middle of June. The computer would not boot after I upgraded. I also have Windows 11 on another drive. I am looking forward to when I can install 24.04 and have it work OK.

[Chris Guiver (guiverc)]

@@nnelson/Neil, this is a specific bug thread on a very rarely triggered specific install case and NOT a support thread/site.

You can get help with problems at the support tracker at https://answers.launchpad.net/ubuntu , or if you would prefer live chat support, you can find an IRC support channel for your flavor of Ubuntu here: https://wiki.ubuntu.com/IRC/ChannelList. You can also find help with your problem in the support forum of your local Ubuntu community http://loco.ubuntu.com/ or asking at https://askubuntu.com or https://ubuntuforums.org.

Ubuntu 22.04 LTS upgrades to 24.04(.1) are NOT open, having been delayed (see details on Ubuntu community hub, which includes a list of the blocker bugs & more), but upgrades from Ubuntu 23.04 were supported only to the next release, and ended when Ubuntu 23.10 reached EOL (23.10 upgrades to 24.04 were open for ~three months prior to 23.10's EOL)