Non trivial grub2 installs no longer fit in small embed areas

Bug #1059827 reported by Phillip Susi
466
This bug affects 109 people
Affects Status Importance Assigned to Milestone
grub2 (Ubuntu)
Triaged
Wishlist
Unassigned

Bug Description

Traditionally the first boot track of the disk was left unpartitioned. This area is used to embed the grub2 core.img file. The size of this area used to typically be 62 sectors. In recent years the typical size has changed to 2048 sectors to keep the partitions aligned to a 1 MiB boundary for performance reasons on SSDs and newer hard disks with 4KiB sector sizes. All but the most trivial configurations of grub no longer fit in the old 62 sector size embed area. This results in grub complaining that your embed area is unusually small.

Upstream appears to have no desire to support such configurations, so this is unlikely to be fixed, but I will leave this bug report open for now. The workaround for the problem is to repartition the disk with modern partitioning tools that will align partitions to 1MiB, thus leaving 2048 sectors for the embed area.

Another workaround is to setup a simple ext4 /boot partition rather than try to boot directly from raid or lvm or btrfs.

Revision history for this message
YannUbuntu (yannubuntu) wrote :

Looking at current duplicates show only 3 cases :

1) Bug #491740 : disk with abnormally small (<32kb) embed area. GRUB Legacy OK , GRUB2 KO --> regression

2) Bug #1004376 : RAID+LVM : 10.04 ok , 12.04 KO --> regression

3) BTRFS:
KO on 11.04 (Bugs #782543 , #781010 , #774217)
KO on 11.10 (Bug #885264 , #881342 , #879109 , #873392 , #842247 , #813259 , #800840)
KO on 12.04 (Bugs #990558 , #988848 , #953559 , #945449 , #928128 , #917212 ),
KO on 12.10 (Bug #1045178)
---> not a regression, unless we show it worked with a previous version of GRUB.

Revision history for this message
YannUbuntu (yannubuntu) wrote :

4) LVM ok on 10.04, KO on 12.04 --> regression ( http://askubuntu.com/questions/196246 )

Revision history for this message
NounouRs (sandburg) wrote :

When you say partitioning disk with modern tools... ok, let's said we have modern tools on 12.04.1 boot CD (that is not really the case since LVM and RAID are only on alternate CD... why ? is libre-office more relevant than a complete installing system ?)

When a disk is double partitionned, for example LVM over RAID, did it need to partition from the basement, that meens RAID level, or only the top final partitions, LVM level ?

If it needs to partition from the low level, and if it is not going to be fixed, I think it will need a tool, a script or whatever to get around this bug.

tags: added: karmic natty oneiric precise quantal
Revision history for this message
Jean.c.h (slug71) wrote :

I tried installing both 12.04 and 12.10 using BTRFS for both / and /home and Grub failed to install with each of them. Tried on both my Netbook as well as my test rig(P4). When I dig a little deeper, it seems Grub complains about core.img being "unusually large".....

An interesting observation I made today though.
On my test rig I have a number of installations of numerous distros. Havent fired this up in a couple of years mind you so I still had the Lucid development install as the latest on here....lol.
On the box I had a install of Fedora 13(which was the latest stable at the time IIRC) on BTRFS. Installed as both / and /home. Fedora 13 still had legacy Grub and I remember Lucid with Grub2 not being able to boot Fedora 13.
Well last night I replaced the Lucid install with 12.04 on Ext4(since BTRFS wounldn't work) and then today decided to check if Fedora would boot. Sure enough it boots! So I KNOW they can play nice.

Revision history for this message
Jean.c.h (slug71) wrote :

So not sure what has changed in Grub2 between Lucid and Quantal but its strange that it will boot F13 just fine but not a current Ubuntu version on BTRFS.

Revision history for this message
Swâmi Petaramesh (swami-petaramesh) wrote :

BTRFS support takes a larger disk space, so you either must leave a couple (say 4) MB of free space between the start of he disk and first partition, or create a small "BIOS GRUB" partition there (that you don't mount or otherwise use, grub will take care of it).

Revision history for this message
YannUbuntu (yannubuntu) wrote :

Possible duplicate: Bug #1066324

Revision history for this message
Phillip Susi (psusi) wrote :

That is incorrect Swami. The bios_grub partition is required when using GPT, and 1MB is quite sufficient either for it, or for the embed area before the first partition on an MBR partitioned disk.

Revision history for this message
Jean.c.h (slug71) wrote :

I have a msdos partition table though. Not GPT.

Revision history for this message
Jean.c.h (slug71) wrote :

@ YannUbuntu

Bug #1066324 is a duplicate of this one. At the bottom of that bug it says to comment under this one. I have that same issue but also using 12.04.

Revision history for this message
Bill Duetschler (bikergeek) wrote :

I have this issue also, trying to upgrade Xubuntu 12.04 to 12.10 on a machine that uses LVM but not RAID.

Revision history for this message
axion (bugzilla-axion) wrote :

This bug is especially bad for people installing ubuntu as a secondary os on a windows system. Some factory windows installs do not leave space at the beginning of the hdd beyond the 62 sectors.
I hope this observation might get this bug escalated.

Revision history for this message
Phillip Susi (psusi) wrote :

Such a setup wouldn't be hit by this problem; it is only exotic setups that require things like raid or lvm that won't fit in 62 sectors.

Revision history for this message
Swâmi Petaramesh (swami-petaramesh) wrote :

Stating that either LVM or RAID are "exotic" is simply bullshit.

Revision history for this message
Swâmi Petaramesh (swami-petaramesh) wrote :

And BTW it's not only LVM or RAID that suffer from this issue, but also BTRFS installs, ZFS installs... Everything that is a bit more complex than ext3/4 on plain good ol'e DOS disk partitions...

Revision history for this message
Colin Watson (cjwatson) wrote :

Escalation won't particularly help as this is just plain difficult to fix: there's no single cause for the code growing beyond the relevant size, but rather gradual accretion of bug-fixes has meant the code is now just slightly larger than will fit. I don't regard LVM or RAID as exotic, and in that case this is a regression; they're close enough to the size limit that we may be able to squeeze things back down again in a future upstream version, but it will take some slow and steady work.

btrfs and ZFS are over the size limit by a very considerable margin, and it is unlikely that we will ever be able to squeeze support for those into 62 sectors. They were never there in the first place, so in that case this isn't a regression.

Revision history for this message
C Filorux (breakfast) wrote :

I managed to get this problem by creating a little partition - sda2 in the list below - trying to make use of that 'unused' space. This worked well enough when grub was on a regular partition, but it does not work when setting up /boot and grub on a LVM. The fix was to delete the partition with fdisk and run partprobe.

   Device Boot Start End Blocks Id System
/dev/sda1 * 2048 15624191 7811072 83 Linux
/dev/sda2 3 1727 862+ 83 Linux
/dev/sda3 15624192 625142447 304759128 5 Extended
/dev/sda5 15624195 625142447 304759126+ 8e Linux LVM

It would be good for grub to display "there are 3 sectors before the first partition available for core.img, and we need 65" (or whatever the case is) rather than to only say that the core image is large - it's not particularly large.

Phillip Susi (psusi)
summary: - Non trival grub2 installs no longer fit in small embed areas
+ Non trivial grub2 installs no longer fit in small embed areas
Phillip Susi (psusi)
description: updated
Revision history for this message
Phillip Susi (psusi) wrote :

That's the problem; the partition starts at sector 63 so that doesn't leave enough room for grub and the btrfs module.

Revision history for this message
Alberto Jovito (thedemon007) wrote :

Ok I reinstall, format the root / partition with the ext4 file system. If you still have not found the solution, the installer should warn about.this. https://bugs.launchpad.net/ubuntu/+source/grub-installer/+bug/1270729

Revision history for this message
D3m0n1q_733rz (cryo-rebirth) wrote :

There is no need to reformat with Ext4. If you wish to use BtrFS for the Boot partition, it's recommended. Just tell the partitioner to start at the 1MB boundary instead of the first sector. To reiterate, you need an empty 1MB at the beginning of your drive to write to. I would call this an error with the partitioner/Ubuntu Installer instead of Grub2.
To do this properly, start GParted and format it using this method.

Revision history for this message
Phillip Susi (psusi) wrote :

It's not an error with the partitioner or installer since it does start the partition at 1 MiB. The problem is when you have an old disk that you partitioned with say, Windows XP, which started the partition at sector 63.

Revision history for this message
Alberto Jovito (thedemon007) wrote :

The installer should detect that the partition starts at sector 63 and display a warning.

My netbook comes with Windows 7 preinstalled.

Revision history for this message
D3m0n1q_733rz (cryo-rebirth) wrote :

Actually, Phillip, BtrFS requires 2MB for a Grub2 installation. So, while you are correct that it does start at 1MB, the partitioner does have an error in not including that the rules for BtrFS are different. The normal 1MB is a part of the normal MBR rules. Since Grub2 requires some extra installation space for the BtrFS partition information, GParted should warn when BtrFS is used and a partition is started less than 2MB from the beginning of the drive. For everything else, 1MB works fine.
So, this would be a problem with the partitioner not realizing the rules are different and the installer not seeing that the partitioner made a mistake and that Grub2 can not be properly installed inside of the 1MB space for a BtrFS partition.

Ext4 and other partitions that are not BtrFS have no problem with the 1MB start point.

Revision history for this message
Phillip Susi (psusi) wrote :

No, it doesn't need anywhere near 1, let alone 2 MB. It is something like 45k.

Revision history for this message
D3m0n1q_733rz (cryo-rebirth) wrote :

Sorry, I was talking about an MBR system, not a UEFI. Core.img is too large to fit into the first 64 sectors (starting from 0). And, so, the next best place to start would be the 127 - 128 boundary. Also known as the 2MB boundary.

Revision history for this message
Phillip Susi (psusi) wrote :

You're not making sense. 2 MB is 4096 sectors. The default start location for the last several years has been sector 2048, or 1 MB, which leaves plenty of room for the core to fit.

Revision history for this message
Alberto Jovito (thedemon007) wrote :

I think the installer should be added to this bug. What is the name of the package that will install ubuntu? Or partition tool?

I do not know well the technical details of the problem but I agree with "cryo-rebirth" .. the installer or partitioner should warn when a partition BtrFS is used and is started partition at sector 63.

Sorry for my bad english...

affects: grub → installation-report
Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in installation-report (Ubuntu):
status: New → Confirmed
affects: installation-report → installation-report (Ubuntu)
Changed in installation-report (Ubuntu):
status: New → Confirmed
no longer affects: installation-report (Ubuntu)
Revision history for this message
D3m0n1q_733rz (cryo-rebirth) wrote :

Sorry, I did get a little confused over the sectors thing. The 63rd LBA sector is the starting point for the 0MB boundary of partitions usually. I wasn't thinking straight. I'm used to using the MB settings for partitioning instead. Anyhow, the normal Grub2 install for any partition that's not BtrFS can fit into the MBR area with no problem normally. However, a larger area is required if you're going to use a BtrFS partition since the data on how to read such a partition in Grub2 makes the boot program a bit larger than the MBR size. Recall that I said Core.img is particularly large. I'll have to check on my installation to see where I had to start it at later. But it is a problem that should be addressed or at least warned about in the installation.
As for the installation report, it should mention that Grub2 failed to install properly and should return to the partitioning step to address the issue. Highlighting the issue may also be beneficial to users. However, with newer hard drives, they utilize 4K sector sizes to increase capacity. This should be taken into account in case the 512b emulation ceases. It's known as Advanced Format and can be a pain in the rear if you're not already familiar with it. This will make 256 sectors equal to 1 MB instead.
Anyhow, I'll be sure to check on how I've managed to install BtrFS and where my partitions start and end so that I can toss together a sort of guideline for failsafes.

Phillip Susi (psusi)
Changed in ubiquity (Ubuntu):
status: New → Triaged
importance: Undecided → Wishlist
Revision history for this message
Chris Bainbridge (chris-bainbridge) wrote :

I hit this bug in a different way with 14.04:

Use fdisk to create a new mbr and single partition on a USB-connected drive
mkfs.btrfs the partition
Run installer and select the partition as root, and drive device as grub target

After grub fails the installer exits. fdisk reports the partition starts at 2048. gdisk reports that the drive has a valid MBR and valid GPT main header but corrupt GPT backup. At no point did I format the drive as GPT - I wonder if ubiquity automatically adds a GPT header causing the subsequent grub install with btrfs to fail?

Revision history for this message
Phillip Susi (psusi) wrote :

Your issue is unrelated to this bug Chris. You must have used GPT on the disk in the past, and used fdisk to write a new MBR, leaving parts of the GPT behind.

Revision history for this message
Ubuntu QA Website (ubuntuqa) wrote :

This bug has been reported on the Ubuntu ISO testing tracker.

A list of all reports related to this bug can be found here:
http://iso.qa.ubuntu.com/qatracker/reports/bugs/1059827

tags: added: iso-testing
amadeus (mesesan56)
Changed in grub2 (Ubuntu):
status: Triaged → Fix Released
Revision history for this message
Phillip Susi (psusi) wrote :

Please don't mark bugs as fixed unless you actually fixed it.

Changed in grub2 (Ubuntu):
status: Fix Released → Triaged
Revision history for this message
Javier Segura (segurajj) wrote :

I finally upgraded my 12.04LTS server to 14.04LTS last week, and I really regret having done this. I am suprised this bug is considered as unimportant, when it completelly spoils the booting system. Nothing seems to work.

Revision history for this message
Arie Skliarouk (skliarie) wrote :

@Javier: Take a bigger drive, format it with first partition starting at sector 2048 and transfer partitions content. Then you will have enough space to fit GRUB bootloader into.

Revision history for this message
Javier Segura (segurajj) wrote :

@Arie: thank you, but I hoped that a software issue could be resolved without the need of
additional hardware (I don't have a bigger drive right now). But I can survive: I can boot
from a live USB with Super Grub2. Anyway, I think that such gross errors should not
take place in an upgrade. This makes me wonder whether Ubuntu is a good choice, particularly
when this type of bugs are not considered as important.

Revision history for this message
Arie Skliarouk (skliarie) wrote :

@Javier: I don't think it is a bug per se. And definitely not ubuntu related. GRUB2 is complex piece of sotfware that keeps developing. I don't quite understand how they managed to put ext2 mounting and kernel+initrd loading into mere 31744 bytes back then. You aren't complaining you can't install today's ubuntu onto i386 processor, aren't you?

Revision history for this message
Javier Segura (segurajj) wrote :

No, I am complaining about the fact that I had a perfectly working ubuntu server which stopped booting after upgrading. That is a bad thing for an operating system. This should not happen.

Phillip Susi (psusi)
no longer affects: ubiquity (Ubuntu)
Revision history for this message
Matti Viljanen (direc85) wrote :

This still seems to exit. I am trying to install Ubuntu Server 18.10 on a Intel C202 based server, and although the installer sees and activates the MDADM it finds, installing the boot loader simply fails.

Revision history for this message
Eugene Crosser (crosser) wrote :

Regression after upgrade disco -> eoan, booted from btrfs disks. Worked normally before upgrade, cannot install grub after upgrade. Ended up with non-bootable system, had to plug an old disk exclusively to put /boot there.

Revision history for this message
steve77 (funmaker-11) wrote :

Had to reinstall Kubuntu 19.10. Used BTRFS for root on a 1TB SSD. Grub-install fails. I think this bug is regressed.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.