Installer crashed when trying to partition 4k/4k sector hard disks

[Kent Baxley]

partman_server crashes on Ubuntu 12.10 server for amd64 when trying to partition an Advanced Format (4k / 4k sector) hard disk.

Disks that utilize 4k sectors with 512 emulation work fine. The installer has no problems with those.

Steps to Reproduce:
1) Begin the installation in EFI mode.
2) At the disk partitioning stage, select "Guided use Entire Disk" or "Manual"
3) Review and accept the proposed partitioning scheme and write changes to disk.

Actual Results:
The system will attempt to create a the EFIboot partition. At this stage the installer appears to hang. Pressing Ctrl+AltF4 reveals that parted_server has crashed with the following message:

parted_server[13865]: segfault at 22fa000 ip 00007fa6eb74bee7 sp 00007fffec3ab500 error 4 in libparted.so.0.0.1[7fa6eb735000+64000]

Expected results:
Installation should carry on without a crash.

[Colin Watson (cjwatson)]

I'm unlikely to be able to debug this without a traceback. Does the same crash by any chance happen in a desktop installation? If so then it might be possible to get a core dump; if we're extra-lucky apport might be able to process it, but even without that the core dump or /var/crash/ file would probably be useful.

[Kent Baxley (kentb)]

Thanks, Colin. I'll try out a desktop installation and see if I'm able to get anything more useful for you.

[Kent Baxley (kentb)]

Attaching the crash file from /var/crash/ . The desktop installation failed in the same way (i.e. installer appeared to get 'stuck' when trying to create filesystems on 4k/4k disks).

[Colin Watson (cjwatson)]

#0 __memcpy_ssse3_back () at ../sysdeps/x86_64/multiarch/memcpy-ssse3-back.S:1517
No locals.
#1 0x00007fd654bfd197 in memcpy (__len=3108864, __src=0x64ee40, __dest=<optimized out>) at /usr/include/x86_64-linux-gnu/bits/string3.h:52
No locals.
#2 linux_write (dev=0x649380, buffer=0x64ee40, start=288, count=<optimized out>) at ../../libparted/arch/linux.c:1927
        arch_specific = 0x649dd0
        ex_status = <optimized out>
        diobuf = 0x7fd653476000
        diobuf_start = <optimized out>
        __PRETTY_FUNCTION__ = "linux_write"
        write_length = 3108864
#3 0x00007fd654bfab60 in ped_geometry_write (geom=<optimized out>, buffer=<optimized out>, offset=32, count=<optimized out>) at ../../libparted/cs/geom.c:392
        exception_status = <optimized out>
        real_start = <optimized out>
        __PRETTY_FUNCTION__ = "ped_geometry_write"
#4 0x00007fd654c0c3a9 in fat_table_write (ft=ft@entry=0x649d20, fs=fs@entry=0x64e440, table_num=table_num@entry=0) at ../../../../libparted/fs/fat/table.c:159
        fs_info = <optimized out>
        __PRETTY_FUNCTION__ = "fat_table_write"
#5 0x00007fd654c0c432 in fat_table_write_all (ft=0x649d20, fs=fs@entry=0x64e440) at ../../../../libparted/fs/fat/table.c:177
        fs_info = 0x64a540
        i = <optimized out>
#6 0x00007fd654c0e9cf in fat_create (geom=0x64b2b8, fat_type=<optimized out>, timer=<optimized out>) at ../../../../libparted/fs/fat/fat.c:393
        fs = 0x64e440
        fs_info = 0x64a540
        table_size = <optimized out>
#7 0x00007fd654bf4f92 in ped_file_system_create (geom=0x64b2b8, type=0x7fd654e5bb40 <fat32_type>, timer=0x64e400) at ../../libparted/filesys.c:534
        fs = <optimized out>
        __PRETTY_FUNCTION__ = "ped_file_system_create"
#8 0x0000000000403983 in ?? ()
No symbol table info available.
#9 0x000000000040ba85 in ?? ()
No symbol table info available.
#10 0x000000000040f3b7 in ?? ()
No symbol table info available.
#11 0x0000000000402b71 in ?? ()
No symbol table info available.
#12 0x00007fd65484676d in __libc_start_main (main=0x402990, argc=1, ubp_av=0x7fffb0cbc8a8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffb0cbc898) at libc-start.c:226
        result = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {0, -8111426171978694122, 4205428, 140736159533216, 0, 0, 8111279010564489750, 8125274207676744214}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x40f5c0,
              0x7fffb0cbc8a8}, data = {prev = 0x0, cleanup = 0x0, canceltype = 4257216}}}
        not_first_call = <optimized out>
#13 0x0000000000402b9d in ?? ()
No symbol table info available.
#14 0x00007fffb0cbc898 in ?? ()
No symbol table info available.
#15 0x000000000000001c in ?? ()
No symbol table info available.
#16 0x0000000000000001 in ?? ()
No symbol table info available.
#17 0x00007fffb0cbdc6b in ?? ()
No symbol table info available.
#18 0x0000000000000000 in ?? ()
No symbol table info available.

[Colin Watson (cjwatson)]

Ah. The problem is that libparted doesn't have 4K sector support for FAT; and isn't going to get it because most filesystem creation code has been removed upstream. We need to use something like mkfs.msdos instead.

[Colin Watson (cjwatson)]

This is also going to require creating a udeb from dosfstools, so that mkfs.msdos is available.

[Launchpad Janitor (janitor)]

This bug was fixed in the package dosfstools - 3.0.13-1ubuntu2

---------------
dosfstools (3.0.13-1ubuntu2) raring; urgency=low

  * Add dosfstools-udeb package, mainly so that we can use mkdosfs in d-i
    instead of deprecated and semi-broken libparted code (LP: #1065281).
 -- Colin Watson <email address hidden> Thu, 17 Jan 2013 20:09:57 +0000

[Launchpad Janitor (janitor)]

This bug was fixed in the package partman-basicfilesystems - 76ubuntu2

---------------
partman-basicfilesystems (76ubuntu2) raring; urgency=low

  * Use mkdosfs to create FAT filesystems, since libparted cannot handle
    doing that on non-512-sector disks (LP: #1065281).
 -- Colin Watson <email address hidden> Thu, 17 Jan 2013 23:05:24 +0000

[Adam Conrad (adconrad)]

Hello Kent, or anyone else affected,

Accepted dosfstools into precise-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/dosfstools/3.0.12-1ubuntu1.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Adam Conrad (adconrad)]

Hello Kent, or anyone else affected,

Accepted dosfstools into quantal-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/dosfstools/3.0.13-1ubuntu0.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Kent Baxley (kentb)]

Tested with today's 13.04 server image which contained the following:

/pool/main/d/dosfstools/dosfstools_3.0.13-1ubuntu2_amd64.deb
/pool/main/p/partman-basicfilesystems/partman-basicfilesystems_76ubuntu2_all.udeb

We ended up with the same symptoms and an identical segfault:

parted_server[14042]: segfault at 1f59000 ip 00007fde7758bea7 sp 00007ff564bf5c0 error 4 in libparted.so.0.0.1[7fde77575000+6400]

I'll try and grab another coredump. Do we need the actual udeb in the build for this to work?

[Kent Baxley (kentb)]

Turns out the image is getting respun later today to add the missing udeb. I'll try again this afternoon.

[Colin Watson (cjwatson)]

You need the actual udeb. That said, I question whether that's actually
an identical segfault because the code path that triggered the previous
one is no longer present.

[Kent Baxley (kentb)]

I just tested the re-spun image 20130118.1 and verified that the dosfsutils udeb is getting loaded.

Unfortunately, the installer still failed in the same spot with the identical segfault message.

Attaching dmesg, partman, and syslog logs.

[Kent Baxley (kentb)]

Based on Colin's comments, the latest crash (although it looks the same) most likely isn't identical since we are now seeing some additional messages logged prior to the crash that we didn't see before (sdb is the 4k sector disk in question):

Jan 18 17:07:30 kernel: [ 354.999903] FAT-fs (sdb1): logical sector size too small for device (logical sector size = 512)
Jan 18 17:07:30 kernel: [ 355.000367] isofs_fill_super: bread failed, dev=sdb1, iso_blknum=17, block=-2147483648
Jan 18 17:07:30 kernel: [ 355.015595] isofs_fill_super: bread failed, dev=sdb2, iso_blknum=17, block=-2147483648
Jan 18 17:07:30 kernel: [ 355.030905] isofs_fill_super: bread failed, dev=sdb3, iso_blknum=17, block=-2147483648
Jan 18 17:07:30 kernel: [ 355.072466] FAT-fs (sdb1): logical sector size too small for device (logical sector size = 512)
Jan 18 17:07:30 kernel: [ 355.072966] isofs_fill_super: bread failed, dev=sdb1, iso_blknum=17, block=-2147483648
Jan 18 17:07:30 kernel: [ 355.088324] isofs_fill_super: bread failed, dev=sdb2, iso_blknum=17, block=-2147483648
Jan 18 17:07:30 kernel: [ 355.102969] isofs_fill_super: bread failed, dev=sdb3, iso_blknum=17, block=-2147483648
Jan 18 17:07:35 kernel: [ 360.214278] Adding 134059004k swap on /dev/sdb3. Priority:-1 extents:1 across:134059004k

[Kent Baxley (kentb)]

Made some on-the-fly changes to 50format_basicfilesystems per Colin's request.

http://paste.ubuntu.com/1546211/

Attaching new syslog

[Launchpad Janitor (janitor)]

This bug was fixed in the package partman-efi - 25ubuntu4

---------------
partman-efi (25ubuntu4) raring; urgency=low

  * Depend on dosfstools-udeb for the changes in 25ubuntu3.
 -- Colin Watson <email address hidden> Fri, 18 Jan 2013 18:54:03 +0000

[Kent Baxley (kentb)]

Much better progress with the image from today (2012.01.21). The installer will no longer crash / hang when trying to partition an Advanced Format disk in EFI mode. The installation itself completed all the way to the end.

Another problem exists, however...

The problem now is that, upon reboot, we can't boot the operating system. There's no option in the EFI boot menu to boot Ubuntu. I thought that maybe we'd be able to add the entry in by hand (at least) to get the OS booted, however, there are no files on the disk to point the boot menu to.

[Kent Baxley (kentb)]

Attaching dmesg, syslog, and partman. These were captured at the end of the installation, but right before rebooting for the first time.

[Kent Baxley (kentb)]

Attaching dmesg, syslog, and partman. These were captured at the end of the installation, but right before rebooting for the first time.

[Colin Watson (cjwatson)]

Could I have a raw dump of the first megabyte of the disk, extracted using dd, as well as a raw dump of the first megabyte of /dev/sdb1? It looks as though perhaps libparted and efibootmgr are disagreeing over what makes a valid GPT, and I'm going to need to walk through it by hand.

[Kent Baxley (kentb)]

Attaching dumps. First MB of sdb, followed by first MB of sdb1

[Colin Watson (cjwatson)]

Ah, I've tracked this down to efibootmgr not supporting 4K-logical-sector disks. Fortunately, Fedora has a patch for this, which I'll backport.

Perhaps somebody at Dell could nudge Matt Domsch and/or Stuart Hayes about applying this patch, or similar, and issuing a new upstream release? You can find it here:

  http://pkgs.fedoraproject.org/cgit/efibootmgr.git/tree/efibootmgr-0.5.4-support-4k-sectors.patch

[Launchpad Janitor (janitor)]

This bug was fixed in the package efibootmgr - 0.5.4-3ubuntu2

---------------
efibootmgr (0.5.4-3ubuntu2) raring; urgency=low

  * Apply Fedora patch (efibootmgr-0.5.4-support-4k-sectors.patch) to
    support non-512-byte logical sectors (LP: #1065281).
 -- Colin Watson <email address hidden> Wed, 23 Jan 2013 12:41:28 +0000

[Jared Dominguez (jared-dominguez)]

Colin: I've subscribed Matt to this bug and will email him requesting that he incorporate that in a new upstream release.

[Jared Dominguez (jared-dominguez)]

Jordan Hargrave now owns efibootmgr. I've asked him about incorporating this patch.

[Colin Watson (cjwatson)]

Kent: Could you please retry with the raring server image I built this afternoon (20130123.1)? I believe it should be better now.

[jordanh (jordan-hargrave)]

The patches have already been in upstream efibootmgr for 2 years.
http://linux.dell.com/cgi-bin/gitweb/gitweb.cgi?p=efibootmgr.git;a=summary

I could cut a new release though.

[Kent Baxley (kentb)]

@ Colin,

Tested the 20130123.1 build. I got a little bit farther with this one, but, still couldn't boot the OS.

Yesterday, there wasn't even an "Ubuntu" boot entry in the EFI boot menu.

Today, I'm able to see the "ubuntu" entry in the menu, but, selecting it brings me to a "Boot Failed" screen.

I'll get another dump of the disk info first thing in the morning and send it over.

[Colin Watson (cjwatson)]

Jordan: Thanks, that would be helpful.

Kent: Sigh, not again. In that case, in addition to the previous dumps,
I'd like the installer logs (syslog and partman) again, and a full copy
of the EFI System Partition. Thanks.

[Kent Baxley (kentb)]

Attaching the logs and 1M dumps for sdb and sdb1.

The efi system partition copy is here, as it turned out to be rather large:

http://people.canonical.com/~kentb/efiboot.img.bz2

[Colin Watson (cjwatson)]

Thanks - I'll investigate that. I should have asked earlier, but could I also get a screenshot of the "Boot Failed" screen? I'd like to get a sense of which layer in the stack is emitting it.

[Kent Baxley (kentb)]

I'm not close to the server right now, but, the "Boot Failed" screen occurs just after the server runs through its POST routine.

I'll try to grab a photo of it and post it tomorrow, but, it basically says:

Boot Failed!

This system is in UEFI mode...(I can't remember the rest)

And then I get the options to press F1 to retry the bootup, F11 to enter the EFI boot options menu or press another function key to enter system setup.

[Jared Dominguez (jared-dominguez)]

Colin: Here's a screenshot.

[Brian Murray (brian-murray)]

Hello Kent, or anyone else affected,

Accepted partman-basicfilesystems into quantal-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/partman-basicfilesystems/74ubuntu1.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Brian Murray (brian-murray)]

Hello Kent, or anyone else affected,

Accepted partman-efi into quantal-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/partman-efi/25ubuntu2.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[James M. Leddy (jm-leddy)]

I've talked with Colin about this bug, and he assured me that he has not forgotten about it. We do have a machine with remote access set up so that this can be tested.

[Chris Halse Rogers (raof)]

Hello Kent, or anyone else affected,

Accepted partman-basicfilesystems into precise-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/partman-basicfilesystems/71ubuntu3.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Chris Halse Rogers (raof)]

Hello Kent, or anyone else affected,

Accepted partman-efi into precise-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/partman-efi/25ubuntu1.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Kent Baxley (kentb)]

I've had a chance to revisit this today.

Part of the problem seems to be related to how the installer is seeing the 4k/4k disk.

The 4k/4k sector disk is 1TB in size, however, the partitioner only sees the disk as 125GB. The installer will partition this space and the installation will continue, but, we end up with an un-bootable machine. dmesg and syslog, however, see the full 1TB of space for the disk.

If I run a df -h at the console during install time, we also see the correct sizes for the partitions (at least they seem to be):

/dev/sdb1 761.7M 32.0K 761.7M 0% /target/boot/efi
/dev/sdb2 73.2G 658.6M 68.8G 1% /target
/dev/sdb4 807.9G 72.2M 766.8G 0% /target/home

Also running parted in rescue mode, here's what we end up with:

Model: SEAGATE ST91000640SS (scsi)
Disk /dev/sdb: 1000GB
Sector size (logical/physical): 4096B/4096B
Partition Table: gpt

Number Start End SIze File System Name Flags
1 1049KB 800MB 799MB boot
2 800MB 80.8GB 80.0GB
3 80.8GB 119GB 38.0GB
4 119GB 1000GB 881GB

[James M. Leddy (jm-leddy)]

Sounds like sector size is still hardcoded in the installer code somewhere.

[Colin Watson (cjwatson)]

I've managed to find a qemu invocation that lets me reproduce this:

  qemu-system-x86_64 -enable-kvm -monitor stdio -m 1024 -cdrom raring-mini-amd64.iso -global scsi-disk.physical_block_size=4096 -global scsi-disk.logical_block_size=4096 -drive file=t.img,if=scsi,index=0,media=disk

This should make it much easier for me to attack this kind of problem.

[Colin Watson (cjwatson)]

This is probably a bit better, since qemu doesn't like booting over emulated SCSI:

  qemu-system-x86_64 -enable-kvm -monitor stdio -m 1024 -bios /usr/share/ovmf/OVMF.fd -cdrom raring-mini-amd64.iso -global ide-drive.physical_block_size=4096 -global ide-drive.logical_block_size=4096 -drive file=t.img,if=ide,index=0,media=disk

So. I can certainly see major bugs in partman-base; it assumes 512-byte sectors throughout. I've changed that to use the device's logical sector size, and uploaded partman-base 163ubuntu2 to raring. (Best test using a server image; desktop won't get this until the next ubiquity upload.) This certainly makes the partitioner behave more reasonably in my qemu-based tests.

However, I still haven't been able to get the resulting system to boot. This could be because qemu/OVMF are weird about booting between them (which they are), or it could be something to do with GRUB not understanding non-512-byte logical sectors entirely correctly, or it could be something else. If somebody's in a position to quickly confirm the behaviour of tomorrow's daily build on real hardware, that would be great.

[Kent Baxley (kentb)]

Thanks for the info, Colin. I'll go into the lab early and test the image first thing in the morning.

[Launchpad Janitor (janitor)]

This bug was fixed in the package partman-base - 163ubuntu2

---------------
partman-base (163ubuntu2) raring; urgency=low

  * Use the device's logical sector size throughout rather than
    PED_SECTOR_SIZE_DEFAULT (LP: #1065281).
 -- Colin Watson <email address hidden> Thu, 28 Mar 2013 18:31:11 +0000

[Kent Baxley (kentb)]

Hi Colin,

I've so far been unable to test this as it seems there's no March 29th build for ubuntu-server in the dailies for 13.04. I've been trying to get some help on IRC but to no avail. If I can't get any help, I'll have to wait until a build shows up on cdimage.ubuntu.com.

[Kent Baxley (kentb)]

I was able to get a current build downloaded today (thanks, Colin).

The fixes to partman-base have solved the disk size calculation issues, but, we're still unable to boot the OS when the installation finishes.

I noticed that there is no "ubuntu" boot entry in the EFI boot menu on this server, which is part of the problem. I'm also unable to add any boot entries for the OS because the hard disk drive no longer shows up in the EFI boot entry, either.

I'll be glad to submit whichever logs are needed from this latest attempt.

[Colin Watson (cjwatson)]

Ah well. I had a suspicion this would continue to be only part of the
problem. :-(

/var/log/installer/syslog should show me whether the installer *thinks*
it added an "ubuntu" boot entry. I expect that it thinks it did, and
that this is related to the hard disk no longer showing up, which I'll
have to investigate after Easter ...

[Kent Baxley (kentb)]

Colin,

Here's the latest /var/log/installer/syslog from a failed attempt today. The 4K/4K disk in question is seen as /dev/sdb (1TB Drive).

[Kent Baxley (kentb)]

Just putting this in for reference. 13.04 GA Server Edition also doesn't boot the OS once the installation completes. Symptoms are the same as before (no ubuntu entry from which to boot in the UEFI boot menu).

[Kent Baxley (kentb)]

Wondering if these bugs have anything to do with it:

https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1167622
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1173423

[Kent Baxley (kentb)]

Aha!: The log from the 8th shows this:

Apr 8 19:32:43 kernel: [ 422.245831] efivars: set_variable() failed: status=8000000000000009

[Kent Baxley (kentb)]

Tried 13.10 Server daily build and added the "efi_no_storage_paranoia=1" parameter from here:

https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/commit/arch/x86/platform/efi/efi.c?id=8c58bf3eec3b8fc8162fe557e9361891c20758f2

With this, I got my ubuntu boot menu entry back for EFI, but, I ended up at the same place I did in comment #42.

Attaching a /var/log/installer/syslog from this latest attempt.

[Kent Baxley (kentb)]

Attempting to boot after an installation of 13.10 even without the command line entry above results in the same situation. Just to make sure, I'll try the drive in another system.

[Kent Baxley (kentb)]

I moved the drive and sas controller to another system, just to make sure there wasn't something wrong with the machine itself that I was testing on.

I got the same results as before once the installer finished up and the server tried to boot again. I definitely have an 'ubuntu' entry in my efi boot menu, but we fail to boot the OS.

The EFI boot entry on both machines does seem to have a path to grubx64.efi. For example, in my latest attempt, this is what I see if I look at the device path for the ubuntu EFI menu entry:

Device Path:
HD(1, GPT, 3F0BA7A2-B80C-46BF-8E22-7559F9251FFD, 0x100, 0xE500)/\EFI\ubuntu\grubx64.efi

[Kent Baxley (kentb)]

I've tried the following Operating Systems in the last week, and here's the results:

Ubuntu 13.10 Server daily build - Installs the OS but does not boot with the same symptoms mentioned above.

OpenSuSE 12.3 - Similar to Ubuntu 13.10, the installation will properly recognize the 4k/4k drive but upon reboot, we cannot boot the OS. It does, however, display a different error. In the case of OpenSUSE, I see the following:

error:disk ',gpt4' not found
Entering rescue mode....
grub rescue>

If I run an ls in grub-rescue I see:
(hd0) (hd1) (hd1,gpt5) (hd1,gpt4) (hd1,gpt3) (hd1,gpt2) (hd1,gpt1) (cd0)

RHEL6.4 Installed AND booted the OS. RHEL6, I'm told, still uses a patched legacy grub, but, it does manage to boot up the OS.

I could not even begin to install Fedora 18 and 19. If I attempt to kick off an installation of these OSes, I am presented with a "Secure Boot not enabled" on the console and at that point it just sits there and doesn't move forward with the installation.

[Kent Baxley (kentb)]

In comparing /var/log/installer/syslog with a working installation using a 4k/512e disk and my non-working installation, I see that there are places where the installer has problems with 4096-byte sectors (not sure if this is a factor or not):

Jul 23 20:05:59 main-menu[764]: (process:13955): mount: mounting /dev/sdb3 on /tmp/tmp.v1Bp6T failed: Invalid argument
Jul 23 20:05:59 main-menu[764]: (process:13955): mount: mounting /dev/sdb3 on /tmp/tmp.yQNN5g failed: Invalid argument
Jul 23 20:05:59 main-menu[764]: (process:13955): ERROR: unsupported sector size 4096 on /dev/sdb.
Jul 23 20:06:00 main-menu[764]: DEBUG: resolver (ext2-modules): package doesn't exist (ignored)

Jul 23 20:17:09 main-menu[764]: (process:53636): Volume group "sdb" not found
Jul 23 20:17:09 main-menu[764]: (process:53636): Skipping volume group sdb
Jul 23 20:17:09 main-menu[764]: (process:53636): ERROR: unsupported sector size 4096 on /dev/sdb.
Jul 23 20:17:09 main-menu[764]: (process:53636): ERROR: unsupported sector size 4096 on /dev/sdb.
Jul 23 20:17:09 main-menu[764]: (process:53636): ERROR: unsupported sector size 4096 on /dev/sdb.
Jul 23 20:17:09 main-menu[764]: (process:53636): ERROR: unsupported sector size 4096 on /dev/sdb.

Jul 23 20:17:14 anna-install: Installing os-prober-udeb
Jul 23 20:17:14 finish-install: ERROR: unsupported sector size 4096 on /dev/sdb.
Jul 23 20:17:14 finish-install: ERROR: unsupported sector size 4096 on /dev/sdb.
Jul 23 20:17:14 finish-install: ERROR: unsupported sector size 4096 on /dev/sdb.
Jul 23 20:17:14 finish-install: ERROR: unsupported sector size 4096 on /dev/sdb.

Full log attached from failed 13.10 attempt on a 4k/4k disk.

[Kent Baxley (kentb)]

SLES 11 SP3 also appears to install fine, but, you have to manually point to the elilo.boot file on the hard drive in the EFI boot menu to get the OS to boot. So, SLES 11 SP3 and RHEL 6.4 are the only Operating Systems that I've been able to boot so far after installation. To be fair, though, RHEL6.4 uses an older, patched grub and SLES uses elilo. The distributions that use grub2 (ours and OpenSuSE) don't appear to boot the OS at all after installing.

[Kent Baxley (kentb)]

Here's a recap of what I've done recently:

I tested Ubuntu 13.10, OpenSuSE 12.3, RHEL 6.4 and SLES 11 SP3.

Of those 4, the only two that booted successfully after installation are RHEL 6.4 and SLES 11.
You do, however, have to point to the elilo.boot file by hand in the EFI boot menu to get SLES 11 to boot, but after that, it seems to work fine.

We need to take into account that RHEL6.4 is using a heavily patched legacy grub from what I understand and SLES is using elilo.

Ubuntu 13.10 and OpenSuSE 12.3 behaved very similarly in that the installer worked fine but once any attempt was made to boot the OS, we were not successful.

Trying to boot Ubuntu brings us to a boot failure screen on the server and we are asked to press F1 to try again, F2 for system setup, etc. We do have an 'ubuntu' entry in the EFI boot menu and if we press "F1" while that entry is highlighted, it shows that we do have a path to grubx64.efi...however, it isn't working for some reason.

In the case of OpenSuSE we are dropped to a grub prompt with grub complaining that ",gpt4" could not be found (although it seems to exist on the disk).

So it looks like the operating systems that are using a more modern version of grub are having the most trouble with booting advanced format disks.

[Kent Baxley (kentb)]

I'm wondering if we need a patch like this:

http://osdir.com/ml/scm-fedora-commits/2013-04/msg02625.html

[Kent Baxley (kentb)]

Here's where we are for 12.04.3:

I tested the latest daily precise images from cdimage.ubuntu.com. They are now including the 3.8-lts kernel bits. The immediate problem I noticed is that the disk size for the 4k drives is not detected properly. The drive is 1TB but the installer sees it as 125GB. This got fixed in the 13.04 installer, so, it still needs to be backported to 12.04 with probably a few other fixes. The installer will also go all the way through without crashing, but, as one may have guessed, we end up with a machine that won't boot. I'm also unsure if the grub version in 12.04 needs any additional patches.

For 13.10:
It looks like everything is working properly (disk size detection, etc) up until we reboot for the first time...which isn't successful. I'm wondering if we need the grub2 patches I mentioned earlier.

[Kent Baxley (kentb)]

Quick testing with RHEL6.4 and SLES 11 SP3 in BIOS mode:

Neither OS successfully works in BIOS mode. RHEL6 will complete the
initial installation but when attempting to boot from the HDD, I am
presented with a black screen and blinking cursor and that's it. This
is on par with how Fedora 19 behaved in BIOS mode as well as Ubuntu
13.10 / 13.04.

SLES 11 will also appear to work fine until it gets to the bootloader
installation stage and we fail with the attached photo. So for EFI
mode, SLES appears to use ELILO but defaults to grub for BIOS
installations.

RHEL6.4 and SLES11 SP3 both work fine with 4k/4k disks in EFI mode. RHEL6 uses a patched legacy grub while SLES uses elilo.

[Kent Baxley (kentb)]

It turns out that BIOS mode is not expected to work with these disks and that getting this to work with GPT and EFI will be the expected path going forward.

[Kent Baxley (kentb)]

We got some additional help from Stuart Hayes at Dell. After some analysis and testing, he got the 4k drive to boot by creating a larger fat32 partition than what we're creating right now:

The version of Ubuntu that you gave me (13.10) was creating a FAT32 filesystem (on the EFI system partition) that was only 100MB, but a FAT32 filesystem on a 4K drive has to be at least 133MB or so--see http://en.wikipedia.org/wiki/File_Allocation_Table#Size_limits (and adjust for the fact that they are assuming sectors are 512 bytes).

On my hard drive, I manually created a new FAT32 partition that was about 1GB, did "mkfs.vfat /dev/sda3" on it, and then mounted it and copied everything from the 100MB FAT partition onto my new 1GB FAT partition. When I rebooted, UEFI firmware did recognize my new partition as a valid FAT32 partition and would let me boot to it.

So the Ubuntu installer should just need to make sure the boot partition meets the requirements of the FAT32 filesystem (i.e., at least 133MB or so).

Stuart

[Kent Baxley (kentb)]

We also ran into another segfault with 13.10 when trying to auto-partition with LVM on these drives. I'll attach a crash file in a little while. Auto-partitioning without LVM works fine.

[Kent Baxley (kentb)]

For 12.04.3, we'll also need whatever fixes are necessary for the fat32 system partition plus these two fixes so that 4k drives are handled correctly. The partman-base fix should take care of the installer not properly detecting the size of the disk.

---------------
partman-base (163ubuntu2) raring; urgency=low

  * Use the device's logical sector size throughout rather than
    PED_SECTOR_SIZE_DEFAULT (LP: #1065281).
 -- Colin Watson <email address hidden> Thu, 28 Mar 2013 18:31:11 +0000

---------------
efibootmgr (0.5.4-3ubuntu2) raring; urgency=low

  * Apply Fedora patch (efibootmgr-0.5.4-support-4k-sectors.patch) to
    support non-512-byte logical sectors (LP: #1065281).
 -- Colin Watson <email address hidden> Wed, 23 Jan 2013 12:41:28 +0000

[Kent Baxley (kentb)]

Here is the specific commit for partman-base that needs to be ported to 12.04:

http://bazaar.launchpad.net/~ubuntu-core-dev/partman-base/ubuntu/revision/1377

For efibootmgr, it is this one:

http://bazaar.launchpad.net/~ubuntu-branches/ubuntu/raring/efibootmgr/raring-proposed/revision/17

Both appear to be pretty small and straightforward.

[Kent Baxley (kentb)]

Here is the crash file from 13.10 when selecting lvm in the partitioning phase. parted_server is what segfaults.

[Kent Baxley (kentb)]

Here's what parted sees from a successful RHEL6.4 install:

Number Start End Size File system Name Flags
1 1049KB 211MB 210MB boot
2 211MB 735MB 524MB
3 735MB 10000GB 999GB lvm

sector view is:

Number Start End Size File system Name Flags
1 256s 54155s 51200s boot
2 51456s 179455s 12800s
3 179456s 244190463s 244011008s lvm

[James M. Leddy (jm-leddy)]

I've merged some changes from the raring branches of efimgr and partman into my own bzr branch. I'm subscribing -sponsors in the hopes that we can get this in for 12.04.3, or 12.04.4 at the very latest. The odds are good that enterprise users that are actually using these disks are going to be on LTS.

@Kent, ping me on Monday so I can get you the binaries to test.

[James M. Leddy (jm-leddy)]

Should have mentioned, I've compile tested both of these changes

[Kent Baxley (kentb)]

@James, Thanks. I'll be ready to look at your changes at any time.

An additional note:

I was also able to repeat Stuart's work in a pure Ubuntu environment...there is one trick, though, and we might want to consider using it in the installer. Here's what I did:

1) Installed Ubuntu 13.10 and verified that it wouldn't boot.

2) Booted into a live desktop session for 13.10

3) Deleted my swap partition on /dev/sda3 and re-created a 1GB partition on /dev/sda3 via parted. I also toggled /dev/sda3 as bootable.

4) Ran mkfs.vfat -s1 /dev/sda3. We had to tell mkfs.vfat to use one sector per cluster or the format wouldn't work. Apparently it isn't smart enough to do that on its own, or the tool is still using 512 hard-coded for sector size where it determines how many sectors to use per cluster (which probably isn't surprising).

5) Once the fat filesystem was laid down, I mounted up /dev/sda1 and /dev/sda3 and copied the EFI directory and its contents from /dev/sda1 to /dev/sda3.

6) I rebooted into the EFI menu and the EFI firmware was finally able to see grubx64.efi by drilling down into the 'boot from file' option in the EFI menu. We've never been able to do this before.

I'm wondering if in partman-efi, where we had to start using mkdosfs to work around some limitations of libparted, we can add the "-s 1" flag to this code, along with making a bigger partition to make all of this work:

from commit.d/format_efi in parman-efi:

 59 log_sector_size="$(blockdev --getss "$(cat device)")"
 60 if log-output -t partman --pass-stdout \
 61 mkdosfs -F "${new_efi_fs#fat}" \
 62 -S "$log_sector_size" \
 63 "$device" >/dev/null; then
 64 sync
 65 status=OK

[Kent Baxley (kentb)]

Another update from Stuart regarding mkdosfs/mkfs.vfat that may be relevant and it may lend some ammo to possibly adding the "-s 1" flag to the mkdosfs command mentioned above?

"So it looks like mkdosfs / mkfs.vfat is calculating an initial guess for "bs.cluster_size" (the internal variable name) based on the size of the disk, just assuming 512 bytes per sector.

The assumption is implicit, because they are using numbers from a table in a Microsoft document, and the number "512" (or any sort of sectors per track) doesn't show up in the code here, so that is probably why it got missed. They clearly take into account sectors per track elsewhere in the code.

The code will actually increase the number of sectors in a cluster until it fits nicely, but it will not decrease it.

I'll try to get a patch in upstream to fix this. For now, though, it should work ok to just use "-s 1", as far as I can tell."

[Kent Baxley (kentb)]

I ran some experiments today and this so far seems to be the 'sweet spot' with regard to the EFI system partition:

1) Before installing, I created a 133MB partition on my gpt disk using parted, type FAT32 and bootable.
2) I formatted the partition using "mkfs.vfat -s1 /dev/sda1"
3) Booted to the 13.10 server installer.
4) At partitioning time, I chose 'manual', and did not touch the EFI system partition (things looked to be recognized properly by the installer, anyway, in that regard). I created a root and swap partition on the disk with the remaining space.
5) Proceeded with the installation.
6) Upon reboot, EFI (for the first time ever) was able to attempt booting automatically from the system partition on this type of hard drive. I was dropped to a grub prompt, but, this is a huge improvement over where we were earlier.

I also repeated the above using what we're currently setting the EFI partition to (which is around 100MB) and the *only* way I could get the system to try and boot was to manually drill down to the grubx64.efi / shimx64.efi via the 'Boot from file' option in the EFI menu.

[Kent Baxley (kentb)]

The above experiment was also performed with today's 12.04.3 builds and patched versions of efibootmgr and partman-base. I can also automatically boot to a grub prompt with 12.04.3 using a 133MB EFI partition formatted with mkfs.vfat -s 1.

[Kent Baxley (kentb)]

I'll also be trying this again with 266MB for the EFI system partition. The reason is because Stuart earlier mentioned that the minimum “proper” FAT32 partition (at least 64K clusters) is ~133MB—it is actually ~266MB. He multiplied incorrectly. It
is ~33MB for 512-byte sectors, so it is ~33MB * 8 (~266MB) for 4096-byte sectors. I'll see if anything is any better with 266MB or if we're about the same.

[Kent Baxley (kentb)]

I can get the system to successfully boot from the grub prompt by running:

grub> linux (hd0,gpt2)/vmlinuz root=/dev/sda2
grub> initrd (hd0,gpt2)/initrd.img
grub> boot

This was done during my experiments with creating a 266MB fat32 fs on the EFI system partition. What I'd like to try next is a slightly larger partition as well as a 133MB one to see if I am successful there as well.

[Kent Baxley (kentb)]

For 13.10, so far, this seems to be the way to get these disks to boot:

1) Prior to installation, carve out a partition of at least 133MB in parted using a GPT partition table. Toggle the boot flag.
2) Format the disk using 'mkdosfs -s 1 /dev/sda1'
3) Start up the server installer.
4) Select manual partitioning.
5) Don't touch the EFI system partition, but, go ahead and set up partitions for "/" and swap.
6) Proceed with the installation and reboot when done.
7) The system should drop to a grub prompt. At the grub prompt, use the commands above to boot the operating system.

I'll also try this on 12.04.3 using the patched versions of partman-base and partman-efi.

[Kent Baxley (kentb)]

For 12.04.3, I can also successfully boot using the steps above, along with feeding the correct commands to grub at the grub prompt. The only difference is that I have to apply the patches to partman-base, partman-utils, and efibootmgr. Those fixes are in 13.04 and above, but, not in precise (yet).

[Steve Langasek (vorlon)]

Reading through the bug log, it appears that the fixes currently staged in precise-proposed are all necessary, but not sufficient, for getting the system to install and boot with 4k disks. Since the validation being done to date for 12.04.3 is with images that include these packages, and since 12.04.3 is meant to happen imminently, I believe the right thing to do here is to go ahead with publishing what we have to date.

[Launchpad Janitor (janitor)]

This bug was fixed in the package dosfstools - 3.0.12-1ubuntu1.1

---------------
dosfstools (3.0.12-1ubuntu1.1) precise; urgency=low

  * Add dosfstools-udeb package, mainly so that we can use mkdosfs in d-i
    instead of deprecated and semi-broken libparted code (LP: #1065281).
 -- Colin Watson <email address hidden> Fri, 18 Jan 2013 13:08:45 +0000

[Launchpad Janitor (janitor)]

This bug was fixed in the package partman-basicfilesystems - 71ubuntu3.1

---------------
partman-basicfilesystems (71ubuntu3.1) precise; urgency=low

  * Use mkdosfs to create FAT filesystems, since libparted cannot handle
    doing that on non-512-sector disks (LP: #1065281).
 -- Colin Watson <email address hidden> Wed, 23 Jan 2013 12:53:16 +0000

[Launchpad Janitor (janitor)]

This bug was fixed in the package partman-efi - 25ubuntu1.1

---------------
partman-efi (25ubuntu1.1) precise; urgency=low

  * Use mkdosfs to create FAT filesystems, since libparted cannot handle
    doing that on non-512-sector disks (LP: #1065281).
 -- Colin Watson <email address hidden> Wed, 23 Jan 2013 13:23:10 +0000

[Launchpad Janitor (janitor)]

This bug was fixed in the package dosfstools - 3.0.13-1ubuntu0.1

---------------
dosfstools (3.0.13-1ubuntu0.1) quantal; urgency=low

  * Add dosfstools-udeb package, mainly so that we can use mkdosfs in d-i
    instead of deprecated and semi-broken libparted code (LP: #1065281).
 -- Colin Watson <email address hidden> Fri, 18 Jan 2013 14:06:47 +0000

[Launchpad Janitor (janitor)]

This bug was fixed in the package partman-basicfilesystems - 74ubuntu1.1

---------------
partman-basicfilesystems (74ubuntu1.1) quantal; urgency=low

  * Use mkdosfs to create FAT filesystems, since libparted cannot handle
    doing that on non-512-sector disks (LP: #1065281).
 -- Colin Watson <email address hidden> Wed, 23 Jan 2013 13:07:11 +0000

[Launchpad Janitor (janitor)]

This bug was fixed in the package partman-efi - 25ubuntu2.1

---------------
partman-efi (25ubuntu2.1) quantal; urgency=low

  * Use mkdosfs to create FAT filesystems, since libparted cannot handle
    doing that on non-512-sector disks (LP: #1065281).
 -- Colin Watson <email address hidden> Wed, 23 Jan 2013 13:25:11 +0000

[Launchpad Janitor (janitor)]

This bug was fixed in the package partman-auto - 107ubuntu2

---------------
partman-auto (107ubuntu2) saucy; urgency=low

  * Bump EFI partition size range to 512-1024MB, in line with Debian; 100MB
    is too small on a disk with 4KiB logical sectors (LP: #1065281).
 -- Colin Watson <email address hidden> Thu, 12 Sep 2013 13:49:08 +0100

[Kent Baxley (kentb)]

With the latest fixes on 13.10 I still fail to boot the OS, even with a larger FAT32 EFI partition in place. In other words the server cannot find anything to boot from and asks me to press F1 to try again, F11 to return to the EFI boot menu, etc.

**HOWEVER**

If I restart the installation and edit "/lib/partman/commit.d/50format_efi" prior the partitioning phase and adjust this line:

mkdosfs -F "${new_efi_fs#fat}" \

to look like this:

mkdosfs -s 1 -F "${new_efi_fs#fat}" \

We are dropped to a grub prompt after installation and reboot...so, I think how the fat partition getting created is one factor in this.

From the grub prompt, I can type the following commands and boot the OS:

grub> linux (hd0,gpt2)/vmlinuz root=/dev/sda2
grub> initrd (hd0,gpt2)/initrd.img
grub> boot

Also, I noticed in the logs if we use the original mkdosfs command *without* the -s 1 option a warning is logged:

WARNING: Not enough clusters for a 32 bit FAT!

Whereas I don't see this if I use -s 1.

I'll attach logs from both an install with the way things currently are and another set with the mkdosfs modifications in place.

[Kent Baxley (kentb)]

Logs from an installation where 50format_efi was modified with the "-s 1" option for mkdosfs. This allowed the server to drop us to a grub prompt and boot the OS by hand with a few grub commands. I'm guessing grub2 might also need some work. Possibly this patch would help?

http://osdir.com/ml/scm-fedora-commits/2013-04/msg02625.html

[Kent Baxley (kentb)]

Logs from an unmodified 50format_efi.

Note the warning in the syslog:

Sep 13 19:28:14 partman: WARNING: Not enough clusters for a 32 bit FAT!

...it appears to try and create the filesystem anyway, but, apparently it's in a state that the EFI firmware on the server can't understand when it comes time to try and boot the hard drive (just a guess)?

[Colin Watson (cjwatson)]

Right, I knew I hadn't quite got all the way. The partman-efi changes for "mkdosfs -s 1" (now that I've understood why that's needed) are on their way in now, and I'll start looking into what's wrong with GRUB.

[Launchpad Janitor (janitor)]

This bug was fixed in the package partman-efi - 25ubuntu5

---------------
partman-efi (25ubuntu5) saucy; urgency=low

  * Pass "-s 1" to mkdosfs if the logical sector size is not equal to 512
    bytes, since mkdosfs has trouble with cluster calculations otherwise
    (LP: #1065281).
 -- Colin Watson <email address hidden> Wed, 18 Sep 2013 13:54:58 +0100

[Colin Watson (cjwatson)]

I'm necessarily flying blind, but that Fedora patch looks like an excellent candidate: it would affect GRUB's detection of where it was booted from, which could cause it to fail to find its configuration file. I'd prefer to backport the commit that ended up upstream (r4795) rather than the one linked to above, but that's OK.

[Launchpad Janitor (janitor)]

This bug was fixed in the package grub2 - 2.00-19ubuntu1

---------------
grub2 (2.00-19ubuntu1) saucy; urgency=low

  * Resynchronise with Debian. Remaining changes:
    - Default to hiding the menu; holding down Shift at boot will show it.
    - Add crashkernel option.
    - Bypass menu unless other OSes are installed or Shift is pressed.
    - Show the boot menu if the previous boot failed.
    - Check hardware support before using gfxpayload=keep.
    - Set vt.handoff=7 for smooth handoff to kernel graphical mode.
    - In recovery mode, add nomodeset to the Linux kernel arguments, and
      remove the 'set gfxpayload=keep' command.
    - Handle probing striped DM-RAID devices.

grub2 (2.00-19) unstable; urgency=low

  [ Colin Watson ]
  * Merge from Ubuntu:
    - debian/build-efi-images: Where possible, make use of the device path
      derived from the EFI Loaded Image Protocol to compute the prefix
      (LP: #1097570).
    - debian/build-efi-images: Add a netboot image target to our set of
      prebuilt EFI images (thanks, Steve Langasek).
  * Backport from upstream:
    - Handle partitions on non-512B EFI disks (LP: #1065281).

  [ Phillip Susi ]
  * restore_mkdevicemap.patch: Fix dmraid uuid check to look for "DMRAID-"
    anywhere instead of only at the start, since kpartx prefixes it with
    "partN-" (LP: #1183915).
 -- Colin Watson <email address hidden> Wed, 18 Sep 2013 21:20:28 +0100

[Brian Murray (brian-murray)]

Hello Kent, or anyone else affected,

Accepted partman-auto into precise-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/partman-auto/101ubuntu2.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Brian Murray (brian-murray)]

Hello Kent, or anyone else affected,

Accepted partman-efi into precise-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/partman-efi/25ubuntu1.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Kent Baxley (kentb)]

For 13.10, I confirmed that the new grub2 packages allow us to fully boot into the operating system (yay!!).

This is true for both LVM and non-LVM auto-partitioning options.

Thanks for all of your help, Colin.

I'll also test what fixes are present for 12.04.

[Kent Baxley (kentb)]

I tried today's build of 12.04, which seemed to have the two fixes committed yesterday.

The installer completed OK, but, it appears that the EFI boot entry somehow didn't get set..In other words, the server couldn't find anything to boot from and, upon further inspection, there was no 'ubuntu' entry available in the EFI boot menu.

I ran out of time today to dive into this, but, I'll try again on Monday and gather up logs if the problem persists.

[Colin Watson (cjwatson)]

It probably isn't worth retesting 12.04 until I've backported the grub2
change, at least.

[Brian Murray (brian-murray)]

Hello Kent, or anyone else affected,

Accepted grub2 into precise-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/grub2/1.99-21ubuntu3.11 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Colin Watson (cjwatson)]

grub2-signed (1.20) saucy; urgency=low

  * Rebuild against grub-efi-amd64 2.00-19ubuntu1.

 -- Colin Watson <email address hidden> Thu, 19 Sep 2013 11:01:28 +0100

[Kent Baxley (kentb)]

Tested today's 12.04 builds that contain what appears to be the bulk of the grub2 version 1.99-21ubuntu3.11 packages.

Install completes without crashing, but, I still end up with no 'ubuntu' entry in the EFI boot menu. I can navigate to the grubx64.efi file on the hard disk via EFI boot menu, though and attempt to boot from file that way.

I'm then dropped to a grub command prompt and from there I can boot the OS.

This particular partitioning scheme (Use entire disk without LVM) left me with a 5GB EFI system partition and a 137GB Swap partition once the OS booted up.

Looks like there are some other fixes being queued up so I'll hold off further testing until I see those go out.

[Kent Baxley (kentb)]

I tried today's build (Oct 4th) and realized that partman-base hasn't had its fixes ported to precise yet (whoops), which means the 1TB disk is still only showing up as 125GB. The efibootmgr stuff also needs to be ported (I think).

I'll wait until those fixes get pushed into 12.04 before continuing further testing.

[Launchpad Janitor (janitor)]

This bug was fixed in the package partman-auto - 101ubuntu2.1

---------------
partman-auto (101ubuntu2.1) precise; urgency=low

  * Bump EFI partition size range to 512-1024MB, in line with Debian; 100MB
    is too small on a disk with 4KiB logical sectors (LP: #1065281).
 -- Colin Watson <email address hidden> Thu, 12 Sep 2013 13:58:05 +0100

[Stéphane Graber (stgraber)]

Hello Kent, or anyone else affected,

Accepted partman-auto into precise-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/partman-auto/101ubuntu2.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Phillip Susi (psusi)]

Wait, what? Why does it need to be 512 MB now? I already thought that 100 MB was absurdly large.

[Stéphane Graber (stgraber)]

Hello Kent, or anyone else affected,

Accepted partman-base into precise-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/partman-base/153ubuntu6 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Colin Watson (cjwatson)]

Note that this is still missing the last currently-known piece, namely efibootmgr. I'll review James' branch for that by tomorrow at the latest.

[Stéphane Graber (stgraber)]

Hello Kent, or anyone else affected,

Accepted efibootmgr into precise-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/efibootmgr/0.5.4-2ubuntu1.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Colin Watson (cjwatson)]

Tomorrow's precise daily build should finally be worth testing for this.

[Phillip Susi (psusi)]

Collin, rather than waste more disk space why not fix the real bug in mkdosfs ( in that it fails to switch to fat16 at < 133 MB )?

[Colin Watson (cjwatson)]

Please note that my first name contains only one "l", not two.

I don't consider it important - 4k/4k sector hard disks used on UEFI
systems are not going to be so uncomfortably tiny that it makes much of
a difference in practice, and it's much more important to get this
high-priority bug fixed so that OEMs can deliver systems that actually
*boot* properly rather than going round again on the details. If you
want to get into this could you please take it to a separate bug?

[Phillip Susi (psusi)]

Oh, so the size increase only applies for 4k drives? I suppose that's ok then, I was just worried that half a gig was significant on a 16 gb ssd.

[Kent Baxley (kentb)]

Tried with today's Precise Server Build.

The good:
1) No installer crashes.
2) Disk is prooperly detected as 1TB in size
3) I have an ubuntu entry to boot from in the EFI boot menu

The bad:
Dropped to a grub prompt at boot time. I can, however, boot the OS manually by providing grub with the path to the vmlinuz and initrd on the disk

Attaching the full set of logs from /var/log/installer

[Kent Baxley (kentb)]

Link to logs: https://bugs.launchpad.net/ubuntu/+source/partman-basicfilesystems/+bug/1065281/+attachment/3912823/+files/installer-logs-11-19-13.tgz

[Colin Watson (cjwatson)]

Hello Kent, or anyone else affected,

Accepted ubiquity into precise-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/ubiquity/2.10.28 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Colin Watson (cjwatson)]

Aha, got it: the efidisk module is working but the FAT file system support isn't quite right, so it's just the EFI System Partition that GRUB has trouble reading. The missing commit is this one, which is in 2.00 thus explaining why this works in more recent releases:

  http://git.savannah.gnu.org/gitweb/?p=grub.git;a=commitdiff;h=daa59f47f6818d2a2c90526be5b2b38a11770dc1

Doing a full boot loader test is awkward due to the involvement of a signed image, but I've confirmed the bug with grub-fstest and grub-mount, and built versions of those tools with that patch which can successfully read all the files under /EFI/ubuntu/. I'll prepare a backport.

[Adam Conrad (adconrad)]

Hello Kent, or anyone else affected,

Accepted grub2 into precise-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/grub2/1.99-21ubuntu3.14 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Adam Conrad (adconrad)]

Hello Kent, or anyone else affected,

Accepted grub2-signed into precise-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/grub2-signed/1.9~ubuntu12.04.6 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Kent Baxley (kentb)]

With today's Precise Server image I can now boot the Operating System!

I do, however, see one seemingly minor error pop up as soon as I select Ubuntu in the gru menu:

error: efidisk write error

Press any key to continue....

The system will boot up after about 10 seconds.

I'm wondering if it is similar to what I filed against 14.04. Error message is a little different, but, the behavior is the same (i.e. the system will boot by itself approximately 10 seconds after the error message pops up):

https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1253443

[Colin Watson (cjwatson)]

We may need to deal with this at some point, but it can safely be
ignored for now; any writes will just be for the environment block,
which isn't critical functionality (and is already documented as missing
in some profiles).

[Launchpad Janitor (janitor)]

This bug was fixed in the package partman-efi - 25ubuntu1.2

---------------
partman-efi (25ubuntu1.2) precise; urgency=low

  * Pass "-s 1" to mkdosfs if the logical sector size is not equal to 512
    bytes, since mkdosfs has trouble with cluster calculations otherwise
    (LP: #1065281).
 -- Colin Watson <email address hidden> Wed, 18 Sep 2013 14:00:33 +0100

[Colin Watson (cjwatson)]

The verification of the Stable Release Update for partman-efi has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regresssions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package partman-base - 153ubuntu6

---------------
partman-base (153ubuntu6) precise-proposed; urgency=low

  * Use the device's logical sector size throughout rather than
    PED_SECTOR_SIZE_DEFAULT (LP: #1065281).
 -- James M Leddy <email address hidden> Fri, 08 Nov 2013 17:57:33 +0000

[Launchpad Janitor (janitor)]

This bug was fixed in the package efibootmgr - 0.5.4-2ubuntu1.1

---------------
efibootmgr (0.5.4-2ubuntu1.1) precise; urgency=low

  * Apply Fedora patch (efibootmgr-0.5.4-support-4k-sectors.patch) to
    support non-512-byte logical sectors (LP: #1065281).
 -- James M Leddy <email address hidden> Wed, 13 Nov 2013 15:59:21 +0000

[Launchpad Janitor (janitor)]

This bug was fixed in the package ubiquity - 2.10.28

---------------
ubiquity (2.10.28) precise; urgency=low

  * Automatic update of included source packages: partman-
    basicfilesystems 71ubuntu3.4. (LP: #978032)

ubiquity (2.10.27) precise; urgency=low

  * Automatic update of included source packages: netcfg 1.68ubuntu14.1
    (LP: #901700), partman-auto 101ubuntu2.2 (LP: #1197766, #1065281),
    partman-base 153ubuntu6 (LP: #1065281), partman-basicfilesystems
    71ubuntu3.3 (LP: #978032, #1215458), partman-btrfs 8ubuntu1.1 (LP:
    #978032), partman-efi 25ubuntu1.2 (LP: #1065281), partman-ext3
    67ubuntu1.1 (LP: #978032).
 -- Dmitrijs Ledkovs <email address hidden> Tue, 03 Dec 2013 18:04:05 +0000

[Launchpad Janitor (janitor)]

This bug was fixed in the package grub2 - 1.99-21ubuntu3.14

---------------
grub2 (1.99-21ubuntu3.14) precise; urgency=low

  * Handle FAT filesystems on non-512B disks (LP: #1065281).
  * Probe FusionIO devices (LP: #1237519).
  * On Linux, read partition start offsets from sysfs if possible
    (LP: #1237519).

grub2 (1.99-21ubuntu3.13) precise; urgency=low

  * Revamp hidden timeout handling by adding a new timeout_style environment
    variable and a corresponding GRUB_TIMEOUT_STYLE configuration key for
    grub-mkconfig. This controls hidden-timeout handling more simply than
    the previous arrangements, and pressing any hotkeys associated with menu
    entries during the hidden timeout will now boot the corresponding menu
    entry immediately (LP: #1178618). As part of merging this, radically
    simplify /etc/grub.d/30_os-prober; if it finds other OSes it can now
    just set timeout_style=menu and make sure the timeout is non-zero.
  * Fix mismerge of GRUB_RECOVERY_TITLE option in 1.99-21ubuntu3.12.

grub2 (1.99-21ubuntu3.12) precise; urgency=low

  * debian/build-efi-images: Where possible, make use of the device path
    derived from the EFI Loaded Image Protocol to compute the prefix
    (LP: #1097570).
  * Add GRUB_RECOVERY_TITLE option, to allow the controversial "recovery
    mode" text to be customised (LP: #1240360).
 -- Colin Watson <email address hidden> Thu, 05 Dec 2013 16:53:48 +0000

[Launchpad Janitor (janitor)]

This bug was fixed in the package grub2-signed - 1.9~ubuntu12.04.6

---------------
grub2-signed (1.9~ubuntu12.04.6) precise; urgency=low

  * Build against grub-efi-amd64 1.99-21ubuntu3.14 (LP: #1065281).
 -- Colin Watson <email address hidden> Fri, 06 Dec 2013 09:54:51 +0000

[Colin Watson (cjwatson)]

ubiquity got the necessary included source packages in trusty some time ago.

[Rolf Leggewie (r0lf)]

quantal has seen the end of its life and is no longer receiving any updates. Marking the quantal task for this ticket as "Won't Fix".