initramfs fails to mount ext4 root partition or provide user intervention (even when ext4 module is available in the image)

[Marques Johansson]

Binary package hint: initramfs-tools

The init scripts in initramfs do not mount an ext4 root partition, this leaves the user stuck at an (initram) prompt at a point that is too late for the user to correct matters. There are a few TODO notes in the local script that mention that there is not error handling when the FSTYPE detection or root mount fails. This should be addressed.

When dropped to the (initram) I was able to "modprobe ext4; mount /dev/sda1 /root", but attempting to run "exec init", resulted in a Kernel Panic.

The init local script had continued running past the root mount failure, so commands like these:
mount -n --bind /root/dev /dev/.static/dev
mount -n --move /dev /root/dev
mount -n --move /proc /root/proc
mount -n --move /sys /root/sys
had already been attempted before I could intervene and manually mount root. Running them myself, and trying to finish off the local script (which ends with something like "exec run-init .... </root/dev/console > /root/dev/console ... " resulted in a kernel panic that /root/dev/console could not be found.

I don't know why the ext4 mount was not successfully mounted, but I was able to work around this by adding a /etc/initramfs-tools/scripts/local-bottom/custom_mount_ext4 script:
#! /bin/sh
modprobe ext4
mount /dev/sda1 /root
mount /dev/disk/by-uuid/..... /root

I also copied this script to local-top. I am mounting it twice and in both locations because I'm not sure which 'mount' command worked or when the device file is available. The first attempt I made at this workaround was to do "modprobe ext4; mount /dev/sda1 /root" from local-top (see bug #309758). That gave the error that /dev/sda1 was not a device file - I suppose that is created by a script run later. It existed by the time I got the (initramfs) prompt.

[Marques Johansson (marques)]

I believe in one of the conditions where I tried to work from the (initram) prompt the error I received when trying to move on to the root partitions /sbin/init was that ~ "(null) was an invalid runlevel".

[Marques Johansson (marques)]

It would be nice if it detected that it did not mount /root and then gave you a notice about that and gave you a shell with the instruction to 'type exit once /root is mounted to continue the start-up process'

[Marques Johansson (marques)]

In my list of commands that I issued to behave like the "local" init script, I referred to /mnt a few times where I meant to say /root. I didn't make this error at the console.

[DiegoCG (diegocg)]

The problem is that fstype doesn't detect a ext4 filesystem. It detects it as ext3, and tries to mount the rootfs with "mount -t ext3...", which fails because ext3 can't mount a ext4 filesystem.

The problem is in scripts/local:

========================
get_fstype ()
{
        local FS FSTYPE FSSIZE RET
        FS="${1}"

        # vol_id has a more complete list of file systems,
        # but fstype is more robust
        eval $(fstype "${FS}" 2> /dev/null)
========================

fstype pass ext3 as filesystem there.

It turns out that the "unreliable" vol_id detects the ext4 properly, while fstype does not...

[feiy (eshangrao)]

i got the bug two:

ubuntu jaunty
kernel:2.28.4

[Martijn vdS (martijn)]

I also see this on my system

[Martijn vdS (martijn)]

Workaround: boot with the 'break=mount' option, mount the fs on /root yourself, and exit the shell. Booting should continue normally.

[Bowmore (bowmore)]

Confirm this bug.
I've tried the break=mount option too without success. I can mount an ext3 volume but still not an ext4.

$ mount /dev/sda1 /root
results in (obvious as ext3 defaults):
EXT3-fs: sda1: couldn't mount because of unsupported optional features (40)
mounting /dev/sda1 on /root failed: Invalid argument

$ mount /dev/sda1 -t ext4 /root
results in (sda1 is ext4):
mounting /dev/sda1 on /root failed: No such device

In my case I made a conversion ext3 to ext4 using this guide
http://kernelnewbies.org/Ext4
that is,
$ tune2fs -O extents,uninit_bg,dir_index /dev/sda1
$ fsck -pf /dev/sda1

uname -r
2.6.28-3-generic

So I guess it's up to what ext4 features you activate that makes the ext4 device mountable or not in the initramfs shell. However, I can boot up another Jaunty system an mount this "sda1" device there with success so the device itself is ok.

[Marques Johansson (marques)]

see Bug #309758 about adding the ext4 module to your initramfs image first. I gave some workarounds in there.. Basically, you have to get ext4 and it's dependent modules in the initramfs image, and you have to get the scripts to 'modprobe ext4' before you can mount the partition.

[Bowmore (bowmore)]

Sorry, but that update of the initramfs image made my sda1 unbootable.
Error 24: Attempt to access block outside partition.

What I did was, I chrooted the device and updated the initramfs image acc to bug #309758. Then at reboot error 24 was issued. I fscked the device with no error! It's still mountable from another system. So one way here might be (or is a must) to update the image before the ext3 to ext4 conversion. I'll give it a try.

[Daniel Swarbrick (pressureman)]

Just to confirm, the custom_mount_ext4 script is only needed in local-bottom.

Also, the `modprobe ext4` is not needed, since it's compiled into the kernel now. However, it's probably wise to using `-t ext4` as the mount option.

#! /bin/sh
mount -t ext4 /dev/sda1 /root
mount -t ext4 /dev/disk/by-uuid/...... /root

Works like a charm for me.

[Bowmore (bowmore)]

Works like a charm here too.

Brought up a second system successfully with ext4. The first try (error 24) showed up to be a misconfiguration in the grub's menu.lst and is now up and running as well.

The "script custom_mount_ext4" implemented in local-bottom only.
The "modprobe ext4" not used.

[Daniel Swarbrick (pressureman)]

An even simpler solution is to append rootfstype=ext4 to kopt in /boot/grub/menu.list, eg.

# kopt=root=UUID=24c707be-c824-4781-89bd-25ed9dba54f8 ro rootfstype=ext4

Then just run "sudo update-grub". No custom initramfs scripts needed.

[Michael Marley (mamarley)]

I can confirm that this works.

[Cleber Santz (clebersantz)]

Same error here ( using kernel 2.6.28-4, ext4 and grub2 ), solved adding rootfstype=ext4

[kede (kede)]

confirmed, rootfstype=ext4 works.

[Colin Watson (cjwatson)]

Hmm, odd, ext4 is supposed to be supported in klibc's fstype which should obviate the need for this. I'll investigate once I get installer support going.

[Colin Watson (cjwatson)]

I think this is fixed in klibc 1.5.15.

[Launchpad Janitor (janitor)]

This bug was fixed in the package klibc - 1.5.14-1~exp1ubuntu2

---------------
klibc (1.5.14-1~exp1ubuntu2) jaunty; urgency=low

  * 20_ext4.patch: Backport Theodore Ts'o's patch from 1.5.15 to fix
    ext4/ext4dev probing (LP: #309762).

 -- Colin Watson <email address hidden> Thu, 08 Jan 2009 13:53:41 +0000