grub-probe seems to have problems reading zfs root volumes:
# uname -a Linux ... 4.4.0-42-generic #62-Ubuntu SMP Fri Oct 7 23:11:45 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
grub2-common 2.02~beta2-36ubuntu3.2
# grub-probe -vv / ... grub-probe: info: opening hostdisk//dev/sda,gpt3. grub-core/kern/disk.c:196: Opening `hostdisk//dev/sda,gpt3'... grub-probe: info: drive = 0. grub-probe: info: the size of hostdisk//dev/sda is 250069680. grub-core/partmap/gpt.c:90: Read a valid GPT header grub-core/partmap/gpt.c:114: GPT entry 0: start=2048, length=262144 grub-core/partmap/gpt.c:114: GPT entry 1: start=264192, length=524288 grub-core/partmap/gpt.c:114: GPT entry 2: start=788480, length=... grub-core/kern/fs.c:56: Detecting zfs... grub-core/osdep/hostdisk.c:415: opening the device `/dev/sda3' in open_device() grub-core/fs/zfs/zfs.c:1192: label ok 0 grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sda3' grub-core/fs/zfs/zfs.c:1007: check 2 passed grub-core/fs/zfs/zfs.c:1018: check 3 passed grub-core/fs/zfs/zfs.c:1025: check 4 passed grub-core/fs/zfs/zfs.c:1035: check 6 passed grub-core/fs/zfs/zfs.c:1043: check 7 passed grub-core/fs/zfs/zfs.c:1054: check 8 passed grub-core/fs/zfs/zfs.c:1064: check 9 passed grub-core/fs/zfs/zfs.c:1086: check 11 passed grub-core/fs/zfs/zfs.c:1112: check 10 passed grub-core/fs/zfs/zfs.c:1128: str=com.delphix:hole_birth grub-core/fs/zfs/zfs.c:1128: str=com.delphix:embedded_data grub-core/fs/zfs/zfs.c:1137: check 12 passed (feature flags) grub-core/fs/zfs/zfs.c:1875: zio_read: E 0: size 2048/2048 grub-core/fs/zfs/zfs.c:1898: endian = -1 grub-core/fs/zfs/zfs.c:595: dva=8, 24001a0 grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sda3' grub-core/fs/zfs/zfs.c:442: checksum fletcher4 verification failed grub-core/fs/zfs/zfs.c:447: actual checksum 0000008850fa484a 00008bc07d1cd92b 005f734808deba9e 309d3301197ab484 grub-core/fs/zfs/zfs.c:452: expected checksum 00000003723052ca 0000065794886ce3 0005d97ca345c97f 039aedaa53c30aa5 grub-core/fs/zfs/zfs.c:1919: incorrect checksum grub-core/kern/fs.c:78: zfs detection failed. ...
So it seems zio_read in grub-core/fs/zfs/zfs.c is failing.
The effect is that the root pool detection is /etc/grub.d/10_linux
rpool=`${grub_probe} --device ${GRUB_DEVICE} --target=fs_label 2>/dev/null || true`
gives an empty name for the root pool, resulting in an wrong kernel parameter
root=ZFS=[empty]/root
in /boot/grub/grub.cfg, rendering the system unbootable.
grub-probe seems to have problems reading zfs root volumes:
# uname -a
Linux ... 4.4.0-42-generic #62-Ubuntu SMP Fri Oct 7 23:11:45 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
grub2-common 2.02~beta2- 36ubuntu3. 2
# grub-probe -vv / /dev/sda, gpt3. kern/disk. c:196: Opening `hostdisk/ /dev/sda, gpt3'.. . partmap/ gpt.c:90: Read a valid GPT header partmap/ gpt.c:114: GPT entry 0: start=2048, length=262144 partmap/ gpt.c:114: GPT entry 1: start=264192, length=524288 partmap/ gpt.c:114: GPT entry 2: start=788480, length=... kern/fs. c:56: Detecting zfs... osdep/hostdisk. c:415: opening the device `/dev/sda3' in open_device() fs/zfs/ zfs.c:1192: label ok 0 osdep/hostdisk. c:394: reusing open device `/dev/sda3' fs/zfs/ zfs.c:1007: check 2 passed fs/zfs/ zfs.c:1018: check 3 passed fs/zfs/ zfs.c:1025: check 4 passed fs/zfs/ zfs.c:1035: check 6 passed fs/zfs/ zfs.c:1043: check 7 passed fs/zfs/ zfs.c:1054: check 8 passed fs/zfs/ zfs.c:1064: check 9 passed fs/zfs/ zfs.c:1086: check 11 passed fs/zfs/ zfs.c:1112: check 10 passed fs/zfs/ zfs.c:1128: str=com. delphix: hole_birth fs/zfs/ zfs.c:1128: str=com. delphix: embedded_ data fs/zfs/ zfs.c:1137: check 12 passed (feature flags) fs/zfs/ zfs.c:1875: zio_read: E 0: size 2048/2048 fs/zfs/ zfs.c:1898: endian = -1 fs/zfs/ zfs.c:595: dva=8, 24001a0 osdep/hostdisk. c:394: reusing open device `/dev/sda3' fs/zfs/ zfs.c:442: checksum fletcher4 verification failed fs/zfs/ zfs.c:447: actual checksum 0000008850fa484a 00008bc07d1cd92b 005f734808deba9e 309d3301197ab484 fs/zfs/ zfs.c:452: expected checksum 00000003723052ca 0000065794886ce3 0005d97ca345c97f 039aedaa53c30aa5 fs/zfs/ zfs.c:1919: incorrect checksum kern/fs. c:78: zfs detection failed.
...
grub-probe: info: opening hostdisk/
grub-core/
grub-probe: info: drive = 0.
grub-probe: info: the size of hostdisk//dev/sda is 250069680.
grub-core/
grub-core/
grub-core/
grub-core/
grub-core/
grub-core/
grub-core/
grub-core/
grub-core/
grub-core/
grub-core/
grub-core/
grub-core/
grub-core/
grub-core/
grub-core/
grub-core/
grub-core/
grub-core/
grub-core/
grub-core/
grub-core/
grub-core/
grub-core/
grub-core/
grub-core/
grub-core/
grub-core/
grub-core/
...
So it seems zio_read in grub-core/ fs/zfs/ zfs.c is failing.
The effect is that the root pool detection is /etc/grub. d/10_linux
rpool=` ${grub_ probe} --device ${GRUB_DEVICE} --target=fs_label 2>/dev/null || true`
gives an empty name for the root pool, resulting in an wrong kernel parameter
root=ZFS= [empty] /root
in /boot/grub/ grub.cfg, rendering the system unbootable.