unable to boot after installing vagrant

Bug #1794318 reported by Jeroen
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
vagrant
New
Undecided
Unassigned
linux (Ubuntu)
Incomplete
Undecided
Unassigned
mdadm (Ubuntu)
Incomplete
Undecided
Unassigned

Bug Description

Reproduction steps:
clean install of kubuntu & reboot
full update & reboot
apt-get install vagrant & reboot
=>busybox

My system won't boot after installing vagrant. When I switch back to the previous kernel version it works until another update-initramfs.

Currently I'm using 2 ssd's for intel fake raid 1(mirror) for kubuntu 18.04

I've reinstalled my system multiple times. I can reproduce this issue every time.

Tags: cscc
Revision history for this message
Jeroen (c0p3rn1c) wrote :
Revision history for this message
Jeroen (c0p3rn1c) wrote :

View attachment for more info.

Revision history for this message
Paul Sladen (sladen) wrote : Which kernel?

Per the earlier conversation:

  https://irclogs.ubuntu.com/2018/09/25/%23ubuntu-devel.html#t09:22

need to know:

  * exact version number of last working Linux kernel
  * exact version number of non-working Linux kernel

Revision history for this message
Dimitri John Ledkov (xnox) wrote :

So which kernel is good and which one is bad?

you didn't mention at all which kernels are borked.... there has been no mdadm updates lately.

Changed in mdadm (Ubuntu):
status: New → Incomplete
Changed in linux (Ubuntu):
status: New → Incomplete
Revision history for this message
Jeroen (c0p3rn1c) wrote : Re: kernel update breaks boot on intel fake raid 1

I reinstalled my system again and did a fool update and a reboot. It looks like something else is causing the problem.

description: updated
Changed in linux (Ubuntu):
status: Incomplete → Invalid
status: Invalid → Incomplete
Changed in mdadm (Ubuntu):
status: Incomplete → Invalid
Revision history for this message
Jeroen (c0p3rn1c) wrote :

full*

summary: - kernel update breaks boot on intel fake raid 1
+ unable to boot
Jeroen (c0p3rn1c)
summary: - unable to boot
+ unable to boot after installing vagrant?
Revision history for this message
Jeroen (c0p3rn1c) wrote : Re: unable to boot after installing vagrant?

I've added my logs after a failed boot (using a usb live disk).

Jeroen (c0p3rn1c)
description: updated
summary: - unable to boot after installing vagrant?
+ unable to boot after installing vagrant
Jeroen (c0p3rn1c)
description: updated
Revision history for this message
Jeroen (c0p3rn1c) wrote :

After installing vagrant it did a update-initramfs, which I presume, broke booting from kernel 4.15.0-34-generic when I switched back, using the grub menu, to a previous kernel 4.15.0-29-generic I could still boot without problems

Revision history for this message
Jeroen (c0p3rn1c) wrote :

initramfs

Revision history for this message
Jeroen (c0p3rn1c) wrote :

after a purge remove vagrant I get raid problems.

Changed in mdadm (Ubuntu):
status: Invalid → Incomplete
Revision history for this message
Jeroen (c0p3rn1c) wrote :

sudo cat /proc/mounts
[sudo] password for jeroen:
sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0
udev /dev devtmpfs rw,nosuid,relatime,size=8140992k,nr_inodes=2035248,mode=755 0 0
devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0
tmpfs /run tmpfs rw,nosuid,noexec,relatime,size=1634112k,mode=755 0 0
/dev/mapper/isw_dafiffhfja_Volume1p2 / ext4 rw,relatime,errors=remount-ro,data=ordered 0 0
securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0
tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0
tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0
tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0
cgroup /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0
cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0
pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0
efivarfs /sys/firmware/efi/efivars efivarfs rw,nosuid,nodev,noexec,relatime 0 0
cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0
cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0
cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0
cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0
cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0
cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0
cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0
cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0
cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0
cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0
cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0
systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=25,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15340 0 0
debugfs /sys/kernel/debug debugfs rw,relatime 0 0
mqueue /dev/mqueue mqueue rw,relatime 0 0
hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0
sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0
configfs /sys/kernel/config configfs rw,relatime 0 0
fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0
nfsd /proc/fs/nfsd nfsd rw,relatime 0 0
/dev/mapper/isw_dafiffhfja_Volume1p1 /boot/efi vfat rw,relatime,fmask=0077,dmask=0077,codepage=437,iocharset=iso8859-1,shortname=mixed,errors=remount-ro 0 0
tmpfs /run/user/119 tmpfs rw,nosuid,nodev,relatime,size=1634108k,mode=700,uid=119,gid=124 0 0
tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=1634108k,mode=700,uid=1000,gid=1000 0 0

Revision history for this message
Jeroen (c0p3rn1c) wrote :

find /etc/initramfs-tools/ -ls
 14417979 4 drwxr-xr-x 5 root root 4096 sep 26 08:25 /etc/initramfs-tools/
 14419131 4 -rw-r--r-- 1 root root 378 jan 5 2018 /etc/initramfs-tools/update-initramfs.conf
 14419128 4 drwxr-xr-x 12 root root 4096 jul 25 05:03 /etc/initramfs-tools/scripts
 14419136 4 drwxr-xr-x 2 root root 4096 feb 20 2018 /etc/initramfs-tools/scripts/local-premount
 14419135 4 drwxr-xr-x 2 root root 4096 feb 20 2018 /etc/initramfs-tools/scripts/local-bottom
 14419137 4 drwxr-xr-x 2 root root 4096 feb 20 2018 /etc/initramfs-tools/scripts/local-top
 14419139 4 drwxr-xr-x 2 root root 4096 feb 20 2018 /etc/initramfs-tools/scripts/nfs-premount
 14419133 4 drwxr-xr-x 2 root root 4096 feb 20 2018 /etc/initramfs-tools/scripts/init-premount
 14419134 4 drwxr-xr-x 2 root root 4096 feb 20 2018 /etc/initramfs-tools/scripts/init-top
 14419140 4 drwxr-xr-x 2 root root 4096 feb 20 2018 /etc/initramfs-tools/scripts/nfs-top
 14419141 4 drwxr-xr-x 2 root root 4096 feb 20 2018 /etc/initramfs-tools/scripts/panic
 14419138 4 drwxr-xr-x 2 root root 4096 feb 20 2018 /etc/initramfs-tools/scripts/nfs-bottom
 14419132 4 drwxr-xr-x 2 root root 4096 feb 20 2018 /etc/initramfs-tools/scripts/init-bottom
 14419126 4 drwxr-xr-x 2 root root 4096 jul 25 05:17 /etc/initramfs-tools/conf.d
 14419130 4 -rw-r--r-- 1 root root 246 jul 25 05:03 /etc/initramfs-tools/modules
 14419129 4 -rw-r--r-- 1 root root 1646 jan 5 2018 /etc/initramfs-tools/initramfs.conf
 14419127 4 drwxr-xr-x 2 root root 4096 feb 20 2018 /etc/initramfs-tools/hooks

Revision history for this message
Paul Sladen (sladen) wrote : Not kernel, mdadm initramfs

TL;DR: probably mdadm commit from April 2018; see end.
Today's IRC logs of debugging session with reporter:

  https://irclogs.ubuntu.com/2018/09/26/%23ubuntu-devel.html#t06:41

Current understanding of the situation from log analysis:

  08:21 reboot with (4.15.0-29.31-generic), no problem
  08:21 apt-get --no-upgrade -o Acquire::gpgv::Options::=--ignore-time-conflict -y install grub-efi-amd64-signed
  08:22 apt-get --no-upgrade -o Acquire::gpgv::Options::=--ignore-time-conflict -y install shim-signed
  08:23 apt upgrade

  08:30 reboot with (4.15.0-34.37-generic), no problem
  08:35 sudo apt-get install vagrant -y && sudo reboot
  Setting up mdadm (4.1~rc1-3~ubuntu18.04.1) ...
  Setting up reiserfsprogs (1:3.6.27-2) ...
  Setting up lvm2 (2.02.176-4.1ubuntu3) ...
  Setting up xfsprogs (4.9.0+nmu1ubuntu2) ...
  Setting up xfsprogs (4.9.0+nmu1ubuntu2) ...
  update-initramfs: deferring update (trigger activated) [all above]
  Processing triggers for initramfs-tools (0.130ubuntu3.3) ...
  update-initramfs: Generating /boot/initrd.img-4.15.0-34-generic

 ~08:37, reboot to (4.15.0-34.37-generic), fails, no syslog saved
  "IMSM RAID geometry validation failed." (screenshot)
  "platform does not support raid1 with 2 disks" (screenshot)

  08:38, reboot to (4.15.0-29.31-generic) [manually in Grub]
  08:59, reboot to (4.15.0-29.31-generic) [manually in Grub]

So, not the kernel. But something that (most likely) mdadm changed in
the copy of mdadm copied into the regenerated 'initramfs'.

  Probably running "mdadm --assemble --scan" should also produce
  the same message.

Both kernels booted fine prior to the mdadm update fetched via the
'apt-get install vagrant'.

That brings us to mdadm, looking at the master branch, there was a
commit just prior to the calculation code that generates the "IMSM
RAID geometry validation failed" message:

  https://www.spinics.net/lists/raid/msg60747.html
  https://github.com/neilbrown/mdadm/commit/fcc2c9daede11fcc67e4032fd6fa8da198aaa319#diff-d9ce63e82fa010385c80834480624fa4L7736

RAID 1 is mirroring. The second message
  "platform does not support raid1 with 2 disks"
might well suggest something is non-function.

Reporter has indicated they are not available for several hours, so
further investigation will likely have to wait.

 -Paul

Revision history for this message
Jeroen (c0p3rn1c) wrote :

sudo mdadm --detail-platform
[sudo] password for jeroen:
       Platform : Intel(R) Rapid Storage Technology
        Version : 12.9.0.2006
    RAID Levels : raid0
    Chunk Sizes : 4k 8k 16k 32k 64k 128k
    2TB volumes : supported
      2TB disks : supported
      Max Disks : 6
    Max Volumes : 2 per array, 4 per controller
 I/O Controller : /sys/devices/pci0000:00/0000:00:1f.2 (SATA)
          Port5 : /dev/sdd (S33HNX0J200966L)
          Port1 : /dev/sdb (S33HNX0J200852F)
          Port4 : /dev/sdc (WDZ29M4J)
          Port0 : /dev/sda (S33HNX0HA01515T)
          Port2 : - no device attached -
          Port3 : - no device attached -

Revision history for this message
Jeroen (c0p3rn1c) wrote :

sudo mdadm --assemble --scan
returns nothing

Revision history for this message
Jeroen (c0p3rn1c) wrote :

sudo mdadm --examine /dev/sd[a-z]
/dev/sda:
          Magic : Intel Raid ISM Cfg Sig.
        Version : 1.2.06
    Orig Family : b64dd376
         Family : b64de296
     Generation : 00000806
     Attributes : All supported
           UUID : fe813c51:b5e27074:8e458e63:fd55063a
       Checksum : d3a21665 correct
    MPB Sectors : 1
          Disks : 2
   RAID Devices : 1

  Disk00 Serial : S33HNX0HA01515T
          State : active
             Id : 00000000
    Usable Size : 976766976 (465.76 GiB 500.10 GB)

[Volume1]:
           UUID : 1cb157dd:43317668:a8db10af:a752a6c1
     RAID Level : 1
        Members : 2
          Slots : [UU]
    Failed disk : none
      This Slot : 0
    Sector Size : 512
     Array Size : 976766976 (465.76 GiB 500.10 GB)
   Per Dev Size : 976767240 (465.76 GiB 500.10 GB)
  Sector Offset : 0
    Num Stripes : 3815496
     Chunk Size : 64 KiB
       Reserved : 0
  Migrate State : idle
      Map State : normal
    Dirty State : clean
     RWH Policy : off

  Disk01 Serial : S33HNX0J200852F
          State : active
             Id : 00000001
    Usable Size : 976766976 (465.76 GiB 500.10 GB)
/dev/sdb:
          Magic : Intel Raid ISM Cfg Sig.
        Version : 1.2.06
    Orig Family : b64dd376
         Family : b64de296
     Generation : 00000806
     Attributes : All supported
           UUID : fe813c51:b5e27074:8e458e63:fd55063a
       Checksum : d3a21665 correct
    MPB Sectors : 1
          Disks : 2
   RAID Devices : 1

  Disk01 Serial : S33HNX0J200852F
          State : active
             Id : 00000001
    Usable Size : 976766976 (465.76 GiB 500.10 GB)

[Volume1]:
           UUID : 1cb157dd:43317668:a8db10af:a752a6c1
     RAID Level : 1
        Members : 2
          Slots : [UU]
    Failed disk : none
      This Slot : 1
    Sector Size : 512
     Array Size : 976766976 (465.76 GiB 500.10 GB)
   Per Dev Size : 976767240 (465.76 GiB 500.10 GB)
  Sector Offset : 0
    Num Stripes : 3815496
     Chunk Size : 64 KiB
       Reserved : 0
  Migrate State : idle
      Map State : normal
    Dirty State : clean
     RWH Policy : off

  Disk00 Serial : S33HNX0HA01515T
          State : active
             Id : 00000000
    Usable Size : 976766976 (465.76 GiB 500.10 GB)
/dev/sdc:
   MBR Magic : aa55
Partition[0] : 3907029167 sectors at 1 (type ee)
/dev/sdd:
   MBR Magic : aa55
Partition[0] : 4294967295 sectors at 1 (type ee)

Revision history for this message
Jeroen (c0p3rn1c) wrote :
Download full text (20.7 KiB)

sudo dmidecode
# dmidecode 3.1
Getting SMBIOS data from sysfs.
SMBIOS 2.8 present.
76 structures occupying 2828 bytes.
Table at 0xCBD62018.

Handle 0x0000, DMI type 11, 5 bytes
OEM Strings
 String 1:
 String 2: $BIOSE1210000000000000000
 String 3:
 String 4:
 String 5:

Handle 0x0001, DMI type 0, 24 bytes
BIOS Information
 Vendor: American Megatrends Inc.
 Version: E1773IMS.110
 Release Date: 11/02/2015
 Address: 0xF0000
 Runtime Size: 64 kB
 ROM Size: 4096 kB
 Characteristics:
  PCI is supported
  BIOS is upgradeable
  BIOS shadowing is allowed
  Boot from CD is supported
  Selectable boot is supported
  EDD is supported
  5.25"/1.2 MB floppy services are supported (int 13h)
  3.5"/720 kB floppy services are supported (int 13h)
  3.5"/2.88 MB floppy services are supported (int 13h)
  Print screen service is supported (int 5h)
  8042 keyboard services are supported (int 9h)
  Serial services are supported (int 14h)
  Printer services are supported (int 17h)
  ACPI is supported
  USB legacy is supported
  BIOS boot specification is supported
  Targeted content distribution is supported
  UEFI is supported
 BIOS Revision: 1.16

Handle 0x0002, DMI type 1, 27 bytes
System Information
 Manufacturer: Micro-Star International Co., Ltd.
 Product Name: GS70 2QE
 Version: REV:1.0
 Serial Number: 9S7177311046ZE9000017
 UUID: 00000000-0000-0000-0000-448A5B6EE822
 Wake-up Type: Power Switch
 SKU Number: To be filled by O.E.M.
 Family: To be filled by O.E.M.

Handle 0x0003, DMI type 2, 15 bytes
Base Board Information
 Manufacturer: Micro-Star International Co., Ltd.
 Product Name: MS-1773
 Version: REV:0.B
 Serial Number: BSS-0123456789
 Asset Tag: To be filled by O.E.M.
 Features:
  Board is a hosting board
  Board is replaceable
 Location In Chassis: To be filled by O.E.M.
 Chassis Handle: 0x0004
 Type: Motherboard
 Contained Object Handles: 0

Handle 0x0004, DMI type 3, 25 bytes
Chassis Information
 Manufacturer: To Be Filled By O.E.M.
 Type: Notebook
 Lock: Not Present
 Version: To Be Filled By O.E.M.
 Serial Number: To Be Filled By O.E.M.
 Asset Tag: To Be Filled By O.E.M.
 Boot-up State: Safe
 Power Supply State: Safe
 Thermal State: Safe
 Security Status: None
 OEM Information: 0x00000000
 Height: Unspecified
 Number Of Power Cords: 1
 Contained Elements: 1
  <OUT OF SPEC> (0)
 SKU Number: To be filled by O.E.M.

Handle 0x0005, DMI type 8, 9 bytes
Port Connector Information
 Internal Reference Designator: J1A1
 Internal Connector Type: None
 External Reference Designator: PS2Mouse
 External Connector Type: PS/2
 Port Type: Mouse Port

Handle 0x0006, DMI type 8, 9 bytes
Port Connector Information
 Internal Reference Designator: J1A1
 Internal Connector Type: None
 External Reference Designator: Keyboard
 External Connector Type: PS/2
 Port Type: Keyboard Port

Handle 0x0007, DMI type 8, 9 bytes
Port Connector Information
 Internal Reference Designator: J2A1
 Internal Connector Type: None
 External Reference Designator: TV Out
 External Connector Type: Mini Centronics Type-14
 Port Type: Other

Handle 0x0008, DMI type 8, 9 bytes
Port Connector Information
 Internal Reference Designator: J2A2A
 Internal Connector Type: None
 External Reference ...

Revision history for this message
Jeroen (c0p3rn1c) wrote :

bios screenshots

Revision history for this message
Jeroen (c0p3rn1c) wrote :
Revision history for this message
Paul Sladen (sladen) wrote : BIOS shows RAID1, newer mdadm --detail-platform does not

  https://irclogs.ubuntu.com/2018/09/27/%23ubuntu-devel.html#t11:31

The BIOS shows a RAID1 (mirror) pair, this is designated as:

  RAID Level: "Recovery (Cont.)" in the BIOS. (screenshot)

This stands for "Recovery Set" == RAID1/Mirror group.

Newer 'mdadm --detail-platform' only reports:

  $ sudo mdadm --examine /dev/sd[ab]
  Platform : Intel(R) Rapid Storage Technology
  Version : 12.9.0.2006
  RAID Levels : raid0

the expected output would instead be:

  "RAID Levels : raid0 raid1 raid10 raid5"

However, the version of 'mdadm' used to actually boot + configure the
system is the older version. It is still necessary to get the output
of 'mdadm --detail-platform' from the working version in the
initramfs.

This is because the version use on bootup is the one in the initramfs
(working) and the one here us on the main root filesystem
(non-working).

There are various other complications on this machine:

  1. Gaming laptop with the BIOS detail not properly populated:
  MSI Gaming Notebook GS70 2QE-046NL 17.3", 4710HQ, 1.5TB, GTX970M

  2. Other (non-RAIDed, non-SSD) disks on the same controller

  3. Apparently a BIOS update at some point.

All of these need testing/eliminating.

Reporter has indicated they are not available for several hours.

 -Paul

Brad Figg (brad-figg)
tags: added: cscc
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.