[LTCTest][OPAL][FW860.20] Upgrade to Ubuntu 16.04.2 Alpha from Ubuntu 16.04.1 is dropping to (initramfs)

[bugproxy]

@kernel-team

Please move ipr module from image-extra to image package.

---Problem Description---
Upgrade to Ubuntu 16.04.2 Alpha from Ubuntu 16.04.1 is dropping to (initramfs)

Contact Information = <email address hidden>

---uname output---
Linux (none) 4.8.0-27-generic #29~16.04.1-Ubuntu SMP Fri Nov 4 17:24:37 UTC 2016 ppc64le GNU/Linux

---Additional Hardware Info---
root@powerkvm3-lp1:~# lspci
0000:00:00.0 PCI bridge: IBM Device 03dc
0000:01:00.0 RAID bus controller: IBM Obsidian-E PCI-E SCSI controller (rev 01)
0001:00:00.0 PCI bridge: IBM Device 03dc
0001:01:00.0 PCI bridge: PLX Technology, Inc. PEX 8732 32-lane, 8-Port PCI Express Gen 3 (8.0 GT/s) Switch (rev ca)
0001:02:01.0 PCI bridge: PLX Technology, Inc. PEX 8732 32-lane, 8-Port PCI Express Gen 3 (8.0 GT/s) Switch (rev ca)
0001:02:08.0 PCI bridge: PLX Technology, Inc. PEX 8732 32-lane, 8-Port PCI Express Gen 3 (8.0 GT/s) Switch (rev ca)
0001:02:09.0 PCI bridge: PLX Technology, Inc. PEX 8732 32-lane, 8-Port PCI Express Gen 3 (8.0 GT/s) Switch (rev ca)
0001:03:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
0001:03:00.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
0001:03:00.2 Ethernet controller: Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
0001:03:00.3 Ethernet controller: Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
0001:04:00.0 RAID bus controller: IBM PCI-E IPR SAS Adapter (ASIC) (rev 01)
0001:05:00.0 RAID bus controller: IBM PCI-E IPR SAS Adapter (ASIC) (rev 01)
0004:00:00.0 PCI bridge: IBM Device 03dc
0004:01:00.0 Fibre Channel: Emulex Corporation Lancer-X: LightPulse Fibre Channel Host Adapter (rev 10)
0004:01:00.1 Fibre Channel: Emulex Corporation Lancer-X: LightPulse Fibre Channel Host Adapter (rev 10)
0005:00:00.0 PCI bridge: IBM Device 03dc
0005:01:00.0 PCI bridge: PLX Technology, Inc. Device 8748 (rev ca)
0005:02:01.0 PCI bridge: PLX Technology, Inc. Device 8748 (rev ca)
0005:02:08.0 PCI bridge: PLX Technology, Inc. Device 8748 (rev ca)
0005:02:09.0 PCI bridge: PLX Technology, Inc. Device 8748 (rev ca)
0005:02:10.0 PCI bridge: PLX Technology, Inc. Device 8748 (rev ca)
0005:02:11.0 PCI bridge: PLX Technology, Inc. Device 8748 (rev ca)
0005:03:00.0 USB controller: Texas Instruments TUSB73x0 SuperSpeed USB 3.0 xHCI Host Controller (rev 02)
0005:09:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
0005:09:00.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
0005:09:00.2 Ethernet controller: Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
0005:09:00.3 Ethernet controller: Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
0005:0f:00.0 Fibre Channel: Emulex Corporation Saturn-X: LightPulse Fibre Channel Host Adapter (rev 03)
0005:0f:00.1 Fibre Channel: Emulex Corporation Saturn-X: LightPulse Fibre Channel Host Adapter (rev 03)
0040:00:00.0 PCI bridge: IBM Device 03dc
0044:00:00.0 PCI bridge: IBM Device 03dc
0044:01:00.0 Ethernet controller: Emulex Corporation OneConnect NIC (Lancer) (rev 10)
0044:01:00.1 Ethernet controller: Emulex Corporation OneConnect NIC (Lancer) (rev 10)
0044:01:00.2 Ethernet controller: Emulex Corporation OneConnect NIC (Lancer) (rev 10)
0044:01:00.3 Ethernet controller: Emulex Corporation OneConnect NIC (Lancer) (rev 10)
0044:01:00.4 Fibre Channel: Emulex Corporation OneConnect FCoE Initiator (Lancer) (rev 10)
0044:01:00.5 Fibre Channel: Emulex Corporation OneConnect FCoE Initiator (Lancer) (rev 10)
0045:00:00.0 PCI bridge: IBM Device 03dc
0045:01:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
0045:01:00.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
0045:01:00.2 Ethernet controller: Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
0045:01:00.3 Ethernet controller: Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)

Machine Type = P8

---Debugger---
A debugger is not configured

---Steps to Reproduce---
 Install Ubuntu 16.04.1 OS using netboot images.
Then upgrade the kernel by Installing the kernel 4.8 on the same.
After upgrading the kernel, we are booting to the new 4.8 kernel installed and the booting drops to (initramfs) prompt.

 Petitboot (v1.2.6-a96e9ae) 8284-22A 100E5CA
 ??????????????????????????????????????????????????????????????????????????????
    Ubuntu 16.04.1 LTS (16.04) (on /dev/sdd2)
    Ubuntu, with Linux 4.4.0-57-generic (recovery mode)
    Ubuntu, with Linux 4.4.0-57-generic
    Ubuntu, with Linux 4.8.0-27-generic (recovery mode)
    Ubuntu, with Linux 4.8.0-27-generic
 * Ubuntu
  [Disk: sdg2 / cbe9647e-7e2c-4fa9-a6ea-482961f5a5a9]
    Red Hat Enterprise Linux Server (0-rescue-1b320bf4f31b40dd8241cdc5f9bfa51c)
    Red Hat Enterprise Linux Server (3.10.0-481.el7.ppc64le) 7.3 (Maipo) (on /d
    Red Hat Enterprise Linux Server (3.10.0-505.el7.ppc64le) 7.3 (Maipo) (on /d
    Red Hat Enterprise Linux Server release 7.3 Beta (Maipo) (on /dev/mapper/rh
    Ubuntu, with Linux 4.8.0-28-generic (recovery mode)
    Ubuntu, with Linux 4.8.0-28-generic (upstart)
    Ubuntu, with Linux 4.8.0-28-generic
    Ubuntu, with Linux 4.8.0-30-generic (recovery mode)
    Ubuntu, with Linux 4.8.0-30-generic (upstart)
    Ubuntu, with Linux 4.8.0-30-generic
    Ubuntu, with Linux 4.8.0-32-generic (recovery mode)
    Ubuntu, with Linux 4.8.0-32-generic (upstart)
 ??????????????????????????????????????????????????????????????????????????????
 Enter=accept, e=edit, n=new, x=exit, l=language, h=help
The system is going down NOW!t
Sent SIGTERM to all processes
Sent SIGKILL to all processes
[ 121.250547] kexec_core: Starting new kernel
 -> smp_release_cpus()
spinning_secondaries = 127
 <- smp_release_cpus()
Gave up waiting for root device. Common problems:
 - Boot args (cat /proc/cmdline)
   - Check rootdelay= (did the system wait long enough?)
   - Check root= (did the system wait for the right device?)
 - Missing modules (cat /proc/modules; ls /dev)
ALERT! UUID=861f5dca-e488-45c5-8950-fc023d7bf564 does not exist. Dropping to a shell!

BusyBox v1.22.1 (Ubuntu 1:1.22.0-15ubuntu1) built-in shell (ash)
Enter 'help' for a list of built-in commands.

(initramfs)

(initramfs) uname -a
Linux (none) 4.8.0-27-generic #29~16.04.1-Ubuntu SMP Fri Nov 4 17:24:37 UTC 2016 ppc64le GNU/Linux

(initramfs) cat /proc/cmdline
root=UUID=861f5dca-e488-45c5-8950-fc023d7bf564 ro quiet splash

(initramfs) cat /proc/modules
crc32c_vpmsum 10099 0 - Live 0xd00000001b8d0000

(initramfs) ls /dev
vga_arbiter tty34 ttyS7 loop4
rfkill tty35 ttyS8 loop5
mem tty36 ttyS9 loop6
null tty37 ttyS10 loop7
zero tty38 ttyS11 net
full tty39 ttyS12 ptp0
random tty40 ttyS13 ptp1
urandom tty41 ttyS14 ptp2
kmsg tty42 ttyS15 ptp3
tty tty43 ttyS16 ptp4
console tty44 ttyS17 ptp5
tty0 tty45 ttyS18 ptp6
vcs tty46 ttyS19 ptp7
vcsa tty47 ttyS20 ptp8
vcs1 tty48 ttyS21 ptp9
vcsa1 tty49 ttyS22 ptp10
tty1 tty50 ttyS23 ptp11
tty2 tty51 ttyS24 ppp
tty3 tty52 ttyS25 bus
tty4 tty53 ttyS26 input
tty5 tty54 ttyS27 psaux
tty6 tty55 ttyS28 uinput
tty7 tty56 ttyS29 rtc0
tty8 tty57 ttyS30 i2c-0
tty9 tty58 ttyS31 i2c-1
tty10 tty59 ttyprintk mapper
tty11 tty60 hwrng cpu_dma_latency
tty12 tty61 lightnvm network_latency
tty13 tty62 ram0 network_throughput
tty14 tty63 ram1 memory_bandwidth
tty15 nvram ram2 pts
tty16 ecryptfs ram3 core
tty17 fuse ram4 fd
tty18 ptmx ram5 stdin
tty19 hvc0 ram6 stdout
tty20 hvc1 ram7 stderr
tty21 hvc2 ram8 char
tty22 hvc3 ram9 rtc
tty23 hvc4 ram10 block
tty24 hvc5 ram11 vcs2
tty25 hvc6 ram12 vcsa2
tty26 hvc7 ram13 vcs3
tty27 ttyS0 ram14 vcsa3
tty28 ttyS1 ram15 vcs4
tty29 ttyS2 loop-control vcsa4
tty30 ttyS3 loop0 vcs5
tty31 ttyS4 loop1 vcsa5
tty32 ttyS5 loop2 vcs6
tty33 ttyS6 loop3 vcsa6

Stack trace output:
 no

Oops output:
 no

System Dump Info:
  The system is not configured to capture a system dump.

*Additional Instructions for <email address hidden>:
-Post a private note with access information to the machine that the bug is occuring on.
-Attach sysctl -a output output to the bug.

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2017-01-02 03:57 EDT-------
Screening the issue.

the wiki instructions are missing one package:

linux-image-extra-4.8.0-32-generic

Without it, you won't have the ipr module, so the boot will fail if your root device is ipr based.

Can you try installing linux-image-extra to the package list for installing the 4.8 kernel?

[Dimitri John Ledkov (xnox)]

Moving the ipr module from linux-image-extra to linux-image package should resolve this issue as well, no? Essential modules for booting should be kept in the linux-image package, whilst auxiliary can be kept in the -extra package.

[Tim Gardner (timg-tpi)]

But ipr.ko is already included. There must be some other missing boot essential module.

~/ubuntu/ubuntu-yakkety$ git grep ipr debian.master/control.d/generic.inclusion-list
debian.master/control.d/generic.inclusion-list:drivers/scsi/ipr.ko