Boot hang after updating and setting up Iscsi with 4.15.0-72 generic kernel

[William]

Hi All,

Host Server - Dell PowerEdge R630, Dual Xeon E5-2650 v4, 768GB Memory, ESX 6.5.0

I created a VM with 1vCPU, 2GB Memory, 16GB Hard drive.

I installed Ubuntu Server 16.04.2 (latest ISO I had on hand), installed fine.

I then ran apt-get update, apt-get upgrade... upgrade failed as repos were out of date. I then ran do-distribution-upgrade, ran through everything and it appears to work.

--I don't want to mislead anyone - I can't remember if I rebooted or not at this point.

After this, I setup an Iscsi initiator to an external target, added to fstab, formatted and confirmed all working.

I then rebooted, and... nothing!

No ping, no ssh or anything... Control + Alt + Del did not work, looked like it had frozen. The last line was starting SSH, but the previous line was the Iscsi initiator. At this point, I figured network is down and ISCSI is hanging as it can't connect/maybe I messed up the ISCSI mount.

I then went to GRUB and did advanced, I did the recovery console for kernel 4.15.0-72 generic which lead to a kernel panic: https://snipboard.io/YCOZwv.jpg

I then went through the options again and selected 4.4.0-62 generic, and it booted fine, and after about 2 seconds at the recovery menu, further ISCSI related text came up: https://snipboard.io/ALZcsF.jpg

I went to the shell as root, deleted the line from fstab, and rebooted, but, it hang again at the same point.

I then went to advanced, booted normally to 4.4.0-62 generic, and, it went through without any issue what so ever.

Messing around with kernels puts me out of my depth... happy to give further info or do more diagnostics if required, however, due to it working in a different kernel and nothing out of the ordinary - I believe this to be a bug.

This is on Ubuntu 18.04.3 LTS / Release 18.04
---
ProblemType: Bug
AlsaDevices:
 total 0
 crw-rw---- 1 root audio 116, 1 Dec 3 01:35 seq
 crw-rw---- 1 root audio 116, 33 Dec 3 01:35 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay': 'aplay'
ApportVersion: 2.20.9-0ubuntu7.9
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord': 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
DistroRelease: Ubuntu 18.04
HibernationDevice: RESUME=UUID=c2e23b2c-adf6-42b4-bdd7-9d825a36d18c
InstallationDate: Installed on 2019-12-02 (0 days ago)
InstallationMedia: Ubuntu-Server 16.04.2 LTS "Xenial Xerus" - Release amd64 (20170215.8)
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig': 'iwconfig'
Lsusb:
 Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
 Bus 002 Device 003: ID 0e0f:0002 VMware, Inc. Virtual USB Hub
 Bus 002 Device 002: ID 0e0f:0003 VMware, Inc. Virtual Mouse
 Bus 002 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
MachineType: VMware, Inc. VMware Virtual Platform
Package: linux (not installed)
PciMultimedia:

ProcFB: 0 svgadrmfb
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.4.0-62-generic root=UUID=dd730eac-9993-408e-aa5c-c79c6cfcb52e ro
ProcVersionSignature: Ubuntu 4.4.0-62.83-generic 4.4.40
RelatedPackageVersions:
 linux-restricted-modules-4.4.0-62-generic N/A
 linux-backports-modules-4.4.0-62-generic N/A
 linux-firmware 1.173.12
RfKill: Error: [Errno 2] No such file or directory: 'rfkill': 'rfkill'
Tags: bionic
Uname: Linux 4.4.0-62-generic x86_64
UpgradeStatus: Upgraded to bionic on 2019-12-02 (0 days ago)
UserGroups: adm cdrom dip lpadmin lxd plugdev sambashare sudo
_MarkForUpload: True
dmi.bios.date: 04/05/2016
dmi.bios.vendor: Phoenix Technologies LTD
dmi.bios.version: 6.00
dmi.board.name: 440BX Desktop Reference Platform
dmi.board.vendor: Intel Corporation
dmi.board.version: None
dmi.chassis.asset.tag: No Asset Tag
dmi.chassis.type: 1
dmi.chassis.vendor: No Enclosure
dmi.chassis.version: N/A
dmi.modalias: dmi:bvnPhoenixTechnologiesLTD:bvr6.00:bd04/05/2016:svnVMware,Inc.:pnVMwareVirtualPlatform:pvrNone:rvnIntelCorporation:rn440BXDesktopReferencePlatform:rvrNone:cvnNoEnclosure:ct1:cvrN/A:
dmi.product.name: VMware Virtual Platform
dmi.product.version: None
dmi.sys.vendor: VMware, Inc.

[William (wilhil)]

Sorry, I'm new here - I was searching for related bugs to not create a duplicate and it appears to automatically got the intel microcode tag... I can't confirm it is related or not to that.

p.s. I put the Iscsi target back in fstab, rebooted and selected the old kernel - it booted and mounted without any issue at all.

[William (wilhil)]

apport information

[William (wilhil)]

apport information

[William (wilhil)]

apport information

[William (wilhil)]

apport information

[William (wilhil)]

apport information

[William (wilhil)]

apport information

[William (wilhil)]

apport information

[William (wilhil)]

apport information

[William (wilhil)]

apport information

[William (wilhil)]

apport information

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This change was made by a bot.

[Kai-Heng Feng (kaihengfeng)]

Can you please find the last working kernel?

[William (wilhil)]

I'm sorry, I'm out of my depth when it comes to kernels. It is purely a test environment and I'm happy to give remote access and/or follow instructions, but, I'm really not sure how to do this.

[Jeff Lane  (bladernr)]

Just out of curiosity, you mention in comment #1 "p.s. I put the Iscsi target back in fstab, rebooted and selected the old kernel - it booted and mounted without any issue at all."

Does this mean that at this point you can reboot and the system boots fine with the iscsi target in fstab? Or do you mean that it ONLY boots successfully using the old kernel, but if you boot the new one again, you're back to hanging?

Which kernel successfully boots? (boot the system and do a `uname -a`)

[Jeff Lane  (bladernr)]

Note, I only ask because the kernel listed in some of the info is Ubuntu 4.4.0-62.83-generic 4.4.40 but you mention 18.04 and 18.04.3 and 4.4 and 4.15 and a whole mess of confusing info. So I just want to nail down exactly what kernel works, and what kernel fails.

[William (wilhil)]

Hi Jeff, sorry - I tried to be as accurate as possible.

In or out of fstab - I can only boot in the older kernel, 4.4.0-62 otherwise it freezes on boot.

As per instructions on bug reporting, I ran lsb_release -rd, which printed 18.04.3 LTS on the first line,
 Release 18.04 on the second line.

Seth on IRC told me to run a command which I did that then put all the debug info on this ticket.