kernel panic when trying to reboot in bionic

Bug #1759445 reported by Eric Vasquez on 2018-03-28
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
MAAS
Undecided
Unassigned
linux (Ubuntu)
High
Joseph Salisbury
Bionic
High
Joseph Salisbury

Bug Description

cpe_foundation test deployment of Bionic failed.
After some investigation, it looks like the nodes deployed and installed bionic, but never came back from a reboot.

Accessing the ILO console of a node in question (all nodes failed), it revealed a kernel panic (attached)

Eric Vasquez (envas) wrote :
Eric Vasquez (envas) wrote :
Andres Rodriguez (andreserl) wrote :

Hi Eric,

I experience the same issue in non-MAAS deployed machines after upgrading to the latest kernel. AS such, I'm marking this as invalid for MAAS and opening a task for the kernel.

Changed in maas:
status: New → Invalid
summary: - MAAS 2.3.1 fails to deploy Bionic due to kernel panic
+ Bionic due to kernel panic

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1759445

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

Changed in linux (Ubuntu):
status: New → Incomplete
tags: added: bionic

This bug is a kernel panic when rebooting at the end of a MAAS deployment of bionic; there is no way to run apport-collect.

Changed in linux (Ubuntu):
status: Incomplete → Confirmed
summary: - Bionic due to kernel panic
+ kernel panic when trying to reboot in bionic
tags: added: foundations-engine
Changed in linux (Ubuntu Bionic):
importance: Undecided → High
status: Confirmed → Triaged
assignee: nobody → Joseph Salisbury (jsalisbury)
Jason Hobbs (jason-hobbs) wrote :

We can no longer reproduce this.

Changed in linux (Ubuntu Bionic):
status: Triaged → Incomplete
Jason Hobbs (jason-hobbs) wrote :

We reproduced it again... looking to try the testing now.

Jason Hobbs (jason-hobbs) wrote :

So far we've only been able to produce this by doing bionic deploys.

One thing that stands out in the rsyslog for bionic deploys is this failure:

http://paste.ubuntu.com/p/y8xXc7PYjp/

Apr 2 17:48:35 leafeon blkdeactivate[1782]: /sbin/blkdeactivate: line 345: /bin/sort: No such file or directory

Could it be related?

Joseph Salisbury (jsalisbury) wrote :

It's possible that the message is related, but it would be difficult to track down without shortening the number of commits to search through between 4.4 and 4.15.

It sounds like you can't reproduce this error with Artful and 4.13 based kernels. Is that correct?

If so, it would be worth testing the 4.14 final kernel and possibly some of the 4.15 release candidates if 4.14. is good. All the kernels are available from:

http://kernel.ubuntu.com/~kernel-ppa/mainline/

Jason Hobbs (jason-hobbs) wrote :

After updating firmware on the servers, we can't reproduce it at all anymore.

Chris Gregan (cgregan) wrote :

We need to continue to investigate this issue as anyone upgrading a certified hardware with older firware will be stuck. This will be a bad user experience.

Changed in linux (Ubuntu Bionic):
status: Incomplete → Confirmed
Chris Gregan (cgregan) wrote :

Mark is pressing to potentially change dist upgrade to detect older firmware versions prior to upgrade to ensure the latest is in place before Ubuntu upgrade

John George (jog) wrote :

Possibly a reproduction but in this case Xenial is deployed and rsyslog shows the kernel upgraded to 4.4.0-119-generic. These are automated runs and we don't have the console output to confirm a kernel panic.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers