kernel panic: stack corrupted in efivar_ssdt_iter+0xd3/0xe8

[Steven Tang]

We have a few 2010 mac pro running 1604 LTS that are just upgraded to Ubuntu 1804 LTS. While they all worked fine with the 1604 LTS and 4.4 kernel, they can no longer boot with the 4.15.0-50 kernel installed with 1804.
We tried various kernel flags without luck, and installed latest 4.18.0-20 from HWE and got the same result.
Unfortunately, no logs are created in /var/logs when the issue happens, and we could only boot with the old 4.4 kernel. We had to enable earlyprintk=efi to get a backtrace in a photo.
The issue happens consistently, i.e. the backtrace looks always the same just happens on different cores.
Attached the boot-info output and the picture taken at the end of the kernel loading.
---
ProblemType: Bug
ApportVersion: 2.20.9-0ubuntu7.6
Architecture: amd64
AudioDevicesInUse:
 USER PID ACCESS COMMAND
 /dev/snd/controlC2: steventang 2047 F.... pulseaudio
 /dev/snd/controlC0: steventang 2047 F.... pulseaudio
 /dev/snd/controlC1: steventang 2047 F.... pulseaudio
CurrentDesktop: ubuntu:GNOME
DistroRelease: Ubuntu 18.04
HibernationDevice: RESUME=UUID=a7ceb7f0-6040-4236-ad7d-35780a1dc870
InstallationDate: Installed on 2019-05-28 (0 days ago)
InstallationMedia: Ubuntu 16.04 LTS "Xenial Xerus" - Release amd64 (20160420.1)
MachineType: Apple Inc. MacPro5,1
NonfreeKernelModules: wl
Package: linux (not installed)
ProcFB: 0 radeondrmfb
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.4.0-148-generic root=UUID=8d18bfd8-303b-4397-9216-a0e48cb6ef54 ro quiet splash
ProcVersionSignature: Ubuntu 4.4.0-148.174-generic 4.4.177
RelatedPackageVersions:
 linux-restricted-modules-4.4.0-148-generic N/A
 linux-backports-modules-4.4.0-148-generic N/A
 linux-firmware 1.173.6
Tags: bionic
Uname: Linux 4.4.0-148-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: adm cdrom dip lpadmin plugdev sambashare sudo
_MarkForUpload: True
dmi.bios.date: 08/08/17
dmi.bios.vendor: Apple Inc.
dmi.bios.version: MP51.88Z.0084.B00.1708080528
dmi.board.asset.tag: 0
dmi.board.name: Mac-F221BEC8
dmi.board.vendor: Apple Inc.
dmi.chassis.type: 7
dmi.chassis.vendor: Apple Inc.
dmi.chassis.version: Mac-F221BEC8
dmi.modalias: dmi:bvnAppleInc.:bvrMP51.88Z.0084.B00.1708080528:bd08/08/17:svnAppleInc.:pnMacPro5,1:pvr0.0:rvnAppleInc.:rnMac-F221BEC8:rvr:cvnAppleInc.:ct7:cvrMac-F221BEC8:
dmi.product.name: MacPro5,1
dmi.product.version: 0.0
dmi.sys.vendor: Apple Inc.

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1830951

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Steven Tang (stevenyutang)]

We cannot find any logs in var/log related to the failed boots, tried grepping the kernel version, journalctl --list-boots

[Steven Tang (stevenyutang)]

apport information

[Steven Tang (stevenyutang)]

apport information

[Steven Tang (stevenyutang)]

apport information

[Steven Tang (stevenyutang)]

apport information

[Steven Tang (stevenyutang)]

apport information

[Steven Tang (stevenyutang)]

apport information

[Steven Tang (stevenyutang)]

apport information

[Steven Tang (stevenyutang)]

apport information

[Steven Tang (stevenyutang)]

apport information

[Steven Tang (stevenyutang)]

apport information

[Steven Tang (stevenyutang)]

apport information

[Steven Tang (stevenyutang)]

apport information

[Steven Tang (stevenyutang)]

apport information

[Steven Tang (stevenyutang)]

apport information

[Steven Tang (stevenyutang)]

apport information

[Steven Tang (stevenyutang)]

I recompiled the 4.18.0-20-general kernel and made the following changes, and things start to work.

258c258,261
<
---
> if (limit < 1) {
> pr_err("efi: Invalid name size %ld/%d.\n", name_size, limit);
> return 0;
> }

Logs proving the fix worked:
[ 0.321692] Registered efivars operations
[ 0.330723] efi: efi: Invalid name size 0/0.

[Kai-Heng Feng (kaihengfeng)]

Please test latest mainline kernel:
https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.2-rc2/

If mainline kernel doesn't fix the issue, we need to do a kernel bisection between v4.4 and v4.15.

[Po-Hsu Lin (cypressyew)]

Status change as per request in comment #25

[Steven Tang (stevenyutang)]

@cypressyew, I cannot have proof that the same issue occurred with v5.2 rc2 from your link, as the kernel does not respond to earlyprintk=efi, but it died pretty much the same way, and no logs were produced.
As I mentioned earlier, the fix in efi.c is a clear indication of a bug in the kernel code. If that isn't sufficient, I'm not sure what is.

The same bug exists in the latest kernel source.
https://git.launchpad.net/~ubuntu-kernel-test/ubuntu/+source/linux/+git/mainline-crack/tree/drivers/firmware/efi/efi.c#n258

That being said, if you could provide me a kernel build that will have proper logging I could capture, then I can try again. But please, please, to save us some time, first take a quick look at the source, it isn't too hard to see if it is a legitimate bug.

[Kai-Heng Feng (kaihengfeng)]

Hi Steven,

I've checked the code and I think you are right, the case doesn't get handled. It should abort when there's no name.
a) Will you upstream the patch with proper commit message?
b) I think it should be "if (limit <= 1)" since it's a NULL terminated string.

[Steven Tang (stevenyutang)]

Hi Kai-Heng,
  Indeed, it should be <=1, btw, do you want me to follow the kernel dev process to send the patch to upstream maintainer, or do you want a patch uploaded here?
  I'm unfamiliar with the process, but I think I can find out if there isn't any guidelines.

Thanks,

Steven

[Kai-Heng Feng (kaihengfeng)]

This can be a good start:
https://www.kernel.org/doc/html/latest/process/submitting-patches.html