linux 6.8 fails to boot on arm64 if any param is more than 140 chars

[Shantur Rathore]

Hi,

Linux 6.8 kernel fails to boot on ARM64 when any Linux command line param is more than 140 characters.

Test Machine
=============

Rockchip RK3399 based RockPro64 with latest u-boot 2024.07-rc3 in EFI mode booting grubaa64.efi

Reproduced on
==============
Ubuntu 22.04.4 with Linux HWE Proposed 6.8
Ubuntu 24.04 with Linux 6.8

Works on
=========
Ubuntu 22.04.4 with Linux 5.15, 6.6, 6.7, 6.9

Steps to reproduce
====================

1. Install Ubuntu 24.04 which comes with Linux 6.8 by default or Ubuntu 22.04.4 install Linux HWE 6.8 from https://launchpad.net/~canonical-kernel-team/+archive/ubuntu/proposed?field.series_filter=jammy

2. Edit /boot/grub/grub.cfg and add the following param to any boot entry with Linux 6.8

testparam=f081c381e7b54edcba27e5f790d47911a4cc3e726d8d256878d3df9175c020e0f081c381e7b54edcba27e5f790d47911a4cc3e726d8d256878d3df9175c020e0f081c381e7b5

3. Reboot the machine and select the boot entry in grub with the testparam as above.

4. Observe kernel never boots.

Cause
========

After bisecting the kernel, I found that the bug was introduced in Linux 6.8-rc1 ( and released in Linux 6.8 ) with commit

commit dc3f5aae06381b43bc9d0d416bd15ee1682940e9
Author: Ard Biesheuvel <email address hidden>
Date: Wed Nov 29 12:16:12 2023 +0100

    arm64: idreg-override: Avoid parameq() and parameqn()

This got fixed with Linux 6.9-rc1 and released in Linux 6.9 as part of rework

commit e223a449125571daa62debd8249fa4fc2da0a961
Author: Ard Biesheuvel <email address hidden>
Date: Wed Feb 14 13:28:50 2024 +0100

    arm64: idreg-override: Move to early mini C runtime

That got merged to Linux 6.9-rc1 with commit

commit 6d75c6f40a03c97e1ecd683ae54e249abb9d922b
Merge: fe46a7dd189e 1ef21fcd6a50
Author: Linus Torvalds <email address hidden>
Date: Thu Mar 14 15:35:42 2024 -0700

    Merge tag 'arm64-upstream' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux

Fix Options
=============

Option 1.
For Linux 6.8 the safest fix would be to revert

commit dc3f5aae06381b43bc9d0d416bd15ee1682940e9
Author: Ard Biesheuvel <email address hidden>
Date: Wed Nov 29 12:16:12 2023 +0100

    arm64: idreg-override: Avoid parameq() and parameqn()

Option 2.
Unless Ubuntu Team is interested in bringing the whole Mini C Runtime rework to Linux 6.8 with

commit 6d75c6f40a03c97e1ecd683ae54e249abb9d922b
Merge: fe46a7dd189e 1ef21fcd6a50
Author: Linus Torvalds <email address hidden>
Date: Thu Mar 14 15:35:42 2024 -0700

    Merge tag 'arm64-upstream' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux

Please revert the change in Linux 6.8

Thanks

[Shantur Rathore (rathore4u)]

Hi @Matthew

Thanks for fixing the bug.
Can you please let me know where can I get the fixed kernel release from?

Thanks

[Matthew Ruffell (mruffell)]

Hi Shantur,

It is not fixed, I just marked it as fixed for "Ubuntu" a.k.a the development release oracular, since it will pick up a 6.10+ kernel when it eventually becomes available.

I added a noble entry, since noble's kernel is the one that actually needs to be fixed.

I did have a look at "arm64: idreg-override: Move to early mini C runtime", but it fails to cherry pick on noble's kernel with quite a few conflicts, and I haven't had time yet to go looking for dependencies.

Thanks,
Matthew

[Shantur Rathore (rathore4u)]

Hi Matthew,

Thanks for the update.
Please find my comments below

> I added a noble entry, since noble's kernel is the one that actually needs to be fixed.

We would need to add one entry for the next proposed hwe 6.8 for jammy

> I did have a look at "arm64: idreg-override: Move to early mini C runtime", but it fails to cherry pick on noble's kernel with quite a few conflicts, and I haven't had time yet to go looking for dependencies.

I believe that will be quite a bit to backport for 6.8, maybe the solution would be to revert the commit

commit dc3f5aae06381b43bc9d0d416bd15ee1682940e9
Author: Ard Biesheuvel <email address hidden>
Date: Wed Nov 29 12:16:12 2023 +0100

    arm64: idreg-override: Avoid parameq() and parameqn()

Thanks,

[Shantur Rathore (rathore4u)]

I have added the fix here

https://git.launchpad.net/~rathore4u/ubuntu/+source/linux/+git/version-seeds/commit/?h=LP2069534-140-char-fix&id=a16c19aee6d6f9b1ceb886ab835055668d662e8e

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Chris MacNaughton (chris.macnaughton)]

Hey Shantur,

Are you uploading that patch to Noble? I'm bumping into this issue at the moment as well.

Thanks!

[Shantur Rathore (rathore4u)]

Merge proposal - https://code.launchpad.net/~rathore4u/ubuntu/+source/linux/+git/version-seeds/+merge/469973

[Shantur Rathore (rathore4u)]

Hi Chris,

I have added the Merge proposal but not sure how to bring this to developers' attention.
May I ask which device did you face this issue on and are you using MAAS ?

Thanks

[Chris MacNaughton (chris.macnaughton)]

Shantur: I'm using MAAS and encountering this on ARM64 hardware.

here's hoping we can find somebody to engage to get this merged and uploaded!

[Shantur Rathore (rathore4u)]

I have a fix for MAAS too which I trying to get merged.

https://bugs.launchpad.net/maas/+bug/2069059