Keyboard and Touchpad Not Working in New Lenovo V15 Gen4 Laptop

[Pradip Kumar Das]

[Impact]

Keyboard and touchpad doesn't work on some recent systems, and also s2idle is broken.

[Fix]

Two upstream commits.

128b0c9781c9f26 x86/i8259: Skip probing when ACPI/MADT advertises PCAT compatibility
3bde7ec13c97144 platform/x86: Add s2idle quirk for more Lenovo laptops

[Test case]

boot a fixed kernel and test that input and s2idle works.

[Where problems could occur]

A buggy bios could maybe advertise a system being PCAT compatible when it's not, though in such a case it might have been already caught before.

--

Hello.

Ubuntu 22.04.3 with (later upgraded to kernel 6.2.0-32-generic) was installed in rewly purchased LENOVO V15 GEN4 AMN (AMD Ryzen 5 7520u) laptop and it was noticed that keyboard, touchpad and microphone are not working. The keyboard and touchpad work fine in BIOS setup and till GRUB (command line). It was found that when external devices such as keyboard, mouse and microphone are connected through USB and 3.5 jack, respectively, these work just fine. To confirm these are not hardware problems, Microsoft Windows 11 (Home) was installed in another disk partition and observed all these working alright. Hence a bug is being reported to draw attention of the concerned team and I request them to refer this issue and do the needful at the earliest.

Regards,
Pradip Kumar Das

ProblemType: Bug
DistroRelease: Ubuntu 22.04
Package: linux-image-6.2.0-32-generic 6.2.0-32.32~22.04.1
ProcVersionSignature: Ubuntu 6.2.0-32.32~22.04.1-generic 6.2.16
Uname: Linux 6.2.0-32-generic x86_64
ApportVersion: 2.20.11-0ubuntu82.5
Architecture: amd64
CasperMD5CheckResult: pass
CurrentDesktop: ubuntu:GNOME
Date: Wed Sep 6 08:04:42 2023
InstallationDate: Installed on 2023-08-14 (22 days ago)
InstallationMedia: Ubuntu 22.04.2 LTS "Jammy Jellyfish" - Release amd64 (20230223)
ProcEnviron:
 LANGUAGE=en_IN:en
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=en_IN
 SHELL=/bin/bash
SourcePackage: linux-signed-hwe-6.2
UpgradeStatus: No upgrade log present (probably fresh install)

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Danielle Satterfield (dsatterfield)]

I am fairly certain this also affects the Lenovo V14 G4 AMN running the same 7520u processor. Symptoms are the same.

[Serge (serge1278345)]

Lenovo V15 G4 AMN (82YU00UCRA) - AMD Ryzen 3 7320U - same problem.

[Morty Williams (user307513)]

Tried all solutions. Nothing helps at this point. This laptop is quite fresh, I think there will be even more requests.

[Rayen Majoul (re0syst)]

Same problem - lenovo V15 G4 AMN - AMD Athlon silver 7120U

[Mirko di marco (straccosss)]

Idem -Lenovo V15 64 AMN rz7520u

[Jean-Marie Tschanz (jmtschanz)]

Same touchpad + mouse issue here. When I'm online I must alternate plugging mouse and keyboard in order to navigate

[rambhau (rambhau)]

Same issue with Lenovo V15 Gen 4 AMN, AMD Ryzen 5 7520u.
Fix Released? where?

[Endre Olah (endreolah68)]

Same issue on Lenovo Ideapad Slim 3 AMD Ryzen 5 7520u. How this above fix will be issued? Any option to be able to test it? Thanks

[Serge (serge1278345)]

Wtf? Why status "Fix Released"? How it works?

[Endre Olah (endreolah68)]

I guess, Rayen was just marked this as fixed, but actually it is not. Can this bug be reopened? I have checked that who can make this status adjustment and it seems that anybody can do it, but unfortunately you just could elevate the status and you cannot put it back to a previous status. This issue might never be fixed, if somebody just say "Fixed" without real confirmation.

[Pradip Kumar Das (pradipkumardas)]

Hello Endre,

I have written email to Rayen to consider this bug status change and requested to let us know about the solution.

Regards,
Pradip Kumar Das

[Endre Olah (endreolah68)]

Hi Pradip,

if Rayen is not in any team to fix the issue, how can he decide if it is solved or not? I think there is an issue how the workflows are running.

There should be a solution explained and then confirmed by at least a few affected users or by time if at least one confirmation is given that the fix is working.

Now somebody is saying the fix issued, but nobody else knows what is that and how to apply. My computer is still not having the fix. I cannot find such a kernel, called linux.signed-hwe-6.2 or I am looking for it at a wrong place.

But thanks for trying to get this to the right direction.

Kind regards

       Endre

[Rayen Majoul (re0syst)]

I accidentally changed the status and can't revert it back to 'confirmed.'

[Rayen Majoul (re0syst)]

btw this problem persists even in the latest kernel version 6.5 on Ubuntu 23.10

[Endre Olah (endreolah68)]

Hi Pradip,

as you have issued this bug report, could you reset it to confirmed somehow or issue a new bug report, please?

There many people who have effected by this bug. I think most of the AMD 6000 and 7000 series laptops from some specific manufacturer. Lot of discussion forums having the same issue mentioned, but no solution so far.

Thanks

      Endre

[Pradip Kumar Das (pradipkumardas)]

As Rayen had changed the bug status from "Confirmed" to "Fix Released" by mistake, I just changed it back to the previous one, and hope that kernel team will now pay attention, work and release an appropriate fix for the above mentioned issues at the earliest.

[Endre Olah (endreolah68)]

Thanks, Pradip! You are great! I hope the fix will come soon. I like Linux, so I would like to use it.

Have a nice day!

[torel (torehl)]

cc. Same issue with 6.2.0-34-generic on an old i5 machine and Logitech K400 kbd w/integrated mousepad.

# dmesg |grep -i logitech
[ 5.648913] usb 2-1.1: Manufacturer: Logitech
[ 13.148210] input: Logitech USB Receiver as /devices/pci0000:00/0000:00:1a.0/usb2/2-1/2-1.1/2-1.1:1.0/0003:046D:C52B.0001/input/input3
[ 13.208627] hid-generic 0003:046D:C52B.0001: input,hidraw0: USB HID v1.11 Keyboard [Logitech USB Receiver] on usb-0000:00:1a.0-1.1/input0
[ 13.208971] input: Logitech USB Receiver Mouse as /devices/pci0000:00/0000:00:1a.0/usb2/2-1/2-1.1/2-1.1:1.1/0003:046D:C52B.0002/input/input4
[ 13.209214] input: Logitech USB Receiver Consumer Control as /devices/pci0000:00/0000:00:1a.0/usb2/2-1/2-1.1/2-1.1:1.1/0003:046D:C52B.0002/input/input5
[ 13.268569] input: Logitech USB Receiver System Control as /devices/pci0000:00/0000:00:1a.0/usb2/2-1/2-1.1/2-1.1:1.1/0003:046D:C52B.0002/input/input6
[ 13.269157] hid-generic 0003:046D:C52B.0002: input,hiddev0,hidraw1: USB HID v1.11 Mouse [Logitech USB Receiver] on usb-0000:00:1a.0-1.1/input1
[ 13.269713] hid-generic 0003:046D:C52B.0003: hiddev1,hidraw2: USB HID v1.11 Device [Logitech USB Receiver] on usb-0000:00:1a.0-1.1/input2

When it was working

# cat /var/log/Xorg.1.log.old | grep -i Logi
[110891.923] (II) systemd-logind: took control of session /org/freedesktop/login1/session/_3438
[110891.925] (II) systemd-logind: got fd for /dev/dri/card1 226:1 fd 15 paused 0
[110891.927] (II) systemd-logind: got fd for /dev/dri/card0 226:0 fd 16 paused 0
[110891.940] (II) systemd-logind: releasing fd for 226:1
[110892.180] (II) systemd-logind: got fd for /dev/input/event1 13:65 fd 42 paused 0
[110892.213] (II) systemd-logind: got fd for /dev/input/event2 13:66 fd 45 paused 0
[110892.220] (II) systemd-logind: got fd for /dev/input/event0 13:64 fd 46 paused 0
[110892.228] (II) config/udev: Adding input device Logitech K400 Plus (/dev/input/event3)
[110892.228] (**) Logitech K400 Plus: Applying InputClass "evdev pointer catchall"
[110892.228] (**) Logitech K400 Plus: Applying InputClass "evdev keyboard catchall"
[110892.228] (**) Logitech K400 Plus: Applying InputClass "libinput pointer catchall"
[110892.228] (**) Logitech K400 Plus: Applying InputClass "libinput keyboard catchall"
[110892.228] (II) Using input driver 'libinput' for 'Logitech K400 Plus'
[110892.229] (II) systemd-logind: got fd for /dev/input/event3 13:67 fd 47 paused 0
[110892.229] (**) Logitech K400 Plus: always reports core events
[110892.232] (II) event3 - Logitech K400 Plus: is tagged by udev as: Keyboard Mouse
[110892.232] (II) event3 - Logitech K400 Plus: device is a pointer
[110892.232] (II) event3 - Logitech K400 Plus: device is a keyboard
[110892.233] (II) event3 - Logitech K400 Plus: device removed
[110892.233] (II) libinput: Logitech K400 Plus: needs a virtual subdevice
[110892.233] (II) XINPUT: Adding extended input device "Logitech K400 Plus" (type: MOUSE, id 9)
[110892.233] (**) Logitech K400 Plus: (accel) selected scheme none/0
[110892.233] (**) Logitech K400 Plus: (accel) acceleration factor: 2.000
[110892.233] (**) Logitech K400 Plus: (accel) acceleration threshold: 4
[110892.235] (II) event3 - Logitech K400 Plus: ...

[Endre Olah (endreolah68)]

This what I saw. External keyboard and mouse attached and working. But internal keyboard and touch pad does not.

Built in microphone also does not work.

[Pradip Kumar Das (pradipkumardas)]

Exactly Endre. These are the major problems. Request to kernel team will be to address these three problems on priority.

In addition to that there is one more problem that I might not have mentioned earlier and that is the issue related to system Suspend functionality. In my LENOVO V15 GEN4 AMN (AMD Ryzen 5 7520u) laptop, system does not enter into standby mode when physical power button is pressed even though the button is mapped to suspend mode done through Settings-->Power-->Power Button Behavior. This is inconveinent especially when I wish to talk for a shorter distance with just laptop lid closed, but cannot do because then the system gets unstable. So, only option left to me is to move around by keeping the laptop lid open.

So, it should also be checked by others if they are experiencing the same behavior. And if so, then it could also be another concern that the kernel team might be interested to look into.

[Rayen Majoul (re0syst)]

Does anyone know of a Linux distribution with the latest kernel version 6.6?

[Endre Olah (endreolah68)]

According to the Kernel list 6.5 is the latest available/official, but even that was not solving the problem.

[suroto (suroto-linux-kernel-bugs)]

Hello,

I have installed fedora workstation 38, with kernel default (versi 6.2 ??).
But keyboard not working.
Then I try ubuntu 22.04.3 LTS . it's same problem.
Then try upgrade kernel of Fedora Workstation 38 to kernel 6.6..
keyboard still not working.

It's possible fix it ?

Thank You
SUroto

[suroto (suroto-linux-kernel-bugs)]

additional details hardware:
S/N : Pf4DM1wG
Model Name: Lenovo V14 G4 AMN. ( before any typo mistake, G14)

[bagasdotme (bagasdotme-linux-kernel-bugs)]

(In reply to Suroto from comment #0)
> Hello,
>
> I have installed fedora workstation 38, with kernel default (versi 6.2 ??).
> But keyboard not working.
> Then I try ubuntu 22.04.3 LTS . it's same problem.
> Then try upgrade kernel of Fedora Workstation 38 to kernel 6.6..
> keyboard still not working.
>

Does hotkeys (usually laptop-specific) have problem?

[suroto (suroto-linux-kernel-bugs)]

hotkey working only on the contras screen key

[aros (aros-linux-kernel-bugs)]

Mario, could you take a look please?

This was reported months ago:

https://forum.manjaro.org/t/lenovo-amd-no-keyboard/141597/5

https://unix.stackexchange.com/questions/751668/lenovo-v15-g4-amn-mouse-and-keyboard-is-not-working-under-linux

https://bugs.launchpad.net/ubuntu/+source/linux-signed-hwe-6.2/+bug/2034477

https://askubuntu.com/questions/1480477/laptop-keyboard-and-mosuepad-dosnt-work-on-lenovo-v15-g4-amn-type-82yu

Multiple Lenovo laptops exhibit this issue and the company doesn't seem to care one bit. They could have fixed the issue themselves. Please just don't buy or recommend their products to anyone who cares about Linux. I've no idea who to ping or what to do.

I've heard that some modern laptops have some sort of I2C (?) multiplexors which need to be set up properly in order to enable keyboard input but how it all works, again I've no clue.

I'm CC'ing Mario Limonciello because I remember he knows something about that. Maybe he could chime in and give the instructions how to debug the issue and fix it all the affected people.

[aros (aros-linux-kernel-bugs)]

CC'ing Tamim Khan for good measure.

[suroto (suroto-linux-kernel-bugs)]

Thank you for your explanation.

Since 15 years ago, I have installed Linux on various brands and types of computers,
Just recently, I encountered a hardware problem that Linux doesn't recognize.

I will try to send an issue ticket to Levono

[Endre Olah (endreolah68)]

On 23.04 the microphone is working already. I have tried to upgrade yesterday to 23.10, but the install windows are not appearing properly, so cannot see the steps. (With the updater tool the upgrade did not appear yet.)

The other windows on the live DVD, like Libre Office, or the settings windows work properly.

[Kyrylo Budnyk (kirry36)]

I have this laptop (V15 AMN on R3 7320U / 16G RAM), have built 6.5.7 kernel with all I2C drivers.
Uploading several of dmesg dump files with different parameters and xinput dump file.

Hope we will see a fix soon.

Also can notice, "extra buttons" kind of working, even can setup screen brightness. Also keyboard is responding to Numpad, Caps Lock states (duplicating it from USB keyboard)

[p0lloc (p0lloc-linux-kernel-bugs)]

Same issue with IdeaPad Slim 3 14AMN8, quite a lot of good info in https://bugzilla.kernel.org/show_bug.cgi?id=217873 but no solution.

I have tried an immense amount of kernel parameter combinations and patching the latest mainline kernel with IRQ quirks of different combinations, to no avail.

The only combination of i8042 parameters that do ANYTHING is "i8042.direct i8042.dumbkbd i8042.noloop i8042.nopnp" which finally gets a working keyboard (but with an insane delay of 15-60 seconds per keypress). Dumbkbd doesn't seem strictly necessary but you receive 10-20 commas before any actual keyboard input gets through.

Really hoping for a solution or any suggestions on what to try next.

[mario.limonciello (mario.limonciello-linux-kernel-bugs)]

Can both of you guys please test with the latest 6.6-rc, and if it continues to fail attach these to the bug:

1) acpidump
2) your dmesg from 6.6-rcX
3) dmidecode

[mario.limonciello (mario.limonciello-linux-kernel-bugs)]

And if you haven't already; please upgrade to the latest BIOS release for your systems from Lenovo.

[p0lloc (p0lloc-linux-kernel-bugs)]

Created attachment 305213
acpidump 6.6-rc5

[p0lloc (p0lloc-linux-kernel-bugs)]

Created attachment 305214
dmesg 6.6-rc5

[p0lloc (p0lloc-linux-kernel-bugs)]

Created attachment 305215
dmidecode 6.6-rc5

[p0lloc (p0lloc-linux-kernel-bugs)]

Attached the requested files, running 6.6-rc5 and the latest BIOS available (L1CN32WW)

[jwrdegoede (jwrdegoede-linux-kernel-bugs)]

Hi p0lloc,

I read in bug 217873 that you already have tried to add your model to the DMI quirks to make drivers/acpi/resource.c use the MADT override info on your AMD Zen based laptops resulting in:

ACPI: IRQ 1 override to edge, high(!)

Showing up in the log, which is what that DMI quirk should do. So it looks like you got the patch right but this unfortunately does not help.

This suggests that your IdeaPad Slim 3 14AMN8 is a whole new category of kbd IRQ bug where neither the DSDT nor the MADT override info is correct (what fun).

This really shows once more that to fix this once and for all we need to find a way to readback the actually IRQ settings setup by the BIOS at boot and use those (ignoring all the tables which tell us the info since those seem to be unreliable).

But for now first lets confirm that this really is an IRQ type / polarity issue.

By default 6.6-rc5 uses "edge low" as trigger type and with the DMI quirk which you tested it uses "edge high" as trigger type and neither works.

We have seen several Intel based laptop models which need to NOT use the MADT override so that they stick with the DSDT setting of "level low" so lets give that a try.

I'll attach a quick hack which ignores all the ACPI tables and just sets IRQ1 directly to "level low". Please build a kernel with this and see if this helps.

If it does not help please try "level high" by changing:

                u8 pol = ACPI_ACTIVE_LOW;

to:

                u8 pol = ACPI_ACTIVE_HIGH;

Note a level IRQ with the wrong polarity (low vs high) will lead to an interrupt storm which might very well lead to a non booting system. So make sure you have an entry for another kernel in your grub menu as fallback.

Also if the IRQ storm is not "bad" enough it may seem as if everything works, while at the same time the CPU is very busy.

So if things seem to work please run:

cat /proc/interrupts

and check that the numbers for IRQ 1 are not going crazy (each keypress should add 2 to the IRQ count one for press + one for release).

[jwrdegoede (jwrdegoede-linux-kernel-bugs)]

Created attachment 305217
[PATCH] ACPI: resource: IRQ polarity hack

Here is the promised patch for testing on the Lenovo IdeaPad Slim 3 15AMN8.

[jwrdegoede (jwrdegoede-linux-kernel-bugs)]

Suroto for starters can you please provide the logs which Mario requested in comment 8 ?

[jwrdegoede (jwrdegoede-linux-kernel-bugs)]

p.s.

p0lloc after building the kernel with the patch dmesg should contain:

ACPI: IRQ 1 override to level(!), low

please check that this is present in dmesg (assuming the kernel boots at all).

[p0lloc (p0lloc-linux-kernel-bugs)]

(In reply to Hans de Goede from comment #14)
> Hi p0lloc,
>
> I read in bug 217873 that you already have tried to add your model to the
> DMI quirks to make drivers/acpi/resource.c use the MADT override info on
> your AMD Zen based laptops resulting in:
>
> ACPI: IRQ 1 override to edge, high(!)
>
> Showing up in the log, which is what that DMI quirk should do. So it looks
> like you got the patch right but this unfortunately does not help.
>
> This suggests that your IdeaPad Slim 3 14AMN8 is a whole new category of kbd
> IRQ bug where neither the DSDT nor the MADT override info is correct (what
> fun).
>
> This really shows once more that to fix this once and for all we need to
> find a way to readback the actually IRQ settings setup by the BIOS at boot
> and use those (ignoring all the tables which tell us the info since those
> seem to be unreliable).
>
> But for now first lets confirm that this really is an IRQ type / polarity
> issue.
>
> By default 6.6-rc5 uses "edge low" as trigger type and with the DMI quirk
> which you tested it uses "edge high" as trigger type and neither works.
>
> We have seen several Intel based laptop models which need to NOT use the
> MADT override so that they stick with the DSDT setting of "level low" so
> lets give that a try.
>
> I'll attach a quick hack which ignores all the ACPI tables and just sets
> IRQ1 directly to "level low". Please build a kernel with this and see if
> this helps.
>
> If it does not help please try "level high" by changing:
>
> u8 pol = ACPI_ACTIVE_LOW;
>
> to:
>
> u8 pol = ACPI_ACTIVE_HIGH;
>
> Note a level IRQ with the wrong polarity (low vs high) will lead to an
> interrupt storm which might very well lead to a non booting system. So make
> sure you have an entry for another kernel in your grub menu as fallback.
>
> Also if the IRQ storm is not "bad" enough it may seem as if everything
> works, while at the same time the CPU is very busy.
>
> So if things seem to work please run:
>
> cat /proc/interrupts
>
> and check that the numbers for IRQ 1 are not going crazy (each keypress
> should add 2 to the IRQ count one for press + one for release).

Hello Hans, thank you for the swift response!
I tried applying your patch but unfortunately it doesn't seem to help.
Curiously I get the message "ACPI: IRQ 1 override to edge(!), high" and not "level(!), low" as you suggested.
Looking at the source code, it doesn't seem that the variables "t" or "p" are ever modified, so I assume it should always show "edge" and "high"?

I went ahead and tried all combinations of edge/level and high/low but sadly to no avail.
This is the output in dmesg:
LEVEL_SENSITIVE ACTIVE_LOW: edge(!), high
LEVEL_SENSITIVE ACTIVE_HIGH: edge(!), high(!)
EDGE_SENSITIVE ACTIVE_HIGH: edge, high(!)
EDGE_SENSITIVE ACTIVE_LOW: no override message

[Endre Olah (endreolah68)]

Ubuntu 23.10 not fixed the issue yet. Kernel 6.5.0

[Pradip Kumar Das (pradipkumardas)]

Thanks Endre for the update, but that kernel v6.5.x was my hope to get these issues addressed. How long can we just be waiting for without any communication from the kernel development team? One option is just wait for indefinite period of time to hear from them or to take decision to invest once more in another computer which is matter of hefty amount money and making me feel low about these whole experience we are going through.

[Endre Olah (endreolah68)]

I partially agree with you, Pradip. It is weird that such input methods are not working on all laptops under Linux. But I can imagine that too many different hardware are on the market, where it is hard to develop to all at once and maybe the developers does not have the chance to test on all of them. So let's be patient for a while. These hardwares are pretty new, so the fix will come, I am sure.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[jwrdegoede (jwrdegoede-linux-kernel-bugs)]

> I tried applying your patch but unfortunately it doesn't seem to help.
Curiously I get the message "ACPI: IRQ 1 override to edge(!), high" and not "level(!), low" as you suggested.
Looking at the source code, it doesn't seem that the variables "t" or "p" are ever modified, so I assume it should always show "edge" and "high"?

Ah right my patch neglects to change t and p, but the (!) does show the settings are actually being changed so it does do what it should.

Its unfortunate that the patch does not help. From your description in bug 217873 where you write "This causes an extreme input delay (15 seconds to 1 minute per key press) but at least something happens." I was hoping this would be an IRQ issue, because that sounds a lot like IRQs not working.

I wonder what happens if you mix my patch which manually sets the interrupt trigger type with some of the i8042 options.

E.g. try i8042.nopnp (as suggested in the dmesg) + i8042.dumbkbd + i8042.noloop note using i8042.direct generally speaking is a bad idea on laptops ...

I wonder what the output of "dmesg | grep i8042" is when adding "i8042.nopnp i8042.dumbkbd i8042.noloop" to the kernel commandline ?

And as said try mixing that with overriding the IRQ trigger type for starters I would go with "edge high" since that is the default override (which gets skipped on your laptop because it is AMD Zen based).

[[e-mail] (p0lloc)]

Hello,
If I'm not mistaken, this is being tracked in multiple kernel issues, https://bugzilla.kernel.org/show_bug.cgi?id=218003 (latest activity) and https://bugzilla.kernel.org/show_bug.cgi?id=217873. It seems that this could be the same IRQ issue that has been affecting many new laptops. The only success I've had is by applying the kernel parameters "i8042.nopnp i8042.direct i8042.noloop i8042.dumbkbd" which makes the keyboard work but with an extreme input delay. I have also tried building my own kernel with an IRQ patch but sadly to no avail. I have the laptop Lenovo IdeaPad Slim 3 14AMN8 so if you have a different model you might have more luck than me.

[suroto (suroto-linux-kernel-bugs)]

(In reply to Mario Limonciello (AMD) from comment #8)
> Can both of you guys please test with the latest 6.6-rc, and if it continues
> to fail attach these to the bug:
>
> 1) acpidump
> 2) your dmesg from 6.6-rcX
> 3) dmidecode

-----------
OK, I'll try kernel 6.6-rc.

Later I will update the results.

[suroto (suroto-linux-kernel-bugs)]

Created attachment 305228
dmesg

[suroto (suroto-linux-kernel-bugs)]

Created attachment 305229
dmidecode34

[jwrdegoede (jwrdegoede-linux-kernel-bugs)]

Suroto, thank you for the logs. To try and debug this further we really also need an acpidump though.

On Fedora can you please do:

sudo dnf install acpica-tools

and then run:

sudo acpidump -o acpidump.txt

And then attach the generated acpidump.txt file here ?

[Endre Olah (endreolah68)]

Bad news. Tested Fedora 39 beta, with kernel 6.5.7-300. Keyboard/Touchpad does not work.

[p0lloc (p0lloc-linux-kernel-bugs)]

I have now tried to apply the i8042 options together with the 3 different IRQ types. Interestingly they all make the keyboard work, but with the same extreme delay as previously mentioned. It doesn't seem that i8042.direct was never needed since it works fine with just "i8042.noloop i8042.dumbkbd i8042.nopnp".

The "dmesg | grep i8042" reads the same for all of them:

i8042: PNP detection disabled
serio: i8042 KBD port at 0x60,0x64 irq 1
serio: i8042 AUX port at 0x60,0x64 irq 12
input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input3

It seems that the AUX port is vital for the keyboard to work, since it only works whenever "AUX port at 0x60,0x64 irq 12" is displayed. This confuses me since I always thought that the AUX port was for the mouse/trackpad.

I tried to add some debug prints in the i8042 code and it seems that i8042_pnp_aux_probe in i8042-acpipnpio.h is never called, which prevents the AUX port from being created. Does this mean that the AUX port could not be found or is it just the IRQ failing? Nothing else in the code seems to be returning any errors.

[mario.limonciello (mario.limonciello-linux-kernel-bugs)]

Something I want to point out a few things about p0lloc's logs I noticed.

1) This system is Mendocino based, which I want to say is the first time we've seen one of these IRQ bugs outside of Cezanne, Barcelo and Rembrandt. There isn't anything fundamental about Mendocino incompatibilities, just want to mention it in case it shows any patterns in the future.

2) The GPIO controller failed to set up. It's possible that there is an IRQ storm going on because those interrupts aren't being serviced.

[ 0.276113] amd_gpio AMDI0030:00: error -EINVAL: IRQ index 0 not found
[ 0.278464] amd_gpio: probe of AMDI0030:00 failed with error -22

I do see that IRQ should be IRQ 7 from that acpidump.

        Device (GPIO)
        {
            Name (_HID, "AMDI0030") // _HID: Hardware ID
            Name (_CID, "AMDI0030") // _CID: Compatible ID
            Name (_UID, 0x00) // _UID: Unique ID
            Method (_CRS, 0, NotSerialized) // _CRS: Current Resource Settings
            {
                Name (RBUF, ResourceTemplate ()
                {
                    Interrupt (ResourceConsumer, Level, ActiveLow, Shared, ,, )
                    {
                        0x00000007,
                    }
                    Memory32Fixed (ReadWrite,
                        0xFED81500, // Address Base
                        0x00000400, // Address Length
                        )
                })
                Return (RBUF) /* \_SB_.GPIO._CRS.RBUF */
            }

            Method (_STA, 0, NotSerialized) // _STA: Status
            {
                If ((TSOS >= 0x70))
                {
                    Return (0x0F)
                }
                Else
                {
                    Return (0x00)
                }
            }
        }

3) I don't see any reasons that it should be on IRQ12.

Regarding Suroto's logs:

1) The system is also Mendocino based.
2) I see the same amd_gpio probe failed error.

IMO we should start out with fixing the problem causing the probe for amd_gpio to fail on both of these systems and hopefully that leads to a solution for the keyboard as well.

Maybe we can start with /proc/ioports and /proc/iomem (both captured as root) to look for any resource overlap or misassignment relative to what the kernel is declaring. Or Hans, do these mentions jog any other ideas for you?

[dlazar (dlazar-linux-kernel-bugs)]

Re: comment #25, the following logs are from a Lenovo V15 G4 AMN, the same machine as Suroto's, as far as I can tell. They are from an official Debian kernel (6.1.0-13-amd64, 6.1.55-1), i.e., without any patches from this bug, but for ioports and iomem, these shouldn't make any difference. For completeness, I'm adding the acpidump output as well.

[dlazar (dlazar-linux-kernel-bugs)]

Created attachment 305230
dmesg-6.1.55.txt

[dlazar (dlazar-linux-kernel-bugs)]

Created attachment 305231
acpidump.txt

[dlazar (dlazar-linux-kernel-bugs)]

Created attachment 305232
dmidecode.txt

[dlazar (dlazar-linux-kernel-bugs)]

Created attachment 305233
/proc/ioports

[dlazar (dlazar-linux-kernel-bugs)]

Created attachment 305234
/proc/iomem

[jwrdegoede (jwrdegoede-linux-kernel-bugs)]

Mario, thank you for your input. I agree that it seems that the IRQ polarity and/or i8042 options seem to be a dead end here.

The GPIO controller issue indeed looks suspect so lets start with trying to fix that and then see from there.

The "error -EINVAL: IRQ index 0 not found" error comes from platform_get_irq() and looking at that function the only way it can return -EINVAL without first printing an error is by acpi_irq_get() returning -EINVAL.

Which means that either one of these 2 calls is returning -EINVAL:

int acpi_irq_get(acpi_handle handle, unsigned int index, struct resource *res)
{
        ...
        rc = acpi_irq_parse_one(handle, index, &fwspec, &flags);
        if (rc)
                return rc;
        ...
        rc = irq_create_fwspec_mapping(&fwspec);
        if (rc <= 0)
                return -EINVAL;

with me suspecting that it is the second call which is failing.

[jwrdegoede (jwrdegoede-linux-kernel-bugs)]

Created attachment 305236
[PATCH] i8259 hack allow mapping of legacy IRQs with NULL PIC

I believe that this is related to this dmesg message:

    0.066641] Using NULL legacy PIC

Which likely is causing mapping of IRQs under 16 to fail, which would explain both the kbd IRQ and the GPIO irq failure to work.

Can someone please give this quick hack a try and see if that helps ?

[mario.limonciello (mario.limonciello-linux-kernel-bugs)]

Excellent find!

If it's using "NULL legacy PIC" that means that 8259 wasn't programmed properly by BIOS (Linux currently expects it). I've seen this in the past, and it's always been fixed by BIOS.

If Hans' hack doesn't work, another hack idea to confirm it is to program registers for it from a "broken" Linux kernel and then kexec into "another" kernel and see if it helps.

You can use any port I/O tools to write "0x68" to port "0x21" followed by "0x1" to port "0x21" to do this.

[mario.limonciello (mario.limonciello-linux-kernel-bugs)]

Created attachment 305238
Another possible hack

I discussed this a bit with Boris, can you try this other hack? This should be tried independently of the two other hacks that Hans and I have suggested.

It would force the system down the ACPI reduced hardware path.
If this "works", please check if suspend/resume works and also share a kernel log.

[Kyrylo Budnyk (kirry36)]

Endre, please. Stop looking for wonder. We're know that this bug NOT FIXED exactly. Why?
Because we're on bugtracker. And a lot another trackers/people also taking care about this bug. Few messages before you i have reported that 6.5.7 kernel isn't working. Moreover, i have tested 6.6-rc5 and its NOT WORK, because there is no working fix. How can it be fixed when this bug report is exist in Open state? This is not bug of several distributives, this is Linux kernel bug. Best thing we can do - start building kernels with patches to find the right fix or even better cooperate with good kernel developers and try to fix together.

[suroto (suroto-linux-kernel-bugs)]

(In reply to Hans de Goede from comment #23)
> Suroto, thank you for the logs. To try and debug this further we really also
> need an acpidump though.
>
> On Fedora can you please do:
>
> sudo dnf install acpica-tools
>
> and then run:
>
> sudo acpidump -o acpidump.txt
>
> And then attach the generated acpidump.txt file here ?

I got problem when install acpica-tools.
Still try to be solving

I will send the file, if I have successfully installed this program

[Endre Olah (endreolah68)]

Kyrylo, these guys are working on patches of the base kernel, so -300 kernel on Fedora might be having some solutions. Actually I can say that on Fedora, even those buttons were not working which was working on Ubuntu. Like the brightness buttons or the airplane mode button. Let me believe in miracles. :D

[k.kirry36 (k.kirry36-linux-kernel-bugs)]

Hello Hans, Mario and Boris!

I have this laptop and applied your patches to Debian (kernel 6.5.7).

dmesg will be attached.

To put it briefly:
1. Hans patch:
-- Keyboard and Trackpad is working
-- Suspend also working (have a bug with multiple "nvme 0000:01:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x000b address=0xb6674000 flags=0x0000]" but still work)

2. Mario/Boris patch:
-- Keyboard and Trackpad is working
-- Suspend isn't work, performing something like poweroff and startup

[k.kirry36 (k.kirry36-linux-kernel-bugs)]

Created attachment 305241
kernel log with Hans patch

[k.kirry36 (k.kirry36-linux-kernel-bugs)]

Created attachment 305242
kernel log with Hans patch (suspend)

[k.kirry36 (k.kirry36-linux-kernel-bugs)]

Created attachment 305243
kernel log with Mario and Boris patch

[k.kirry36 (k.kirry36-linux-kernel-bugs)]

Created attachment 305244
kernel log with Mario and Boris patch (suspend)

[jwrdegoede (jwrdegoede-linux-kernel-bugs)]

> 1. Hans patch:
> -- Keyboard and Trackpad is working
> -- Suspend also working (have a bug with multiple "nvme 0000:01:00.0: AMD-Vi:
> Event > logged [IO_PAGE_FAULT domain=0x000b address=0xb6674000 flags=0x0000]"
> but still work)

Ok, so we are definitely on the right track here. As simple as my patch is I don't expect it to be able to go upstream as is though.

I'm a bit out of my depth here. The whole PIC <-> IOAPIC logic is very convoluted. Mario can you ask Boris to take a look a this issue?

I realize this is a BIOS bug, but I still think we should fix this in the kernel since chasing all BIOSes which have this and get them updated is not going to work.

Kyrylo, about the nvme suspend/resume issue, does this also happen without my patch ?

[jwrdegoede (jwrdegoede-linux-kernel-bugs)]

Looking at the code more I do think that doing something like my hack is probably the way to deal with this.

Maybe something like this:

diff --git a/arch/x86/kernel/i8259.c b/arch/x86/kernel/i8259.c
index 30a55207c000..2136e85dd5b0 100644
--- a/arch/x86/kernel/i8259.c
+++ b/arch/x86/kernel/i8259.c
@@ -317,6 +317,16 @@ static int probe_8259A(void)
  if (new_val != probe_val) {
   printk(KERN_INFO "Using NULL legacy PIC\n");
   legacy_pic = &null_legacy_pic;
+#ifdef CONFIG_X86_IO_APIC
+ /*
+ * Some AMD Zen based devices have the PIC disabled by the BIOS
+ * but they still use legacy ISA IRQs attached through the IOAPIC.
+ */
+ if (boot_cpu_has(X86_FEATURE_ZEN)) {
+ printk(KERN_INFO "Using IOAPIC for legacy PIC/ISA IRQs\n");
+ legacy_pic->nr_legacy_irqs = NR_IRQS_LEGACY;
+ }
+#endif
  }

  raw_spin_unlock_irqrestore(&i8259A_lock, flags);

?

[k.kirry36 (k.kirry36-linux-kernel-bugs)]

(In reply to Hans de Goede from comment #42)
> > 1. Hans patch:
> > -- Keyboard and Trackpad is working
> > -- Suspend also working (have a bug with multiple "nvme 0000:01:00.0:
> AMD-Vi:
> > Event > logged [IO_PAGE_FAULT domain=0x000b address=0xb6674000
> flags=0x0000]"
> > but still work)
>
> Ok, so we are definitely on the right track here. As simple as my patch is I
> don't expect it to be able to go upstream as is though.
>
> I'm a bit out of my depth here. The whole PIC <-> IOAPIC logic is very
> convoluted. Mario can you ask Boris to take a look a this issue?
>
> I realize this is a BIOS bug, but I still think we should fix this in the
> kernel since chasing all BIOSes which have this and get them updated is not
> going to work.
>
> Kyrylo, about the nvme suspend/resume issue, does this also happen without
> my patch ?

Tested it, issue is happening but logged only once (one bunch of record), not as it happens with patch (every 5 sec)

[k.kirry36 (k.kirry36-linux-kernel-bugs)]

Created attachment 305246
raw kernel log 6.1.0 before suspend

[k.kirry36 (k.kirry36-linux-kernel-bugs)]

Created attachment 305247
raw kernel log 6.1.0 after suspend

[Kyrylo Budnyk (kirry36)]

Endre, i don't know what "these guys" are doing, but now we're have temporary fix for it. Tested but can be unstable. Mario, Boris from AMD and Hans from Fedoraproject keep working on it.

https://bugzilla.kernel.org/attachment.cgi?id=305236&action=diff

Working keyboard/trackpad and suspend with this kernel patch.

[mario.limonciello (mario.limonciello-linux-kernel-bugs)]

Created attachment 305248
run missing BIOS init sequence

#43:
Although simplistic I'm worried that is overzealous. It's going to apply to Epyc processors too, and those happen to be used in things like Azure. In Azure don't they use the NULL PIC path? I don't know if the Zen X86 feature exposes into confidential computing VM though.

How about instead we just run the missing initialization sequence from the BIOS as a quirk when these systems are detected?

Have a try with this patch instead.

[dlazar (dlazar-linux-kernel-bugs)]

Just a side note, regarding Mario's patch in #47, I've noticed that the exact same BIOS update[0] applies to the following systems:

  IdeaPad 14s AMN8
  IdeaPad 15s AMN8
  IdeaPad Slim 3 14AMN8
  IdeaPad Slim 3 14AMN8 1
  IdeaPad Slim 3 15AMN8
  IdeaPad Slim 3 15AMN8 1
  Lenovo V14 G4 AMN
  Lenovo V14 G4 AMN 1
  Lenovo V14 G4 AMN 2
  Lenovo V15 G4 AMN
  Lenovo V15 G4 AMN 1
  Lenovo V15 G4 AMN 2

[0] https://pcsupport.lenovo.com/de/en/products/laptops-and-netbooks/lenovo-v-series-laptops/lenovo-v15-g4-amn/82yu/downloads/driver-list/component?name=BIOS%2FUEFI&id=5AC6A815-321D-440E-8833-B07A93E0428C

So I'm assuming they're all affected by this bug. I'm not sure how to get their DMI_PRODUCT_NAME, maybe they're already covered by "82Y" and "82X". Anyone know?

[Endre Olah (endreolah68)]

Hi Kyrylo, what does this mean exactly? When it will be publicly available on Ubuntu updates?

[mario.limonciello (mario.limonciello-linux-kernel-bugs)]

> So I'm assuming they're all affected by this bug. I'm not sure how to get
> their DMI_PRODUCT_NAME, maybe they're already covered by "82Y" and "82X".
> Anyone know?

@Mark - can you help here?

> Just a side note, regarding Mario's patch in #47, I've noticed that the exact
> same BIOS update[0] applies to the following systems:

Let's see if this patch works for the ones I quirked so far. If it does work, I'd rather move the DMI detection into the error path in which case we can probably change it to "82" to cover all the Lenovo systems from this year.

[dlazar (dlazar-linux-kernel-bugs)]

Created attachment 305249
dmesg-6.6.0-rc6.txt

Mario's patch in comment #47 seems to work (see dmesg) on my machine (V15 G4 AMN, model number 82YU). The keyboard works, suspend-resume also works (only tried suspend to ram).

[mario.limonciello (mario.limonciello-linux-kernel-bugs)]

Created attachment 305250
run missing bios init sequence (v2)

OK thanks! Can you see if this version works still too?

[mario.limonciello (mario.limonciello-linux-kernel-bugs)]

> [ 1244.309246] PM: hibernation: hibernation entry

Also, I notice in your log you tested hibernate (suspend to disk), not suspend-to-ram.

> [ 0.355620] Low-power S0 idle used by default for system suspend

Can you please test suspend to idle (the default for your system).

[suroto (suroto-linux-kernel-bugs)]

Created attachment 305251
acpidump 6.6.rc6

Mario,

Sorry late respond, attached file - acpidump

[jwrdegoede (jwrdegoede-linux-kernel-bugs)]

(In reply to Mario Limonciello (AMD) from comment #51)
> Created attachment 305250 [details]
> run missing bios init sequence (v2)
>
> OK thanks! Can you see if this version works still too?

First of all let me say that I'm happy that we seem to be heading to a conclusion with a fix here.

2 comments about the v2 patch:

1. Are all models starting with 82 AMD based laptops? I thought the 82 indicated the introduction year of the model. So this can also run on Intel based models now ? IOW I think this maybe needs a if (boot_cpu_has(X86_FEATURE_ZEN)) check ?

2. I'm wondering if it would not be better to re-run the PIC detection after force-enabling it. So something like this:

        bool retried = false;

        /* ... */
        raw_spin_lock_irqsave(&i8259A_lock, flags);

retry:
        outb(0xff, PIC_SLAVE_IMR); /* mask all of 8259A-2 */
        outb(probe_val, PIC_MASTER_IMR);
        new_val = inb(PIC_MASTER_IMR);
        if (new_val != probe_val) {
                if (!retried && boot_cpu_has(X86_FEATURE_ZEN) &&
                    dmi_check_system(amd_force_enable_pic_table)) {
                        retried = true;
                        goto retry;
                }
                printk(KERN_INFO "Using NULL legacy PIC\n");
                ...

At least it seems to me that re-doing the check after the force enable to see if the force-enable actually worked is maybe a good idea ?

[suroto (suroto-linux-kernel-bugs)]

(In reply to David Lazar from comment #48)
> Just a side note, regarding Mario's patch in #47, I've noticed that the
> exact same BIOS update[0] applies to the following systems:
>
> IdeaPad 14s AMN8
> IdeaPad 15s AMN8
> IdeaPad Slim 3 14AMN8
> IdeaPad Slim 3 14AMN8 1
> IdeaPad Slim 3 15AMN8
> IdeaPad Slim 3 15AMN8 1
> Lenovo V14 G4 AMN
> Lenovo V14 G4 AMN 1
> Lenovo V14 G4 AMN 2
> Lenovo V15 G4 AMN
> Lenovo V15 G4 AMN 1
> Lenovo V15 G4 AMN 2
>
> [0]
> https://pcsupport.lenovo.com/de/en/products/laptops-and-netbooks/lenovo-v-
> series-laptops/lenovo-v15-g4-amn/82yu/downloads/driver-list/
> component?name=BIOS%2FUEFI&id=5AC6A815-321D-440E-8833-B07A93E0428C
>
> So I'm assuming they're all affected by this bug. I'm not sure how to get
> their DMI_PRODUCT_NAME, maybe they're already covered by "82Y" and "82X".
> Anyone know?

FYI,
After BIOS update , keyboard still is not working

[dlazar (dlazar-linux-kernel-bugs)]

(In reply to Mario Limonciello (AMD) from comment #52)

I finally managed to test the v2 patch (from comment #51). The keyboard and touchpad work fine.

> Also, I notice in your log you tested hibernate (suspend to disk), not
> suspend-to-ram.
>
> > [ 0.355620] Low-power S0 idle used by default for system suspend
>
> Can you please test suspend to idle (the default for your system).

I was imprecise with my choice of words: what I had tested was `systemctl hybrid-sleep`, which I assumed was equivalent to suspend-to-ram unless the battery runs empty while suspended. That appears not to be the case, so this time I tested all three:

1. systemctl hibernate -- "suspend to disk", this seems to work fine.

2. systemctl hybrid-sleep -- also seems to work fine

3. systemctl suspend -- "suspend to ram", had some issues: the first attempt managed to suspend, but upon resume the file system was reporting IO errors, to the point that even "ls" didn't work anymore. I couldn't get a dmesg output in this state. (Interestingly enough, my ssh connection resumed fine, presumably because it was already in RAM). I tried again, this time running in background `dmesg -w > dmesg-6.6.0-rc6-v2-suspend.txt`, so I'd at least capture output up to the crash (attached). This time, resume seemed to work, so I tried again, and this third time, the network did not resume properly, so I couldn't continue my ssh session, but the file system seemed to be ok, so I got some dmesg output (dmesg-6.6.0-rc6-v2-suspend-2.txt). There's a kernel stack trace in there, with this message:

> [ 804.018115] Hardware became unavailable upon resume. This could be a
> software issue prior to suspend or a hardware issue.

I'll attach the logs right away.

[dlazar (dlazar-linux-kernel-bugs)]

Created attachment 305252
dmesg-6.6.0-rc6-v2-hibernate.txt

systemctl hibernate -- seemed to work fine

[dlazar (dlazar-linux-kernel-bugs)]

Created attachment 305253
dmesg-6.6.0-rc6-v2-hybrid-sleep.txt

systemctl hybrid-sleep -- seemed to work fine

[dlazar (dlazar-linux-kernel-bugs)]

Created attachment 305254
dmesg-6.6.0-rc6-v2-suspend.txt

systemctl suspend -- this one seemed to resume fine, but see next log

[dlazar (dlazar-linux-kernel-bugs)]

Created attachment 305255
dmesg-6.6.0-rc6-v2-suspend-2.txt

systemctl suspend -- network did not come back up from sleep

[mario.limonciello (mario.limonciello-linux-kernel-bugs)]

#54

1. I had a similar concern while adjusting v2. HOWEVER I think it's mitigated by the fact the DMI check ONLY runs in the failure path now. So even though it technically applies to all laptops from "82" family for Lenovo only ones with a problem will run it.

I would much rather we don't use a Zen heuristic because this has nothing to do with it being an AMD bug. It happens to be a BIOS bug on some Lenovo AMD systems.

I don't see this issue on Mendocino "reference hardware", nor on some other vendor's Mendocino hardware I've seen.

The only way we'll have a definitively correct list is if Mark (Lenovo) can provide it.

2. So you mean like the PIC malfunctions specifically on the quirked systems? Seems unlikely to me.

Another idea is that we can put the DMI check and writing to those registers somewhere else that runs "earlier" than probe_8259A() runs (earlier than device_initcall I guess). I'm not sure a good place for that though. Nothing in arch_initcall() stands out to me.

#56

OK thanks!

#59

I would say that this doesn't look fine. I see NVME IOMMU page faults on both suspend and resume, which means there is still "something" wrong. I *think* it's a different issue.

If it was just resume I would say it's similar to one that happened on a lot of other Lenovo systems which was another BIOS bug. By chance was your "time to resume" 10+ seconds?

Can you please try (separately) iommu=pt and amd_iommu=off?

[jwrdegoede (jwrdegoede-linux-kernel-bugs)]

(In reply to Mario Limonciello (AMD) from comment #61)
> #54
>
> 1. I had a similar concern while adjusting v2. HOWEVER I think it's
> mitigated by the fact the DMI check ONLY runs in the failure path now. So
> even though it technically applies to all laptops from "82" family for
> Lenovo only ones with a problem will run it.

So what I'm wondering is if the magic:

 outb(0x68, PIC_MASTER_IMR);
 outb(0x1, PIC_MASTER_IMR);

sequence is something AMD specific or some generic PIC enable sequence ?

If this is AMD specific then adding an boot_cpu_has(X86_FEATURE_ZEN) check seems to make sense.

If this is not AMD specific then I'm thinking to maybe make the fix much more generic (continued below)

> I would much rather we don't use a Zen heuristic because this has nothing to
> do with it being an AMD bug. It happens to be a BIOS bug on some Lenovo AMD
> systems.

See above, this may not be an AMD bug, but if the magic enable sequence is AMD specific then IMHO it should still be guarded by some AMD check.

> 2. So you mean like the PIC malfunctions specifically on the quirked
> systems? Seems unlikely to me.

It just generally seems like a good idea to re-check things to ensure that the enable sequence actually worked.

Also since the probe path is already poking the PIC_MASTER_IMR
register anyway doing the enable-sequence magic should not do anything on systems where there really is no PIC. I quick duckduckgo shows that the "Using NULL legacy PIC" message has only ever been seen in the wild (before now) on virtual machines either oracle vms (virtualbox?) or WSL2. Which presumably really don't have anything at the PIC io addresses in the specific VM config which triggers this.

Also note that such configs really should set the acpi-reduced-hw flag at which point probe_8259A() will not be called at all, because the legacy_pic pointer is already set to the NULL PIC early on then.

So what I'm thinking is that since the Lenovo BIOS setup does work under Windows we may see more of this and a more generic non DMI quirked fix would be better (with the retry).

Specifically I'm thinking about doing something like this:

        bool retried = false;

        /* ... */
        raw_spin_lock_irqsave(&i8259A_lock, flags);

retry:
        outb(0xff, PIC_SLAVE_IMR); /* mask all of 8259A-2 */
        outb(probe_val, PIC_MASTER_IMR);
        new_val = inb(PIC_MASTER_IMR);
        if (new_val != probe_val) {
                if (!retried) {
                        printk(KERN_INFO "No PIC trying to force enable PIC\n");
                        outb(0x68, PIC_MASTER_IMR);
                        outb(0x1, PIC_MASTER_IMR);
                        retried = true;
                        goto retry;
                }
                printk(KERN_INFO "Using NULL legacy PIC\n");
                ...

This is non AMD specific and should be harmless in the case where there really is no PIC:

If there really is nothing at the PIC_MASTER_IMR IO address this just does 4 extra writes (2 enable sequence + 2 for retry probe) + 1 extra read and then continues as before.

And if there actually is a PIC there it should now be enabled and things should work the same as...

[dlazar (dlazar-linux-kernel-bugs)]

Created attachment 305257
dmesg-6.6.0-rc6-v2-iommu-pt.txt

This is `systemctl suspend` with `iommu=pt`. The system appeared to resume fine (to my untrained eyes), and the resume time was under one second.

[dlazar (dlazar-linux-kernel-bugs)]

Created attachment 305258
dmesg-6.6.0-rc6-v2-amd-iommu-off.txt

This is `systemctl suspend` with `amd_iommu=off`. Again, resuming was fast and appeared to work ok.

[mario.limonciello (mario.limonciello-linux-kernel-bugs)]

#62

> See above, this may not be an AMD bug, but if the magic enable sequence is
> AMD specific then IMHO it should still be guarded by some AMD check.

No; it is not an AMD specific sequence. No magic here, this behaves like a standard PIC. If anything; "the magic base address" is specific to Lenovo's BIOS.

It's setting the 8259 vector base to 0x68 and then configuring it to run in 8086 mode. You can see some similar code examples here:

https://wiki.osdev.org/8259_PIC#Code_Examples

>Which presumably really don't have anything at the PIC io addresses in the
>specific VM config which triggers this.

I think we need the author of this commit to confirm. When we agree on a patch we need them in the "To" line.
e179f6914152 ("x86, irq, pic: Probe for legacy PIC and set legacy_pic appropriately")

>Also note that such configs really should set the acpi-reduced-hw flag at
>which point probe_8259A() will not be called at all, because the legacy_pic
>pointer is already set to the NULL PIC early on then.

ACPI hardware reduced has lots of other non-obvious implications.
As you noticed from the above testing suspend/resume got broken by using ACPI hardware reduced.

> This is non AMD specific and should be harmless in the case where there
> really is no PIC:

So the problem with this approach is that it's attempting to program a vector base that might not work for all systems. It /happens/ to work on these Lenovo ones, but I don't know about others.

#63

OK then there is a secondary issue with the BIOS configuration and usage of the IOMMU.
Let's treat it separately from this issue.

[mario.limonciello (mario.limonciello-linux-kernel-bugs)]

David can you check if reverting e179f6914152 fixes this issue? I would think it's still broken without the missing sequence.

[jwrdegoede (jwrdegoede-linux-kernel-bugs)]

(In reply to Mario Limonciello (AMD) from comment #65)
> #62
>
> It's setting the 8259 vector base to 0x68 and then configuring it to run in
> 8086 mode. You can see some similar code examples here:
>
> https://wiki.osdev.org/8259_PIC#Code_Examples

Thanks this is useful, this does show though that your init sequence seems to be wrong, the proper init sequence is:

        outb_pic(0x11, PIC_MASTER_CMD); /* ICW1: select 8259A-1 init, missing */
        outb_pic(ISA_IRQ_VECTOR(0), PIC_MASTER_IMR); /* your 0x68 */
        outb_pic(0x04, PIC_MASTER_IMR); /* missing */
        outb_pic(0x01, PIC_MASTER_IMR);

But this is all done by init_8259A(), so for the force-init you can
just call init_8259A(false). Except that you need to unlock the i8259A_lock first and relock it after.

This also illustrates why I think the retry is a good idea, that will test if a re-init is sufficient or if the PIC is disabled at some deeper level.

What I believe happens with your patch now is that since you make probe() succeed then either:

1. init_8259A() fixes things up properly, so it would be enough with the quirk to just make probe() succeed; or

2. The PIC really is disabled at some lower level, but Linux re-routes the IRQs to the IOAPIC anyway so this does not really matter. This is basically what my hack does, add mappings for the Legacy IRQs to make them work and then rely on the IOAPIC to handle them.

> I think we need the author of this commit to confirm. When we agree on a
> patch we need them in the "To" line.
> e179f6914152 ("x86, irq, pic: Probe for legacy PIC and set legacy_pic
> appropriately")

Ack.

> > This is non AMD specific and should be harmless in the case where there
> > really is no PIC:
>
> So the problem with this approach is that it's attempting to program a
> vector base that might not work for all systems. It /happens/ to work on
> these Lenovo ones, but I don't know about others.

See above, the kernel resets the vector base to its own value in
init_8259A(). But since it re-inits anyways the only thing which
we really need to do is make probe_8259A() not set the NULL PIC,
although I would prefer to make probe_8259A() call init_8259A()
when we hit the bug and then have it retry the probe.

[mario.limonciello (mario.limonciello-linux-kernel-bugs)]

> See above, the kernel resets the vector base to its own value in
init_8259A(). But since it re-inits anyways the only thing which
we really need to do is make probe_8259A() not set the NULL PIC,
although I would prefer to make probe_8259A() call init_8259A()
when we hit the bug and then have it retry the probe.

Ah it seems to me that given the kernel resets the vector base anyway; what the probe code as it stands today was "really" doing was checking if PIC was enabled by the BIOS already or not.

In that case I think that reverting e179f6914152 will fix this issue too.

> 1. init_8259A() fixes things up properly, so it would be enough with the
> quirk to just make probe() succeed; or

Another option is that we revert e179f6914152 and come up with another way to fix kexec for HyperV.

[dlazar (dlazar-linux-kernel-bugs)]

(In reply to Mario Limonciello (AMD) from comment #66)
> David can you check if reverting e179f6914152 fixes this issue? I would
> think it's still broken without the missing sequence.

The code has changed since 2014, when that commit was made, so it no longer reverts cleanly, and I had to do it by hand. The kernel is still compiling, and I'll post the results when that's done, but I wanted to confirm that you just want me to remove the probing code. This is what probe_8259A() looks in my test:

static int probe_8259A(void)
{
        unsigned long flags;
        raw_spin_lock_irqsave(&i8259A_lock, flags);

        outb(0xff, PIC_MASTER_IMR); /* mask all of 8259A-1 */
        outb(0xff, PIC_SLAVE_IMR); /* mask all of 8259A-2 */

        raw_spin_unlock_irqrestore(&i8259A_lock, flags);
        return nr_legacy_irqs();
}

Basically, it never touches legacy_pic.

> OK then there is a secondary issue with the BIOS configuration and usage of
> the IOMMU.
> Let's treat it separately from this issue.

Should I file a separate bug for that?

[mario.limonciello (mario.limonciello-linux-kernel-bugs)]

> This is what probe_8259A() looks in my test:
> Basically, it never touches legacy_pic.

Yup, you did what I had in mind. I think that will do the trick.

> Should I file a separate bug for that?

Yes, please file it separately.

[dlazar (dlazar-linux-kernel-bugs)]

Created attachment 305261
dmesg-6.6.0-rc6-rev-pic-probe.txt

(In reply to Mario Limonciello (AMD) from comment #66)
> David can you check if reverting e179f6914152 fixes this issue? I would
> think it's still broken without the missing sequence.

Indeed, reverting e179f6914152 makes the keyboard work. I've also attached the dmesg output (running with amd_iommu=off, and tried a suspend/resume, it worked).

> Yes, please file it separately.

Filed bug 218024.

[mario.limonciello (mario.limonciello-linux-kernel-bugs)]

> Indeed, reverting e179f6914152 makes the keyboard work. I've also attached
> the dmesg output (running with amd_iommu=off, and tried a suspend/resume, it
> worked).

Great, thanks!

So that reaffirms the above findings from Hans that the magic sequence isn't needed, it's just a way to claim that the PIC really is there even if the BIOS didn't program it.

The problem is the method that e179f6914152 used to detect the PIC doesn't work here.

I see three distinct approaches:

1. We revert e179f6914152 and store a variable to indicate whether a PIC was found and pass this information on to future kexec runs. This should fix the intent of e179f6914152

2. We revert e179f6914152 and as part of the shutdown sequence or kexec sequence disable the PIC.

3. We Go back to a quirk for detection for ZEN. All AMD Zen SoCs do have a PIC, whether it's programmed or not by BIOS appears to be a decision made by the BIOS vendor.

[mpearson-lenovo (mpearson-lenovo-linux-kernel-bugs)]

(In reply to Mario Limonciello (AMD) from comment #49)
> > So I'm assuming they're all affected by this bug. I'm not sure how to get
> > their DMI_PRODUCT_NAME, maybe they're already covered by "82Y" and "82X".
> > Anyone know?
>
> @Mark - can you help here?
>
> > Just a side note, regarding Mario's patch in #47, I've noticed that the
> exact
> > same BIOS update[0] applies to the following systems:
>
> Let's see if this patch works for the ones I quirked so far. If it does
> work, I'd rather move the DMI detection into the error path in which case we
> can probably change it to "82" to cover all the Lenovo systems from this
> year.

Does this link with the other Mendocino bug I just commented on (218024) or is it different?

For getting device IDs - suggest using PSREF. e.g:
https://psref.lenovo.com/Product/IdeaPad/IdeaPad_Slim_3_14AMN8
Go to the model tab and click on the filter icon next to "Machine Type" and that will show what models that platform has.
I couldn't find what an Ideapad 14s was though - doesn't seem to exist. I don't know that model line up at all I'm afraid.

[mario.limonciello (mario.limonciello-linux-kernel-bugs)]

They're two separate BIOS bugs on the same laptop (family).

[mario.limonciello (mario.limonciello-linux-kernel-bugs)]

Created attachment 305264
fix probing GPIO controller

I started a thread on LKML about this issue: https://lore.kernel.org/all/878r7z4kb4.ffs@tglx/.
Can someone affected please attach the requested items from tglx to this issue?

Also - I've got another patch that I think should help fix the GPIO controller probing. Can you try this (alone) and let me know if it helps that issue?

If it helps fix the keyboard as well, that would be a pleasant surprise.

[dlazar (dlazar-linux-kernel-bugs)]

Created attachment 305266
no-pic-probe.tar.gz

Attached is the information that Thomas Gleixner requested in the LKML thread[0].

> 1) dmesg with 'apic=verbose' on the command line
> 2) /proc/interrupts
> 3) /sys/kernel/debug/irq/irqs/{0..15}
>
> Two versions of that please:
>
> 1) Unpatched kernel
> 2) Patched kernel

[0] https://lore.kernel.org/all/878r7z4kb4.ffs@tglx/

[Kyrylo Budnyk (kirry36)]

Endre, i can't say exactly when it will be available on Ubuntu repositories but you always can build the kernel with patch by yourself.

[mario.limonciello (mario.limonciello-linux-kernel-bugs)]

I managed to force what I think the problem here is on one of my machines by hardcoding NULL PIC. The patch in #75 isn't enough to fix things.

Let's see what tglx has to say about this situation.

[Devis (devis5)]

Same problem with Lenovo IdeaPad Slim 3 15AMN8 with Ryzen 5 7520U

[Danielle Satterfield (dsatterfield)]

I can confirm that the referenced patch does resolve the issues on my V14 G4 AMN. Reading through everything though, I would not call the patch a fix as it seems to be treating a symptom of a deeper issue that is still being investigated.

[mario.limonciello (mario.limonciello-linux-kernel-bugs)]

I expect this series fixes this issue, can you guys please test.

https://lore.<email address hidden>/

[Quinten Van Ginderen (quintenvg1)]

Hey guys, how would one go ahead and get the patch into a current kernel? i'm willing to test it out and play with it for a while since i also have one of those lenovo ideapad lovechilds...

for now a wireless keyboard with integrated touchpad will do, but i'm really stoked to see this issue resolved. i got this from the mint forums, setting up pci=nocrs in grub default, but that doesn't fix the keyboard/trackpad issues instead it breaks the realtek wifi driver.

I've also noticed that indeed the F5/6 keys to change brightness work and the flight mode key.

Really stoked to see this bug being squashed =D

edit;

it's been a few hours, i've downloaded the patch file, and kernel 6.1.59. my pc is now building the patched kernel, after if it installs i will return with news, hopefully good news, this could provide a workaround until a better solution is found.

edit edit;
typing this on the laptop in question.
Touchpad and mouse are now working flawlessly with the patch appplied, i wonder when, or if it will ever be part of the upcoming kernels. Its not too hard to patch it in myself but it takes about 2 to 3 hours for my laptop to compile the kernel.

I consider myself a noob so anyone should be able to do it.

can i post a link to the kernel i patched? it's on my google drive. i'm not sure if i may post this link.

if you want the details, just ask me here, i will be checking every so often the upcoming few days.

happy patching :-p

[dlazar (dlazar-linux-kernel-bugs)]

Created attachment 305269
mario-v3.tar.gz

Attached is the dmesg, /proc/interrupts, and /sys/kernel/debug/irq/irqs from Mario's patch series linked from comment #78.

The keyboard and mouse work fine.

Likely unrelated, but maybe worth noting: the display was blank with this kernel (although I could see the desktop fine over VNC). I suspect it's caused by me using the kernel from: git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git. If I understand correctly, this is the tree we're supposed to use for submitting patches to the IRQ subsystem.

I'll also try the patch series against 6.6.0-rc6, which I've been using so far, to make sure the blank screen is not caused by the patch.

[mario.limonciello (mario.limonciello-linux-kernel-bugs)]

Thanks, all the interrupts look like I would expect.

> Likely unrelated, but maybe worth noting: the display was blank with this
> kernel (although I could see the desktop fine over VNC). I suspect it's
> caused by me using the kernel from:
> git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git. If I understand
> correctly, this is the tree we're supposed to use for submitting patches to
> the IRQ subsystem.

Yeah that should be unrelated; it's because the x86 tree didn't pick up other stuff in 6.6-rcs. I see the UCSI NULL pointer bug too which got fixed in later RCs.

> I'll also try the patch series against 6.6.0-rc6, which I've been using so
> far, to make sure the blank screen is not caused by the patch.

Thanks, hopefully everything should be good there. If you pair it with your NVME suspend fix patch (or iommu=pt) I'd expect you're in good shape.

[dlazar (dlazar-linux-kernel-bugs)]

(In reply to Mario Limonciello (AMD) from comment #80)
> > Likely unrelated, but maybe worth noting: the display was blank with this
> > kernel (although I could see the desktop fine over VNC). I suspect it's
> > caused by me using the kernel from:
> > git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git. If I understand
> > correctly, this is the tree we're supposed to use for submitting patches to
> > the IRQ subsystem.
>
> Yeah that should be unrelated; it's because the x86 tree didn't pick up
> other stuff in 6.6-rcs. I see the UCSI NULL pointer bug too which got fixed
> in later RCs.

Just to close the loop on this: everything works fine when running off of 6.6.0-rc6.

Thanks for your help, Mario. Looking forward to the discussion on the LKML.

[Endre Olah (endreolah68)]

Hi Quinten!

If you could explain how to do it and what to do to get it done. I would not mind to try, but never did such thing before.

In the discussions everybody is talking about it like others are on the same level, but some of us should start from scratch.

So if you can describe:

1. What tools are needed?
2. What source files needed?
3. What to be changed?
4. How to compile?
5. What should we do with the result of compile?

But, you can also share the link to your patch. Why not?

Thanks

     Endre

[Quinten Van Ginderen (quintenvg1)]

Hi Endre, to answer your many questions;

Take your time, read the entire thing first before doing anything.

First on i followed the link to the patch file: https://bugzilla.kernel.org/attachment.cgi?id=305236&action=diff

I downloaded the file by pressing 'view' button on that website.

second I went to the kernel.org documentation to find the following page: https://www.kernel.org/doc/html/v4.14/process/applying-patches.html

it explains how to patch a kernel

the required packages are (already in the apt command line :p) found here:https://people.montefiore.uliege.be/gain/courses/info0940/tutorials/tutorial3/compilekernel.html

sudo apt install build-essential libncurses5-dev libssl-dev flex libelf-dev bc bison gcc make bc dwarves libelf-dev

After this you should have: the patch file and all the packages required for patching, i personally used debian, but it's perfectly doable on ubuntu i guess.

then head over to kernel.org and download a kernel. i suggest as close as possible to the current kernel your os is running.
I personally downloaded 6.1.59, and will today try with another kernel to see if results are persistent.
i will test it on kernel 5.15.136.

decompress the downloaded kernel.
from here on all commands are issued as the root user.
aquire root privileges. (sudo su)
enter the decompressed kernel root directory so you are in linux-image-X/
then as root in that directory run patch -p1 < /path/to/downloaded_patch

let is complete
it should notify you when completed.
make menuconfig and just immediatly press save and follow default prompt.
then exit.

next up is to build the actual kernel itself, depending on the hard disk speed it could take 3 hours. my laptop (the ryzen 5-7520u combined with a usb ssd took about 3 and a half hours) you will need about 60Gb of hard disk/ssd space.
(today i will run it in my ramdisk as my desktop workstation has 64Gb of ram for ramdisk purposes :p) i will post total build time after it's done.

build the kernel by running: make
 seems simple enough, just type make and press enter, now get a coffee or a tea or go watch grass grow
 because now it will start building, it will take time.

if you are running ubuntu:
 make might complain about certificates (No rule to make target 'debian/canonical-certs.pem) , nothing
 too bad, !!just run the following!!

scripts/config --disable SYSTEM_TRUSTED_KEYS
scripts/config --disable SYSTEM_REVOCATION_KEYS

 and then run make again.
 if it asks something just press enter.

 then let it cook until al dente;

 then you will need to make the modules for the kernel i don't exactly know what this does but it's
 important.

run: sudo make modules
 this shouldn't take too long to complete.
 After this is done:
 congrats, you now have a patched kernel, but what to do with it now?

installing it in the current running system assuming all apt packages from compiling are present :
 make install

after a reboot into the installed kernel (grub should do this automatically as the version is later than the currently installed ones don't quote me on this :p) the keyboard/touchpad should be working now. if not then maybe a step failed.

what if i need to reinstall my di...

[Endre Olah (endreolah68)]

Hi Quinten,

thanks for sharing this. I was already trying to find out it by myself. But could not find the right solution. On the Ubuntu site they have a realy outdated thing, and many things not there where it should be.

Anyway, I read your steps and try to reproduce.

Rgds

     Endre

[Quinten Van Ginderen (quintenvg1)]

Hi Endre,

if you plan on following the steps, so yourself a favor and install debian12 to your laptop.

Then install the kernel onto that it's the only way i consistently got it working.

also specifically kernel 6.1.59.
i compiled 5.13.136 but it refused to install using make in zorin os 16. then i copied the current kernel config to a clean patched kernel directory, however then i got some weird efi handover error.

TLDR; use debian12 and kernel 6.1.59 for the least amount of hassle.

Kind regards;
as always happy patching :p

really hope there will be a solution in upcoming kernels
because this workaround is really cumbersome.

[Endre Olah (endreolah68)]

Hi Quinten,

first attempt on Ubuntu failed. I have downloaded the source package for the kernel 6.5.0-9.9, than I have followed the steps and compile went without error.

During install, however, it said that some modules are missing for 6.5.3 kernel, which is a bit weird.

Restarted the system, and the 6.5.3 kernel was there, but not as a default. The default remained the 6.5.0.

6.5.3 was not booting up. It jumped out to commandline.

I make anouther try now. Just cloning the latest Ubuntu git source package and try to patch that.

Update: This is weird. Make went well, no warnings. Make modules took longer then in the previous case. After install in grub the versions is 6.1.6. How can it be when I have cloned the latest Ubunu Mantic release source code, which should be 6.5.0-x.

System not loading, stuck at the same place as before.

Update: I have downloaded Debian 12 and the above mentioned kernel. Went through the steps. No errors during make or install. But the system does not load. It jumps out to the shell and stops. I have tried it in recover mode and it seems to get stuck on loading a module, but not clear what. I give up now and waiting for the official solution. But as I see the Ubuntu patch updates, it will take a while until we can see any progress on it.

Rgds

     Endre

[martin.fillon (martin.fillon-linux-kernel-bugs)]

Hey I have about 30 computers to dump with linux that have the problem from this bug. I saw that a patch has been made, but I just need confirmation (if possible) that it is in live linux kernel (at least the one from arch repositories).

[dlazar (dlazar-linux-kernel-bugs)]

(In reply to Martin FILLON from comment #82)
> but I just need confirmation (if
> possible) that it is in live linux kernel (at least the one from arch
> repositories).

Hans and Mario have identified the issue, but the fix is still being discussed, so the patch isn't yet available in any kernels. If you're fine compiling your own kernel, you can apply the patches in comment #78 (but see also bug 218024). Otherwise, hang tight, it's being worked on.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Danielle Satterfield (dsatterfield)]

Endre,

You probably need to copy over Ubuntu's stock kernel config file.

They normally can be found at '/boot/config-*'. Copy that to the kernel source root directory as '.config'.

[Quinten Van Ginderen (quintenvg1)]

Endre, install the kernel you built into said debian12 install, and then run that debian install on the laptop, and that should do the trick :)

Happy patching.
Im' not sure about copying the ubuntu kernel config, that got my compilation to crash on some 32 bit support stuff that i had not time to figure out.

Happy patching =p

[Endre Olah (endreolah68)]

Copying the stock config file to .config, did not cause compilation issues, but did not change anything at the end. I have the feeling that I was not downloading the right kernel source, even if I have followed the instruction on the Ubuntu website.

At my latest trial the installed kernel appears as 6.1.6+, what is weird as the stock kernel is 6.5.3 which appears as 6.5.0-9-generic. This is also not really clear why the coding is not following the official base kernel coding.

I am ok to wait for the solution, as without the background knowledge of what I am doing, this seems to be an impossible mission. And I am not Tom Cruise either. :D

So, I leave making the solution to the professionals. :D

[akshay (akshayvijapur)]

I have bought five new AMD laptops, now this issue is hitting me hard. What is the estimated time for fixing this ?

[Quinten Van Ginderen (quintenvg1)]

Hy Endre,

I'm making another attempt on ubuntu 20.04 with kernel 6.1.59.
i will report my findings here, if it doesn't work then i will attempt download the current kernel and test it on that.

Happy patching. =D

[Pius Leins (pi0us0)]

I am on arch and have the same issue with my V15 Gen4. Is it possible that the bug patch works for me too? I am somewhat suspicious of intervening in my kernel.

[Quinten Van Ginderen (quintenvg1)]

Hi,

It could be this patch works for arch too, i'm not certain, test it out in a VM first, patch the kernel install the patched kernel and if it boots then test the same steps on the baremetal.

I just compiled the kernel on ubuntu, 5.15.136 and will test if it works soon enough when i get the change, if i find the time for it i will write documentation on how you can patch the kernel up yourself, i personally used ubuntu 20.04.2 but might give it a shot in ubuntu 22.04.3 lts i just used VMS up until now but will test it out with external drives. i'm having an extremely busy week so i might have this sit up until next week.

This was an exciting breakthrough!
I'ill be back..
Happy patching!!

[Danielle Satterfield (dsatterfield)]

Pius,

My V14 G4 AMN seems to run Arch with a patched kernel just fine.

[Quinten Van Ginderen (quintenvg1)]

Has editing posts been disabled?

anyway i may have found why it might not have worked on ubuntu, i saw some errors during the compilation, basically install zstd from apt making the install line the following

sudo apt install build-essential libncurses5-dev libssl-dev flex libelf-dev bc bison gcc make dwarves zstd

i hope to see it work on ubuntu based distro's soon.

Happy patching :)

[Quinten Van Ginderen (quintenvg1)]

update;
I was able to compile it on zorin os (ubuntu 20.04.2) install it, and touchpad/keyboard works great.

I'm now switching to 22.04 to see how that fares, the closest available kernel for that distro is 6.5.XX coming from 6.2.XX that could potentially mess things up, but it's worth testing i guess.

Will report my findings here.
Currently every distro i eventually manage to install the patched kernel in seems to work fine.

Happy patching :p

[Quinten Van Ginderen (quintenvg1)]

Okay so far i tested:

debian12 on kernel 6.1.59
zorinos 16 on kernel 5.15.136
ubuntu 22.04.3 on kernel 6.5.3

All of the above configs work, if patched. stability was not tested because of lack of time. But i would guess it's stable enough.

Happy patching :)

[dlazar (dlazar-linux-kernel-bugs)]

Created attachment 305291
tglx-v4.tar.gz

I've tested Thomas' patch from [0], and the keyboard and mouse work fine.

[0] https://lore.kernel.org/all/87v8avawe0.ffs@tglx/

Attached is the usual debug info.

[Endre Olah (endreolah68)]

Hi Quinten,

I would do another try on the Ubuntu 22.04.3, but I am not sure if I have downloaded earlier the right kernel source. Could you send the link to the one you used?

What anout the .config file. Did you made the config as described earlier? So no stock config, just running the make menuconfig and that is all?

Any specific thing you did differently than above?

Thanks

     Endre

[Quinten Van Ginderen (quintenvg1)]

Hi Endre,

I have kinda made progress in regards of being able to patch and install a kernel.

First i'ill walk you through my assumptions,

1) you have the laptop and installed a debian/ubuntu based distro but the keyboard and touchpad don't work.
2) there is at least 64Gb of disk space available
3) you have the option of using the root user.

so first off.
Download the patch from above. https://bugzilla.kernel.org/attachment.cgi?id=305236
run uname -r, this will return your kernel version, download the closest matching version to that version from kernel.org, but remeber only download later versions, so you are not using a version older than your current kernel.

then run sudo apt install build-essential libncurses5-dev libssl-dev flex libelf-dev bc bison gcc make dwarves zstd

then unzip the downloaded kernel and enter it as root.
appply the patch by running the following as roon in the unzipped kernel directory: patch -p1 < path_to_patchfile

it will return something notifying it's ready.

then copy over the latest kernel config to the unzipped kernel this can be found in the /boot directory mine is /boot/config-6.1.59 for example. this is beceause i patched up the old kernel to this one. cp -r /boot/<latestkernel>.conf .config replace latestkernel with the one you find in /boot

then run make menuconfig as the root in the unzipped kernel.
immediatly just hit save and exit

run the following 2 lines as the root user in the unzipped kernel individually :) (been there done that)

scripts/config --disable SYSTEM_TRUSTED_KEYS
scripts/config --disable SYSTEM_REVOCATION_KEYS

then run make -j 16

and just hit enter whenever it prompts something, oh and get a coffee or a tea this is going to take a while :))

when it completes, run as root in the unzipped kernel: make modules_install
should'nt take too long
whenever it completes run make install
let it complete,
congrats kernel installed,
run; update-grub
verify that the version of the kernel is somewhere in it's output.

reboot, and pray.
if it boots up, run a uname -r, try it with the laptop's builtin keyboard.

if you've made it this far congrats.
you have now patched and installed a kernel.

above works on any distro i've tried. (zorin16 debian12 ubuntu2204)

[Jean-Marie Tschanz (jmtschanz)]

Thank you so much Quinten and every helper !

I have Ubuntu 22.04 on Lenovo V15 G4

When patching it returns

"*can't find file to patch at input line 15*

*From 7a62c5f716bb3a0dd119d6a6eb5994d906ffbb32 Mon Sep 17 00:00:00
2001From: Hans de Goede <<email address hidden> <email address hidden>>Date:
Mon, 16 Oct 2023 22:40:47 +0200Subject: [PATCH] i8259 hack allow mapping of
legacy IRQs with NULL PICSigned-off-by: Hans de Goede <<email address hidden>
<email address hidden>>--- arch/x86/kernel/i8259.c | 2 +- 1 file changed, 1
insertion(+), 1 deletion(-)diff --git a/arch/x86/kernel/i8259.c
b/arch/x86/kernel/i8259.cindex 30a55207c000..e34b2c884d70 100644---
a/arch/x86/kernel/i8259.c+++ b/arch/x86/kernel/i8259.c*"

which is strange as line 15 reads (in red in the file)

"*@@ -394,7 +394,7 @@ static int legacy_pic_probe(void)"*

On Wed, 25 Oct 2023 at 20:26, Quinten Van Ginderen <
<email address hidden>> wrote:

> Hi Endre,
>
> I have kinda made progress in regards of being able to patch and install
> a kernel.
>
> First i'ill walk you through my assumptions,
>
> 1) you have the laptop and installed a debian/ubuntu based distro but the
> keyboard and touchpad don't work.
> 2) there is at least 64Gb of disk space available
> 3) you have the option of using the root user.
>
> so first off.
> Download the patch from above.
> https://bugzilla.kernel.org/attachment.cgi?id=305236
> run uname -r, this will return your kernel version, download the closest
> matching version to that version from kernel.org, but remeber only
> download later versions, so you are not using a version older than your
> current kernel.
>
> then run sudo apt install build-essential libncurses5-dev libssl-dev
> flex libelf-dev bc bison gcc make dwarves zstd
>
> then unzip the downloaded kernel and enter it as root.
> appply the patch by running the following as roon in the unzipped kernel
> directory: patch -p1 < path_to_patchfile
>
> it will return something notifying it's ready.
>
> then copy over the latest kernel config to the unzipped kernel this can
> be found in the /boot directory mine is /boot/config-6.1.59 for example.
> this is beceause i patched up the old kernel to this one. cp -r
> /boot/<latestkernel>.conf .config replace latestkernel with the one you
> find in /boot
>
> then run make menuconfig as the root in the unzipped kernel.
> immediatly just hit save and exit
>
> run the following 2 lines as the root user in the unzipped kernel
> individually :) (been there done that)
>
> scripts/config --disable SYSTEM_TRUSTED_KEYS
> scripts/config --disable SYSTEM_REVOCATION_KEYS
>
> then run make -j 16
>
> and just hit enter whenever it prompts something, oh and get a coffee or
> a tea this is going to take a while :))
>
> when it completes, run as root in the unzipped kernel: make modules_install
> should'nt take too long
> whenever it completes run make install
> let it complete,
> congrats kernel installed,
> run; update-grub
> verify that the version of the kernel is somewhere in it's output.
>
> reboot, and pray.
> if it boots up, run a uname -r, try it with the laptop's builtin keyboard.
>
> if you've made it this far congrats.
> you have now patched and i...

[Quinten Van Ginderen (quintenvg1)]

Hi Jean-Marie,

Are you sure you have downloaded the right kernel from kernel.org?
And are you root in the unpacked kernel directory? the prompt should say something like /dir/linux-X.X.X/#

then run the patch command :)

Maybe this helped.

Happy patching =)

[Endre Olah (endreolah68)]

Hi Quinten,

I am typing this on the laptop keyboard (Lenovo IdeaPad Slim 3 15AMN8), thanks to your instructions. :D

You are a genious, Man! Thanks a lot.

But let me have one question. How can I prevent that this patched kernel is overwritten by a newer one at some point of time of an update?

Now I am happy, after successful patching.

Kind regards

       Endre

[Jean-Marie Tschanz (jmtschanz)]

Thanks for the kind answer

*Are you sure you have downloaded the right kernel from kernel.org
<http://kernel.org/>?*

Er.... What's this kernel download all about. ?.. Is my ubuntu 22.04 system
*obsolete* already... Why can't a patch meant to provide access to such
basic things like mouse/ touchpad rely on good old solid code fitting every
system for laptop...?

I hadn't bothered downloading the latest kernel, but it seems to be
important so I've dowloaded linux 6.5.9, and extracted with tar

*****NB: NOT simple to do... I had to look for the info then copy/paste the
code in order to launch the tar program... Nothing like a right click on
the file and it'd show the "extract to..." option... WHY !? I ask, because
the hardest part of the job = the extracting program, was done.
Programmers are obviously capable of adding a right-click option and save
time for everyone. The only reason I see is, linux would be designed for
those for whom the very use of a terminal is *the ultimate fun****

... I've entered "sudo su" and have that "#" root sign. My next step is
trying to figure out what "enter [the new kernel]" means. Do we enter
kernels ?

On Thu, 26 Oct 2023 at 09:52, Quinten Van Ginderen <
<email address hidden>> wrote:

> Hi Jean-Marie,
>
> Are you sure you have downloaded the right kernel from kernel.org?
> And are you root in the unpacked kernel directory? the prompt should say
> something like /dir/linux-X.X.X/#
>
> then run the patch command :)
>
> Maybe this helped.
>
> Happy patching =)
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/2034477
>
> Title:
> Keyboard and Touchpad Not Working in New Lenovo V15 Gen4 Laptop
>
> Status in ideapad-laptop:
> New
> Status in libinput package in Ubuntu:
> Confirmed
> Status in linux-signed-hwe-6.2 package in Ubuntu:
> Confirmed
> Status in linux-signed-oem-6.5 package in Ubuntu:
> Confirmed
> Status in Fedora:
> New
>
> Bug description:
> Hello.
>
> Ubuntu 22.04.3 with (later upgraded to kernel 6.2.0-32-generic) was
> installed in rewly purchased LENOVO V15 GEN4 AMN (AMD Ryzen 5 7520u)
> laptop and it was noticed that keyboard, touchpad and microphone are
> not working. The keyboard and touchpad work fine in BIOS setup and
> till GRUB (command line). It was found that when external devices such
> as keyboard, mouse and microphone are connected through USB and 3.5
> jack, respectively, these work just fine. To confirm these are not
> hardware problems, Microsoft Windows 11 (Home) was installed in
> another disk partition and observed all these working alright. Hence a
> bug is being reported to draw attention of the concerned team and I
> request them to refer this issue and do the needful at the earliest.
>
> Regards,
> Pradip Kumar Das
>
> ProblemType: Bug
> DistroRelease: Ubuntu 22.04
> Package: linux-image-6.2.0-32-generic 6.2.0-32.32~22.04.1
> ProcVersionSignature: Ubuntu 6.2.0-32.32~22.04.1-generic 6.2.16
> Uname: Linux 6.2.0-32-generic x86_64
> ApportVersion: 2.20.11-0ubuntu82.5
> Architecture: amd64
> CasperMD5CheckResult: pass
> CurrentDesktop: ubun...

[Endre Olah (endreolah68)]

Hi Jean-Marie,

Download an d isntall Ubunte 22.04 LTS as suggested be Quinten, than follow his instructions: Download teh kernel 6.5.9 on kernel.org. And it seems to work. I can guarantee as I am just typing this on my laptop keyboard, which did nkt work before. So again Quinten's istructions are here:

"I have kinda made progress in regards of being able to patch and install a kernel.

First i'ill walk you through my assumptions,

1) you have the laptop and installed a debian/ubuntu based distro but the keyboard and touchpad don't work.
2) there is at least 64Gb of disk space available
3) you have the option of using the root user.

so first off.
Download the patch from above. https://bugzilla.kernel.org/attachment.cgi?id=305236
run uname -r, this will return your kernel version, download the closest matching version to that version from kernel.org, but remeber only download later versions, so you are not using a version older than your current kernel.

then run sudo apt install build-essential libncurses5-dev libssl-dev flex libelf-dev bc bison gcc make dwarves zstd

then unzip the downloaded kernel and enter it as root.
appply the patch by running the following as roon in the unzipped kernel directory: patch -p1 < path_to_patchfile

it will return something notifying it's ready.

then copy over the latest kernel config to the unzipped kernel this can be found in the /boot directory mine is /boot/config-6.1.59 for example. this is beceause i patched up the old kernel to this one. cp -r /boot/<latestkernel>.conf .config replace latestkernel with the one you find in /boot

then run make menuconfig as the root in the unzipped kernel.
immediatly just hit save and exit

run the following 2 lines as the root user in the unzipped kernel individually :) (been there done that)

scripts/config --disable SYSTEM_TRUSTED_KEYS
scripts/config --disable SYSTEM_REVOCATION_KEYS

then run make -j 16

and just hit enter whenever it prompts something, oh and get a coffee or a tea this is going to take a while :))

when it completes, run as root in the unzipped kernel: make modules_install
should'nt take too long
whenever it completes run make install
let it complete,
congrats kernel installed,
run; update-grub
verify that the version of the kernel is somewhere in it's output.

reboot, and pray.
if it boots up, run a uname -r, try it with the laptop's builtin keyboard.

if you've made it this far congrats.
you have now patched and installed a kernel.

above works on any distro i've tried. (zorin16 debian12 ubuntu2204)"

Fully confirmed working on Ubuntu 22.04 LTS.

Kind regards

       Endre

[akshay (akshayvijapur)]

I am able to get my keyboard and mouse working after following the instructions provided here https://bugs.launchpad.net/ubuntu/+source/linux-signed-hwe-6.2/+bug/2034477/comments/64

Thanks @Quinten Van Ginderen (quintenvg1) @quintenvg1 for the help.

Akshay

[akshayvijapur (akshayvijapur-linux-kernel-bugs)]

I have tested the patch on my lenovo laptop. Now the keyboard and mouse seems to work fine. I am requesting team to release this fix as soon as possible to help many users who are stuck without keyboard and trackpad.

This the patch https://bugzilla.kernel.org/attachment.cgi?id=305236 and install instructions are here https://bugs.launchpad.net/ubuntu/+source/linux-signed-hwe-6.2/+bug/2034477/comments/64

Akshay

[Quinten Van Ginderen (quintenvg1)]

Guys important ceveat!! if you patch your own kernel, don't forget to sign it or virtualbox won't work...

I'm going to try to get it working today.

[Quinten Van Ginderen (quintenvg1)]

After some consideration, I can wait for the official patch.
I have another machine that can run VM's in the meantime. So i will not be going trough the trouble of signing and building external modules.

Happy Patching :-) learning something new every day.

[Jean-Marie Tschanz (jmtschanz)]

Thank you Ender,

Yes I have kernel 6.5.9 ready. But the instructions reads:

"then unzip the downloaded kernel and *enter it *as root"

I don't understand this instruction "enter it" .

On Fri, 27 Oct 2023 at 00:26, Endre Olah <email address hidden> wrote:

> Hi Jean-Marie,
>
> Download an d isntall Ubunte 22.04 LTS as suggested be Quinten, than
> follow his instructions: Download teh kernel 6.5.9 on kernel.org. And it
> seems to work. I can guarantee as I am just typing this on my laptop
> keyboard, which did nkt work before. So again Quinten's istructions are
> here:
>
> "I have kinda made progress in regards of being able to patch and
> install a kernel.
>
> First i'ill walk you through my assumptions,
>
> 1) you have the laptop and installed a debian/ubuntu based distro but the
> keyboard and touchpad don't work.
> 2) there is at least 64Gb of disk space available
> 3) you have the option of using the root user.
>
> so first off.
> Download the patch from above.
> https://bugzilla.kernel.org/attachment.cgi?id=305236
> run uname -r, this will return your kernel version, download the closest
> matching version to that version from kernel.org, but remeber only
> download later versions, so you are not using a version older than your
> current kernel.
>
> then run sudo apt install build-essential libncurses5-dev libssl-dev
> flex libelf-dev bc bison gcc make dwarves zstd
>
> then unzip the downloaded kernel and enter it as root.
> appply the patch by running the following as roon in the unzipped kernel
> directory: patch -p1 < path_to_patchfile
>
> it will return something notifying it's ready.
>
> then copy over the latest kernel config to the unzipped kernel this can
> be found in the /boot directory mine is /boot/config-6.1.59 for example.
> this is beceause i patched up the old kernel to this one. cp -r
> /boot/<latestkernel>.conf .config replace latestkernel with the one you
> find in /boot
>
> then run make menuconfig as the root in the unzipped kernel.
> immediatly just hit save and exit
>
> run the following 2 lines as the root user in the unzipped kernel
> individually :) (been there done that)
>
> scripts/config --disable SYSTEM_TRUSTED_KEYS
> scripts/config --disable SYSTEM_REVOCATION_KEYS
>
> then run make -j 16
>
> and just hit enter whenever it prompts something, oh and get a coffee or
> a tea this is going to take a while :))
>
> when it completes, run as root in the unzipped kernel: make modules_install
> should'nt take too long
> whenever it completes run make install
> let it complete,
> congrats kernel installed,
> run; update-grub
> verify that the version of the kernel is somewhere in it's output.
>
> reboot, and pray.
> if it boots up, run a uname -r, try it with the laptop's builtin keyboard.
>
> if you've made it this far congrats.
> you have now patched and installed a kernel.
>
> above works on any distro i've tried. (zorin16 debian12 ubuntu2204)"
>
> Fully confirmed working on Ubuntu 22.04 LTS.
>
> Kind regards
>
> Endre
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/2034477
>
> Title:
> Keyboard and Touchpad Not Working i...

[Endre Olah (endreolah68)]

Hi Jean-Marie,

"enter" is used in some comments and means either that you have to switch your current folder/directory when you are in terminal. So commands should be executed in the right folder, otherwise you might do something else or maybe nothing.

So "enter kernel" means to to cd to the decompressed folder of the kernel source files.

Quinten's description worked for me and it is not that difficult to follow.

Rgds

     Endre

[mario.limonciello (mario.limonciello-linux-kernel-bugs)]

#85, you tested the wrong patch.
Here is the correct patch:

https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git/commit/?h=x86/urgent&id=38d54ecfe293ed8bb26d05e6f0270a0aaa6656c6

[Mario Limonciello (superm1)]

Here is the upstream patch for this issue:

https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git/commit/?h=x86/urgent&id=38d54ecfe293ed8bb26d05e6f0270a0aaa6656c6

[Mario Limonciello (superm1)]

Please also take this fix at the same time for this same laptop. It will fix s2idle for it.

https://git.kernel.org/pub/scm/linux/kernel/git/pdx86/platform-drivers-x86.git/commit/?h=fixes&id=3bde7ec13c971445faade32172cb0b4370b841d9

[Jean-Marie Tschanz (jmtschanz)]

DONE...It works !!

I had to install flex + libelf-dev

I had to disable Secure Boot in order to avoid a "bad shim signature" error

So no clean way...

Thnak you for your precious time-saving answer 👍

On Fri, 27 Oct 2023 at 17:12, Endre Olah <email address hidden> wrote:

> Hi Jean-Marie,
>
> "enter" is used in some comments and means either that you have to
> switch your current folder/directory when you are in terminal. So
> commands should be executed in the right folder, otherwise you might do
> something else or maybe nothing.
>
> So "enter kernel" means to to cd to the decompressed folder of the
> kernel source files.
>
> Quinten's description worked for me and it is not that difficult to
> follow.
>
> Rgds
>
> Endre
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/2034477
>
> Title:
> Keyboard and Touchpad Not Working in New Lenovo V15 Gen4 Laptop
>
> Status in ideapad-laptop:
> New
> Status in libinput package in Ubuntu:
> Confirmed
> Status in linux-signed-hwe-6.2 package in Ubuntu:
> Confirmed
> Status in linux-signed-oem-6.5 package in Ubuntu:
> Confirmed
> Status in Fedora:
> New
>
> Bug description:
> Hello.
>
> Ubuntu 22.04.3 with (later upgraded to kernel 6.2.0-32-generic) was
> installed in rewly purchased LENOVO V15 GEN4 AMN (AMD Ryzen 5 7520u)
> laptop and it was noticed that keyboard, touchpad and microphone are
> not working. The keyboard and touchpad work fine in BIOS setup and
> till GRUB (command line). It was found that when external devices such
> as keyboard, mouse and microphone are connected through USB and 3.5
> jack, respectively, these work just fine. To confirm these are not
> hardware problems, Microsoft Windows 11 (Home) was installed in
> another disk partition and observed all these working alright. Hence a
> bug is being reported to draw attention of the concerned team and I
> request them to refer this issue and do the needful at the earliest.
>
> Regards,
> Pradip Kumar Das
>
> ProblemType: Bug
> DistroRelease: Ubuntu 22.04
> Package: linux-image-6.2.0-32-generic 6.2.0-32.32~22.04.1
> ProcVersionSignature: Ubuntu 6.2.0-32.32~22.04.1-generic 6.2.16
> Uname: Linux 6.2.0-32-generic x86_64
> ApportVersion: 2.20.11-0ubuntu82.5
> Architecture: amd64
> CasperMD5CheckResult: pass
> CurrentDesktop: ubuntu:GNOME
> Date: Wed Sep 6 08:04:42 2023
> InstallationDate: Installed on 2023-08-14 (22 days ago)
> InstallationMedia: Ubuntu 22.04.2 LTS "Jammy Jellyfish" - Release amd64
> (20230223)
> ProcEnviron:
> LANGUAGE=en_IN:en
> PATH=(custom, no user)
> XDG_RUNTIME_DIR=<set>
> LANG=en_IN
> SHELL=/bin/bash
> SourcePackage: linux-signed-hwe-6.2
> UpgradeStatus: No upgrade log present (probably fresh install)
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/ideapad-laptop/+bug/2034477/+subscriptions
>
>

[Mario Limonciello (superm1)]

Here's the fix in Linus' tree.

https://github.com/torvalds/linux/commit/128b0c9781c9f2651bea163cb85e52a6c7be0f9e

[Endre Olah (endreolah68)]

Hi Mario,

thanks for all your and the others efforts, who worked on the solution.

May I ask, how will we know that the fix is implemented in a distribution? Myself using Ubuntu 22.04, onwhich the patch was available and now I can use the keyboard, but I am afraid, if an update is popping up and installs a new kernel, I might loose the keyboard again.

Thanks

     Endre

[Pradip Kumar Das (pradipkumardas)]

Thanks Endre for bring up this question.

I had upgraded to 23.04 few weeks ago, and when I got to know that 23.10, which is considered as development version as of now and comes with kernel 6.5 and now available in the official site, I got the previous version upgraded to this latest yesterday, and found that keyboard and touchpad issue were not addressed in that kernel yet.

Many who are willing to get official update instead of applying patch by themselves, a response on tentative availability of fix through offical distribution will be helful.

Regards,

[Mario Limonciello (superm1)]

#166

Canonicals' kernel team needs to apply it. When they do there would be notification to this bug.

[Endre Olah (endreolah68)]

Thanks, Mario! Let's see how quick they are. Of course, respect to their works as they are working on an awesome distro.

[dragit.clemns (dragit.clemns-linux-kernel-bugs)]

The patch from #86 works without problems on my IdeaPad Slim 3 14AMN8 Ryzen 3 7320U.
Kernel: 6.5.9-arch2-1, EndeavourOS

A big thank you to Mario and Hans, I had a great first experience compiling my own kernel.

[ertugruluyar.personal (ertugruluyar.personal-linux-kernel-bugs)]

I have had the same problem for a long time. I have had the same problem for a long time. I'am using Lenovo Thinkbook 15 Gen2 ARE 20VG006XTX.
I only find these solutions worked:
https://lucraymond.net/2021/07/09/fixing-suspend-resume-on-lenovo-thinkbook-15-g2-are-laptop-with-amd-in-linux/
https://lucraymond.net/2022/10/04/linux-fixing-suspend-resume-on-amd-renoir-lenovo-thinkbook-g2-are-on-kernel-5-19-6-0-and-up/
memsleep_default=deep doesn't work. Only the amd_iommu=soft (for older kernels) and amd_iommu=off (kernel 6.0+) works. Is this (https://www.phoronix.com/news/Linux-6.6-Fixes-9-Lenovo-Laptop) fix this issue for my laptop? Please help me, I'am waiting very very long time for get rid of this issue.
This is the distro and kernel info that I using now:
OS: TUXEDO OS 2 x86_64
Host: LENOVO LNVNB161216
Kernel: 6.5.0-10006-tuxedo
Uptime: 16 mins
Packages: 2177 (dpkg), 67 (flatpak)
Shell: bash 5.1.16
Resolution: 1920x1080
DE: Plasma 5.27.8 (Wayland)
WM: kwin_wayland_wr
Theme: [Plasma], Breeze [GTK2/3]
Icons: [Plasma], breeze [GTK2/3]
Terminal: konsole
CPU: AMD Ryzen 5 4500U with Radeon Graphics (6) @ 2.375GHz
GPU: AMD ATI Renoir
Memory: 4314MiB / 6790MiB

[mario.limonciello (mario.limonciello-linux-kernel-bugs)]

You have a different issue, please open a separate one.

[Rayen Majoul (re0syst)]

how can I fix the problem in my Debian distribution with kernel 6.5

[Rayen Majoul (re0syst)]

I tried this github repository
https://github.com/hacklian/lenovo_82yu_dsdt_patches
It seems its made for arch linux

[ertugruluyar.personal (ertugruluyar.personal-linux-kernel-bugs)]

(In reply to Mario Limonciello (AMD) from comment #89)
> You have a different issue, please open a separate one.

Okay, but I have a similar issue that when I suspend my laptop and woke up the laptop, it is hanging on black screen and I have to force to shutdown laptop by hold pressed button on keyboard. Unless I add amd_iommu=off to my grub file. If it is really so different issue, I'am gonna open a new issue. Could you help me?

[suroto (suroto-linux-kernel-bugs)]

#86. Fixed.

Thank you very much to Mario, Hans

[Timo Aaltonen (tjaalton)]

skipping 5.15, this will be for 6.2 and up (plus oem-6.1)

[Quinten Van Ginderen (quintenvg1)]

@timo,

Just for some clarity, when you say no longer affects; does that mean you can get the kernel from kernel.org, install it and it will work just like that or does it mean when you install a kernel with a tool say mainline, it should work?

Or are we now waiting for the patch to be in the apt/dnf/pacman/zypper repos?
I'm a bit of a noob on the patching cycle.
For now i'm running a patched unsigned kernel 6.1.59. unfortunatly Virtualbox won't run unless the modules are signed however.
I was looking in how to sign it myself but it seems like certificate hassle.

Kind regards.
Quinten.

Happy patching :)

[ertugruluyar.personal (ertugruluyar.personal-linux-kernel-bugs)]

(In reply to Mario Limonciello (AMD) from comment #89)
> You have a different issue, please open a separate one.

I opened a different issue, but I was not sure which context should I choose for this. So I choose Kernel. If anyone help, I appreciate.

[ertugruluyar.personal (ertugruluyar.personal-linux-kernel-bugs)]

I forgot to add a link to the issue.
https://bugzilla.kernel.org/show_bug.cgi?id=218092

[Timo Aaltonen (tjaalton)]

I mean that there's no need to add hwe-* as being affected, because the parent kernel for hwe-6.2 is lunar, and it's already covered here.

And I'm actually reluctant to add these to oem kernels, because they'd need to be tested as well.

[Timo Aaltonen (tjaalton)]

..but instead wait until they get applied via upstream stable

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This bug is awaiting verification that the linux-oem-6.1/6.1.0-1026.26 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-jammy-linux-oem-6.1' to 'verification-done-jammy-linux-oem-6.1'. If the problem still exists, change the tag 'verification-needed-jammy-linux-oem-6.1' to 'verification-failed-jammy-linux-oem-6.1'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

[jwrdegoede (jwrdegoede-linux-kernel-bugs)]

Note the fixes for this have landed in 6.5.10 now, which should be available through your distro for Arch and Fedora users soon.

[emirmelikakts (emirmelikakts-linux-kernel-bugs)]

Thanks Mario and Hans for their precious efforts on this issue.

Is there any distros using 6.6 kernel officially? And if not, can I download the kernel from and apply it to my own OS?

- Emir

[jwrdegoede (jwrdegoede-linux-kernel-bugs)]

> Is there any distros using 6.6 kernel officially? And if not, can I download
> the kernel from and apply it to my own OS?

The fixes are also available 6.5.10 and 6.5.10 is currently available in Fedora updates-testing.

So you can install Fedora 38 with an USB keyboard + mouse attached to the laptop and then do:

sudo dnf --enablerepo=updates-testing update 'kernel*'

Reboot into the new kernel and then the keyboard and touchpad should work.

Note coming Tuesday Fedora 39 gets released, so you may want to wait till Tuesday and then download and install F39 right away to avoid having to upgrade later.

To download Fedora go to: https://fedoraproject.org/

[Endre Olah (endreolah68)]

Downloaded yesterday the 6.5.10 kernel and compiled it for Ubuntu 22.04 LTS. It works now without any patching needed. So I can confirm that it fixed in this kernel version.

Upgraded to Ubuntu 23.10 yesterday, which installed an earlier kernel, but left he 6.5.10 kernel on the machine too. Forcing to load this kernel automatically made it work under 23.10.

So for me the problem is solved until the official update is including the improved kernel.

[jnemeiksis (jnemeiksis-linux-kernel-bugs)]

I've tried install Fedora 39 from bootable ISO and there are not working mouse and keyboard in installation area.

Lenovo V15 G4 AMN

[jwrdegoede (jwrdegoede-linux-kernel-bugs)]

> I've tried install Fedora 39 from bootable ISO and there are not working
> mouse and keyboard in installation area. Lenovo V15 G4 AMN

This is expected, F39's images use 6.5.6 as kernel and the fixes for this are only available in 6.5.10 and later.

So you need to either install 6.5.10 for your current distribution (most distributions have some mainline kernel repository somewhere) or install Fedora 39 using an usb keyboard + mouse and then install all updates and after that things should work.

[jnemeiksis (jnemeiksis-linux-kernel-bugs)]

yeah, after update kernel everything okay. Thanks!

[Quinten Van Ginderen (quintenvg1)]

I can confirm too, i'm just waiting for the latest kernels to show up in mainline, so i can install them and have them signed. So that virtualbox will finally worrk too.

It's been an honor to test along you people!!

Happy patching. :)

[Endre Olah (endreolah68)]

I have switched to Debian and downloaded the latest kernel from kernel.org. As the current kernel on Debian was 6.1.0-13, I have selected the 6.1.62 kernel source. I have followed the steps and Quinten written earlier, except the patching steps as it is not needed anymore and after installing the compiled kernel, it works perfectly.

[Quinten Van Ginderen (quintenvg1)]

Hey Guys,

I have recently installed the zabbly kernel in debian, it is based on version 6.5.10, no patching is needed the modules are signed, i'd say we're set until ubuntu and ubuntu based dsitro's are on kernel 6.5.10 or newer.

Happy patching, even tho patching for this bug is probably nearly done at this point :)

Kind regards.
Quinten.

[Martin Green (greenm)]

I also have same problem on Lenovo Yoga Duet 7 13IML05 model 82AS , Intel Core i7-10510U

[Nerio Miguel Rincon Rodriguez (n3rio)]

Hi Guys,

I just finished installing the kernel 6.5.10, and still no keyboard, no trackpad/mouse, I'm running Ubuntu 23.10 x86_64 on a 82KC Lenovo V14 G2 ALC, this is my CPU AMD Ryzen 7 5700U with Radeon G, any ideas? I take anything from changing OS to try the same distro different version.

[Mario Limonciello (superm1)]

Open a new bug at kernel bugzilla and attach kernel log, acpidump and dmidecode.

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This bug is awaiting verification that the linux-oem-6.5/6.5.0-1008.8 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-jammy-linux-oem-6.5' to 'verification-done-jammy-linux-oem-6.5'. If the problem still exists, change the tag 'verification-needed-jammy-linux-oem-6.5' to 'verification-failed-jammy-linux-oem-6.5'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

[Nerio Miguel Rincon Rodriguez (n3rio)]

Hi,

Does the last messages indicates that the fix will be available in an upcoming update for the OS?

[Danielle Satterfield (dsatterfield)]

Maybe not for your G2 ALC. This ticket was specifically focused on the G4 AMN laptops.

[Timo Aaltonen (tjaalton)]

for the generic distro kernel it'll get fixed via normal stable backports, which at this point means the fix is available to download in February next year, at the earliest

[Nerio Miguel Rincon Rodriguez (n3rio)]

Hi people,

I guess it would be a good idea to let you know that the problem on Lenovo V14 G2 ALC is now fixed, didn't do anything, it just worked after an update.

[Oliver Hamann (theageman)]

For all Linux beginners that don't want to compile their own kernel:

It seems that it's already possible to download and install a pre-compiled kernel via GUI:

Simply follow these instructions: https://9to5linux.com/you-can-now-install-linux-kernel-6-5-on-ubuntu-heres-how

You'll need to enter the following commands into your terminal:

sudo add-apt-repository ppa:cappelikan/ppa
sudo apt update && sudo apt full-upgrade
sudo apt install -y mainline

After that is done you will find an application called "Mainline Kernels" in your desktop-environment.
Start that application and install a new Kernel (I installed kernel 6.5.11).

After you installed that kernel, enter this line in your terminal:

sudo apt --fix-broken install

Restart your notebook and enjoy your built-in keyboard and trackpad.

PLEASE NOTE: Like yourself I'm a linux-noob. I can only confirm that this works on my configuration:
Ubuntu 22.04/Gnome 42/Kernel 6.5.11

But in my case it works like a charm.

Good luck!
theageman

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux-oem-6.1 - 6.1.0-1026.26

---------------
linux-oem-6.1 (6.1.0-1026.26) jammy; urgency=medium

  * jammy/linux-oem-6.1: 6.1.0-1026.26 -proposed tracker (LP: #2041950)

  * Packaging resync (LP: #1786013)
    - [Packaging] update annotations scripts
    - [Packaging] update helper scripts

  * Fix system suspend problem for Cirrus CS35L41 HDA codec on HP ZBook Fury 16
    G9 (LP: #2042060)
    - ALSA: hda: cs35l41: Override the _DSD for HP Zbook Fury 17 G9 to correct
      boost type
    - ALSA: hda: cs35l41: Use reset label to get GPIO for HP Zbook Fury 17 G9
    - ALSA: hda: cs35l41: Assert reset before system suspend
    - ALSA: hda: cs35l41: Assert Reset prior to de-asserting in probe and system
      resume
    - ALSA: hda: cs35l41: Run boot process during resume callbacks
    - ALSA: hda: cs35l41: Force a software reset after hardware reset
    - ALSA: hda: cs35l41: Do not unload firmware before reset in system suspend
    - ALSA: hda: cs35l41: Check CSPL state after loading firmware
    - ASoC: cs35l41: Detect CSPL errors when sending CSPL commands

  * Support speaker mute hotkey for Cirrus CS35L41 HDA codec (LP: #2039151)
    - ALSA: hda: cs35l41: Support systems with missing _DSD properties
    - ALSA: hda: cs35l41: Fix the loop check in cs35l41_add_dsd_properties
    - ALSA: hda: cs35l41: Add notification support into component binding
    - ALSA: hda/realtek: Support ACPI Notification framework via component binding
    - ALSA: hda: cs35l41: Support mute notifications for CS35L41 HDA
    - ALSA: hda: cs35l41: Add read-only ALSA control for forced mute

  * Keyboard and Touchpad Not Working in New Lenovo V15 Gen4 Laptop
    (LP: #2034477)
    - x86/i8259: Skip probing when ACPI/MADT advertises PCAT compatibility
    - platform/x86: Add s2idle quirk for more Lenovo laptops

  * CVE-2023-31085
    - ubi: Refuse attaching if mtd's erasesize is 0

  * CVE-2023-5178
    - nvmet-tcp: Fix a possible UAF in queue intialization setup

  * CVE-2023-5090
    - x86: KVM: SVM: always update the x2avic msr interception

  * CVE-2023-5717
    - perf: Disallow mis-matched inherited group reads

 -- Timo Aaltonen <email address hidden> Wed, 01 Nov 2023 14:45:18 +0200

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux-oem-6.5 - 6.5.0-1008.8

---------------
linux-oem-6.5 (6.5.0-1008.8) jammy; urgency=medium

  * jammy/linux-oem-6.5: 6.5.0-1008.8 -proposed tracker (LP: #2041878)

  * Packaging resync (LP: #1786013)
    - [Packaging] resync git-ubuntu-log
    - [Packaging] resync update-dkms-versions helper

  * System hang after unplug/plug DP monitor with AMD W7500 card (LP: #2042912)
    - SAUCE: drm/amd/pm: Fix error of MACO flag setting code

  * Fix after-suspend-mediacard/sdhc-insert test failed (LP: #2042500)
    - SAUCE: PCI/ASPM: Add back L1 PM Substate save and restore

  * Keyboard and Touchpad Not Working in New Lenovo V15 Gen4 Laptop
    (LP: #2034477)
    - x86/i8259: Skip probing when ACPI/MADT advertises PCAT compatibility
    - platform/x86: Add s2idle quirk for more Lenovo laptops

  * Fix RPL-U CPU C-state alway keep at C3 when system run PHM with idle screen
    on (LP: #2042385)
    - SAUCE: r8169: Add quirks to enable ASPM on Dell platforms

  * Fix system suspend problem for Cirrus CS35L41 HDA codec on HP ZBook Fury 16
    G9 (LP: #2042060)
    - ALSA: hda: cs35l41: Override the _DSD for HP Zbook Fury 17 G9 to correct
      boost type
    - ALSA: hda: cs35l41: Use reset label to get GPIO for HP Zbook Fury 17 G9
    - ALSA: hda: cs35l41: Assert reset before system suspend
    - ALSA: hda: cs35l41: Assert Reset prior to de-asserting in probe and system
      resume
    - ALSA: hda: cs35l41: Run boot process during resume callbacks
    - ALSA: hda: cs35l41: Force a software reset after hardware reset
    - ALSA: hda: cs35l41: Do not unload firmware before reset in system suspend
    - ALSA: hda: cs35l41: Check CSPL state after loading firmware
    - ASoC: cs35l41: Detect CSPL errors when sending CSPL commands

  * Miscellaneous Ubuntu changes
    - [Packaging] Add ppa2 to getabis

  [ Ubuntu: 6.5.0-13.13 ]

  * mantic/linux: 6.5.0-13.13 -proposed tracker (LP: #2042652)
  * arm64 atomic issues cause disk corruption (LP: #2042573)
    - locking/atomic: scripts: fix fallback ifdeffery

  [ Ubuntu: 6.5.0-11.11 ]

  * mantic/linux: 6.5.0-11.11 -proposed tracker (LP: #2041879)
  * CVE-2023-31085
    - ubi: Refuse attaching if mtd's erasesize is 0
  * CVE-2023-4244
    - netfilter: nft_set_rbtree: skip sync GC for new elements in this transaction
  * CVE-2023-5633
    - drm/vmwgfx: Keep a gem reference to user bos in surfaces
  * CVE-2023-5345
    - fs/smb/client: Reset password pointer to NULL
  * CVE-2023-5090
    - x86: KVM: SVM: always update the x2avic msr interception
  * Packaging resync (LP: #1786013)
    - [Packaging] update helper scripts

 -- Timo Aaltonen <email address hidden> Fri, 10 Nov 2023 13:04:39 +0200