Microsoft Surface Laptop 4 keyboard not available during early boot (can't enter disk unlock password)

[Tom Reynolds]

[ Impact ]

With 22.04 LTS, fully updated, freshly built initrd's for both 5.15 (linux-generic-hwe-22.04) and 5.19 (linux-generic-hwe-22.04-edge) kernels, the embedded keyboard of a Microsoft Surface 4 Laptop model 1950 (Intel Gen 11), namely "Microsoft Surface 045E:09AE Keyboard", is not available during early boot, such as for entering a full disk encryption passphrase. Just pressing enter at said prompt has no effect. Connecting an external keyboard via USB and typing the passphrase from there works around it.

The keyboard works fine on the fully booted system.

Secure boot is disabled on this system.

[ Test Plan ]

The -generic kernel and probably all kernel flavors contain the needed driver (on amd64). On mantic systems, you can verify the current initrd doesn't contain these modules by decompressing the initramfs and listing its files:
$ lsinitramfs /boot/initrd.img-6.5.0-10-generic | grep 'surface_aggregator_registry\|intel-lpss-pci'

After installing the updated initramfs-tools package, your current initramfs should be automatically rebuilt and pick up the kernel modules:
$ lsinitramfs /boot/initrd.img-6.5.0-10-generic | grep 'surface_aggregator_registry\|intel-lpss-pci'
usr/lib/modules/6.5.0-10-generic/kernel/drivers/mfd/intel-lpss-pci.ko.zst
usr/lib/modules/6.5.0-10-generic/kernel/drivers/platform/surface/surface_aggregator_registry.ko.zst

Test case on Microsoft Surface Laptop 4: Remove all previous workarounds. Install the fixed initramfs-tools version. Then the keyboard should work during boot to enter the passphrase.

[ Where problems could occur ]

This is making the initramfs slightly bigger. In my testing, both added were 11,868 bytes in total (or 0.01 % on a 102 MB initramfs).

[ Apport report ]

ProblemType: Bug
ApportVersion: 2.20.11-0ubuntu82.3
Architecture: amd64
CasperMD5CheckResult: pass
CurrentDesktop: ubuntu:GNOME
Date: Mon Feb 13 00:53:34 2023
Dependencies:
 adduser 3.118ubuntu5
 apt 2.4.8
 apt-utils 2.4.8
 busybox-initramfs 1:1.30.1-7ubuntu3
 ca-certificates 20211016ubuntu0.22.04.1
 coreutils 8.32-4.1ubuntu1
 cpio 2.13+dfsg-7
 debconf 1.5.79ubuntu1
 debconf-i18n 1.5.79ubuntu1
 dmsetup 2:1.02.175-2.1ubuntu4
 dpkg 1.21.1ubuntu2.1
 gcc-12-base 12.1.0-2ubuntu1~22.04
 gettext-base 0.21-4ubuntu4
 gpgv 2.2.27-3ubuntu2.1
 grub-common 2.06-2ubuntu7.1
 grub-gfxpayload-lists 0.7
 grub-pc 2.06-2ubuntu7.1
 grub-pc-bin 2.06-2ubuntu7.1
 grub2-common 2.06-2ubuntu7.1
 init-system-helpers 1.62
 initramfs-tools 0.140ubuntu13.1
 initramfs-tools-bin 0.140ubuntu13.1
 initramfs-tools-core 0.140ubuntu13.1
 klibc-utils 2.0.10-4
 kmod 29-1ubuntu1
 libacl1 2.3.1-1
 libapt-pkg6.0 2.4.8
 libattr1 1:2.5.1-1build1
 libaudit-common 1:3.0.7-1build1
 libaudit1 1:3.0.7-1build1
 libblkid1 2.37.2-4ubuntu3
 libbrotli1 1.0.9-2build6
 libbz2-1.0 1.0.8-5build1
 libc6 2.35-0ubuntu3.1
 libcap-ng0 0.7.9-2.2build3
 libcap2 1:2.44-1build3
 libcom-err2 1.46.5-2ubuntu1.1
 libcrypt1 1:4.4.27-1
 libdb5.3 5.3.28+dfsg1-0.8ubuntu3
 libdevmapper1.02.1 2:1.02.175-2.1ubuntu4
 libefiboot1 37-6ubuntu2
 libefivar1 37-6ubuntu2
 libffi8 3.4.2-4
 libfreetype6 2.11.1+dfsg-1ubuntu0.1
 libfuse3-3 3.10.5-1build1
 libgcc-s1 12.1.0-2ubuntu1~22.04
 libgcrypt20 1.9.4-3ubuntu3
 libgmp10 2:6.2.1+dfsg-3ubuntu1
 libgnutls30 3.7.3-4ubuntu1.1
 libgpg-error0 1.43-3
 libgssapi-krb5-2 1.19.2-2ubuntu0.1
 libhogweed6 3.7.3-1build2
 libidn2-0 2.3.2-2build1
 libk5crypto3 1.19.2-2ubuntu0.1
 libkeyutils1 1.6.1-2ubuntu3
 libklibc 2.0.10-4
 libkmod2 29-1ubuntu1
 libkrb5-3 1.19.2-2ubuntu0.1
 libkrb5support0 1.19.2-2ubuntu0.1
 liblocale-gettext-perl 1.07-4build3
 liblz4-1 1.9.3-2build2
 liblzma5 5.2.5-2ubuntu1
 libmount1 2.37.2-4ubuntu3
 libnettle8 3.7.3-1build2
 libnsl2 1.3.0-2build2
 libp11-kit0 0.24.0-6build1
 libpam-modules 1.4.0-11ubuntu2.3
 libpam-modules-bin 1.4.0-11ubuntu2.3
 libpam0g 1.4.0-11ubuntu2.3
 libpcre2-8-0 10.39-3ubuntu0.1
 libpng16-16 1.6.37-3build5
 libseccomp2 2.5.3-2ubuntu2
 libselinux1 3.3-1build2
 libsemanage-common 3.3-1build2
 libsemanage2 3.3-1build2
 libsepol2 3.3-1build1
 libsmartcols1 2.37.2-4ubuntu3
 libssl3 3.0.2-0ubuntu1.8
 libstdc++6 12.1.0-2ubuntu1~22.04
 libsystemd0 249.11-0ubuntu3.6
 libtasn1-6 4.18.0-4build1
 libtext-charwidth-perl 0.04-10build3
 libtext-iconv-perl 1.7-7build3
 libtext-wrapi18n-perl 0.06-9
 libtinfo6 6.3-2
 libtirpc-common 1.3.2-2ubuntu0.1
 libtirpc3 1.3.2-2ubuntu0.1
 libudev1 249.11-0ubuntu3.6
 libunistring2 1.0-1
 libuuid1 2.37.2-4ubuntu3
 libxxhash0 0.8.1-1
 libzstd1 1.4.8+dfsg-3build1
 linux-base 4.5ubuntu9
 linux-modules-5.19.0-28-generic 5.19.0-28.29~22.04.1
 logsave 1.46.5-2ubuntu1.1
 lsb-base 11.1.0ubuntu4
 mount 2.37.2-4ubuntu3
 openssl 3.0.2-0ubuntu1.8
 os-prober 1.79ubuntu2
 passwd 1:4.8.1-2ubuntu2.1
 perl-base 5.34.0-3ubuntu1.1
 sensible-utils 0.0.17
 systemd-hwe-hwdb 249.11.2
 tar 1.34+dfsg-1build3
 ubuntu-keyring 2021.03.26
 ucf 3.0043
 udev 249.11-0ubuntu3.6
 util-linux 2.37.2-4ubuntu3
 uuid-runtime 2.37.2-4ubuntu3
 zlib1g 1:1.2.11.dfsg-2ubuntu9.2
 zstd 1.4.8+dfsg-3build1
DistroRelease: Ubuntu 22.04
InstallationDate: Installed on 2023-02-12 (0 days ago)
InstallationMedia: Ubuntu 22.04.1 LTS "Jammy Jellyfish" - Release amd64 (20220809.1)
Package: linux-image-5.19.0-28-generic 5.19.0-28.29~22.04.1
PackageArchitecture: amd64
ProcCpuinfoMinimal:
 processor : 7
 vendor_id : GenuineIntel
 cpu family : 6
 model : 140
 model name : 11th Gen Intel(R) Core(TM) i7-1185G7 @ 3.00GHz
 stepping : 1
 microcode : 0xa4
 cpu MHz : 4643.410
 cache size : 12288 KB
 physical id : 0
 siblings : 8
 core id : 3
 cpu cores : 4
 apicid : 7
 initial apicid : 7
 fpu : yes
 fpu_exception : yes
 cpuid level : 27
 wp : yes
 flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l2 invpcid_single cdp_l2 ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid rdt_a avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb intel_pt avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves split_lock_detect dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp hwp_pkg_req avx512vbmi umip pku ospke avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg tme avx512_vpopcntdq rdpid movdiri movdir64b fsrm avx512_vp2intersect md_clear ibt flush_l1d arch_capabilities
 vmx flags : vnmi preemption_timer posted_intr invvpid ept_x_only ept_ad ept_1gb flexpriority apicv tsc_offset vtpr mtf vapic ept vpid unrestricted_guest vapic_reg vid ple shadow_vmcs pml ept_mode_based_exec tsc_scaling
 bugs : spectre_v1 spectre_v2 spec_store_bypass swapgs eibrs_pbrsb
 bogomips : 5990.40
 clflush size : 64
 cache_alignment : 64
 address sizes : 39 bits physical, 48 bits virtual
 power management:
ProcEnviron:
 LC_TIME=de_DE.UTF-8
 LC_MONETARY=de_DE.UTF-8
 TERM=xterm-256color
 PATH=(custom, no user)
 LC_ADDRESS=de_DE.UTF-8
 XDG_RUNTIME_DIR=<set>
 LANG=en_US.UTF-8
 LC_TELEPHONE=de_DE.UTF-8
 SHELL=/bin/bash
 LC_NAME=de_DE.UTF-8
 LC_MEASUREMENT=de_DE.UTF-8
 LC_IDENTIFICATION=de_DE.UTF-8
 LC_NUMERIC=de_DE.UTF-8
 LC_PAPER=de_DE.UTF-8
ProcVersionSignature: Ubuntu 5.19.0-28.29~22.04.1-generic 5.19.17
SourcePackage: linux-signed-hwe-5.19
Tags: wayland-session jammy
Uname: Linux 5.19.0-28-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
_MarkForUpload: True
---
ProblemType: Bug
ApportVersion: 2.20.11-0ubuntu82.3
Architecture: amd64
AudioDevicesInUse:
 USER PID ACCESS COMMAND
 /dev/snd/controlC0: mona 5835 F.... pulseaudio
CRDA: N/A
CasperMD5CheckResult: pass
DistroRelease: Ubuntu 22.04
InstallationDate: Installed on 2023-02-12 (0 days ago)
InstallationMedia: Ubuntu 22.04.1 LTS "Jammy Jellyfish" - Release amd64 (20220809.1)
MachineType: Microsoft Corporation Surface Laptop 4
Package: linux (not installed)
ProcFB: 0 i915drmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.19.0-28-generic root=/dev/mapper/vgubuntu-root ro quiet splash vt.handoff=7
ProcVersionSignature: Ubuntu 5.19.0-28.29~22.04.1-generic 5.19.17
RelatedPackageVersions:
 linux-restricted-modules-5.19.0-28-generic N/A
 linux-backports-modules-5.19.0-28-generic N/A
 linux-firmware 20220329.git681281e4-0ubuntu3.9
Tags: jammy
Uname: Linux 5.19.0-28-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: adm cdrom dip lpadmin lxd plugdev sambashare sudo
_MarkForUpload: True
dmi.bios.date: 06/17/2021
dmi.bios.vendor: Microsoft Corporation
dmi.bios.version: 10.300.141
dmi.board.name: Surface Laptop 4
dmi.board.vendor: Microsoft Corporation
dmi.chassis.type: 9
dmi.chassis.vendor: Microsoft Corporation
dmi.modalias: dmi:bvnMicrosoftCorporation:bvr10.300.141:bd06/17/2021:svnMicrosoftCorporation:pnSurfaceLaptop4:pvr124I00061T000M0100000D0B05F1C01P16Y1U00S0:rvnMicrosoftCorporation:rnSurfaceLaptop4:rvr:cvnMicrosoftCorporation:ct9:cvr:skuSurface_Laptop_4_19501951:
dmi.product.family: Surface
dmi.product.name: Surface Laptop 4
dmi.product.sku: Surface_Laptop_4_1950:1951
dmi.product.version: 124I:00061T:000M:0100000D:0B:05F:1C:01P:16Y:1U:00S:0
dmi.sys.vendor: Microsoft Corporation

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 2007050

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Tom Reynolds (tomreyn)]

/proc/bus/input/devices

[Tom Reynolds (tomreyn)]

apport information

[Tom Reynolds (tomreyn)]

apport information

[Tom Reynolds (tomreyn)]

apport information

[Tom Reynolds (tomreyn)]

apport information

[Tom Reynolds (tomreyn)]

apport information

[Tom Reynolds (tomreyn)]

apport information

[Tom Reynolds (tomreyn)]

apport information

[Tom Reynolds (tomreyn)]

apport information

[Tom Reynolds (tomreyn)]

apport information

[Tom Reynolds (tomreyn)]

apport information

[Tom Reynolds (tomreyn)]

apport information

[Tom Reynolds (tomreyn)]

apport information

[Tom Reynolds (tomreyn)]

apport information

[Tom Reynolds (tomreyn)]

apport information

[Tom Reynolds (tomreyn)]

apport information

[Tom Reynolds (tomreyn)]

apport information

[Tom Reynolds (tomreyn)]

apport information

[Tom Reynolds (tomreyn)]

apport information

[Tom Reynolds (tomreyn)]

apport information

[Tom Reynolds (tomreyn)]

Thanks to oerheks for pointing me to a related (though AMD there) report at
https://github.com/linux-surface/linux-surface/issues/839#issuecomment-1140287888

Placing the following in /etc/initramfs-tools/modules
#-- Starts here --
surface_hid_core
surface_hid
surface_aggregator_registry
surface_aggregator
pinctrl_tigerlake
8250_dw
#-- Ends here --

then running
  sudo update-initramfs -c -k $(uname -r)
and finally rebooting works around it for me.

It appears that whatever detects the modules to be included in the (default) "MOST" set when generating an initrd will need updating.

(It may be possible to reduce the above list of modules to specifically include in the initrd can be reduced, yet - let me know if you want me to work this out.)

[Daniel van Vugt (vanvugt)]

Confirmed. The workaround I've been using so far in /etc/initramfs-tools/modules is:

surface_aggregator
surface_aggregator_registry
surface_hid_core
surface_hid
8250_dw
intel_lpss
intel_lpss_pci
pinctrl_tigerlake

but comment #22 suggests two of those aren't necessary.

[Benjamin Drung (bdrung)]

I checked the included modules on Ubuntu 23.10 (linux 6.5.0-10-generic) and following modules are included:

8250_dw
pinctrl_tigerlake (pinctrl-tigerlake.ko)
surface_aggregator
surface_hid
surface_hid_core

Following modules are not included, but will be included by the fix for bug #2042710:

intel_lpss
intel_lpss_pci

Following module needs to be included (also dracut is affected):

surface_aggregator_registry

[Benjamin Drung (bdrung)]

Forwarded adding surface_aggregator_registry to dracut: https://github.com/dracutdevs/dracut/pull/2558

[Benjamin Drung (bdrung)]

initramfs-tools 0.142ubuntu15.1 debdiff for mantic attached.

[Launchpad Janitor (janitor)]

This bug was fixed in the package initramfs-tools - 0.142ubuntu16

---------------
initramfs-tools (0.142ubuntu16) noble; urgency=medium

  * Install nvmem, soc, power, and System Power Management Interface (SPMI)
    kernel modules on ARM/RISC-V for MediaTek boards (LP: #2038512)
  * Install intel_lpss_pci and spi_pxa2xx_platform kernel modules for keyboard
    on MacBook Pro 2017 (LP: #2042710)
  * Install surface_aggregator_registry for keyboard on Surface Laptop 4
    (LP: #2007050)
  * framebuffer: Add privacy screen modules to the initrd (LP: #1954320)

 -- Benjamin Drung <email address hidden> Tue, 14 Nov 2023 19:34:42 +0100

[Timo Aaltonen (tjaalton)]

Hello Tom, or anyone else affected,

Accepted initramfs-tools into mantic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/initramfs-tools/0.142ubuntu15.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-mantic to verification-done-mantic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-mantic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted initramfs-tools (0.142ubuntu15.1) for mantic have finished running.
The following regressions have been reported in tests triggered by the package:

initramfs-tools/0.142ubuntu15.1 (armhf)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/mantic/update_excuses.html#initramfs-tools

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Daniel van Vugt (vanvugt)]

Verified fixed in mantic on a Surface Laptop 4 using version 0.142ubuntu15.1

[Brian Murray (brian-murray)]

The verification of the Stable Release Update for initramfs-tools has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package initramfs-tools - 0.142ubuntu15.1

---------------
initramfs-tools (0.142ubuntu15.1) mantic; urgency=medium

  * Install intel_lpss_pci and spi_pxa2xx_platform kernel modules for keyboard
    on MacBook Pro 2017 (LP: #2042710)
  * Install surface_aggregator_registry for keyboard on Surface Laptop 4
    (LP: #2007050)

 -- Benjamin Drung <email address hidden> Tue, 14 Nov 2023 20:02:42 +0100