mt7921e 0000:02:00.0: PM: failed to resume async: error -110

[You-Sheng Yang]

[SRU Justification]

[Impact]

Mediatek MT7921 may fail to resume from runtime suspend can cause chip
reset.

  mt7921e 0000:02:00.0: PM: failed to resume async: error -110
  mt7921e 0000:02:00.0: chip reset

[Fix]

This is fixed with both bluetooth and WiFi firmware upgrade to
upstream commit 094d7adc1136 ("linux-firmware: update firmware for
MT7921 WiFi device") and commit 1e744b85cd4c ("linux-firmware: update
firmware for mediatek bluetooth chip(MT7921)").

[Test Case]

This can be triggered by checkbox warm reset stress test:

  $ checkbox-cli run com.canonical.certification::stress/reboot

[Where problems could occur]

This imports fixes requiring both kernel driver and proprietary firmware
updates, the firmware part might cause instability or so.

[Other Info]

This affects all kernels supporting MT7921, which means F/H/I/J.

========== original bug report ==========

[Summary]
Boot into Ubuntu desktop, MT7921 shows IP is obtained, however there is no response from gateway pings, neither is 8.8.8.8.

kernel: [ 588.893227] mt7921e 0000:02:00.0: PM: failed to resume async: error -110
kernel: [ 588.893236] mt7921e 0000:02:00.0: chip reset

[Reproduce Steps]
1. Boot into Ubuntu desktop
2. Check if MT7921 obtains IP, ping gateway and ping 8.8.8.8
3. No response, need to do cold boot to recover
4. The issue is easily be seen after warm reset stress, I locally test warm reset stress for 100 times

[Results]
Expected: Has traffic and response by ping
Actual: No response

[You-Sheng Yang (vicamo)]

https://bugs.launchpad.net/somerville/+bug/1951825/comments/17

> MTK upstream firmware to fix the bug, BT/WiFi are both required.
> https://lore.kernel.org/linux-firmware/46921394.S1zCe5f9UI@ripper/T/#m5484a716f22fb111fbf87919a70f2090a4247ea3
> https://lore<email address hidden>/T/#md498a7ddeac7d86beaea8e610ae60ed131403733

They're now commit 094d7adc1136 ("linux-firmware: update firmware for MT7921 WiFi device") and commit 1e744b85cd4c ("linux-firmware: update firmware for mediatek bluetooth chip(MT7921)") in the mainline linux-firmware tree.

[You-Sheng Yang (vicamo)]

Included in https://launchpad.net/~canonical-hwe-team/+archive/ubuntu/linux-firmware-staging version 1.187.25+staging.43.

[You-Sheng Yang (vicamo)]

SRU:
* https://lists.ubuntu.com/archives/kernel-team/2022-January/127348.html (linux-firmware/focal)
* https://lists.ubuntu.com/archives/kernel-team/2022-January/127349.html (linux-firmware/impish)
* https://lists.ubuntu.com/archives/kernel-team/2022-January/127350.html (linux-firmware/jammy)

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux-firmware - 20220124.git0c6a7b3b-0ubuntu1

---------------
linux-firmware (20220124.git0c6a7b3b-0ubuntu1) jammy; urgency=medium

  * mt7921e 0000:02:00.0: PM: failed to resume async: error -110 (LP: #1958286)
    - Rebase to upstream commit 0c6a7b3bf728b95c8b7b95328f94335e2bb2c967
      Rebase against git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git
      - QCA: Add Bluetooth default nvm file for WCN685x
      - i915: Add DMC firmware v2.14 for ADL-P
      - rtw89: 8852a: update fw to v0.13.33.0
      - cxgb4: Update firmware to revision 1.26.4.0
      - i915: Add GuC v69.0.3 for all platforms
      - rtl_bt: Update RTL8761B BT USB firmware to 0x09A9_8A6B
      - rtl_bt: Update RTL8761B BT UART firmware to 0x0CA9_8A6B
      - linux-firmware: wilc1000: update WILC1000 firmware to v15.4.1
      - linux-firmware: update frimware for mediatek bluetooth chip (MT7921)
      - bnx2x: Add FW 7.13.21.0
      - amdgpu: update green sardine PSP firmware
      - QCA: Update Bluetooth WCN685x firmware to 2.1.0-00298
      - rtw88: 8822c: Update normal firmware to v9.9.11
      - linux-firmware: update firmware for mediatek bluetooth chip (MT7921)
      - linux-firmware: update firmware for MT7921 WiFi device
      - cnm: add chips&media wave521c firmware.
      - cxgb4: Update firmware to revision 1.26.6.0
      - QCA: Update Bluetooth WCN685x 2.0 firmware to 2.0.0-00609
      - QCA: Update Bluetooth WCN685x 2.1 firmware to 2.1.0-00324
      - QCA: Add Bluetooth nvm file for WCN685x
      - amdgpu: update yellow carp dmcub firmware
      - WHENCE: add missing symlink for NanoPi R1
      - linux-firmware: Update firmware file for Intel Bluetooth AX210
      - linux-firmware: Update firmware file for Intel Bluetooth AX211
      - linux-firmware: Update firmware file for Intel Bluetooth AX211
      - linux-firmware: Update firmware file for Intel Bluetooth AX201
      - linux-firmware: Update firmware file for Intel Bluetooth AX201
      - linux-firmware: Update firmware file for Intel Bluetooth 9560
      - linux-firmware: Update firmware file for Intel Bluetooth 9560
      - linux-firmware: Update firmware file for Intel Bluetooth 9462
      - linux-firmware: Update firmware file for Intel Bluetooth 9462
      - linux-firmware: add firmware for MT7916
      - iwlwifi: update 9000-family firmwares to core66-88
      - iwlwifi: add new FWs from core66-88 release
      - iwlwifi: add new FWs from core63-136 release
      - linux-firmware: update firmware for MT7915
      - linux-firmware: add marvell CPT firmware images
      - Mellanox: Add new mlxsw_spectrum firmware xx.2010.1232
      - linux-firmware: update firmware for MT7921 WiFi device
      - linux-firmware: update firmware for mediatek bluetooth chip(MT7921)
      - linux-firmware: Update firmware file for Intel Bluetooth AX201
      - linux-firmware: Update firmware file for Intel Bluetooth AX200
      - linux-firmware: Update firmware file for Intel Bluetooth 9560
      - linux-firmware: Update firmware file for Intel Bluetooth AX210
      - linux-firmware: Update firmware file for Intel Bluetooth AX211
      - linux-firmware: Update ...

[Timo Aaltonen (tjaalton)]

Hello You-Sheng, or anyone else affected,

Accepted linux-firmware into impish-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/linux-firmware/1.201.4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-impish to verification-done-impish. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-impish. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Timo Aaltonen (tjaalton)]

Hello You-Sheng, or anyone else affected,

Accepted linux-firmware into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/linux-firmware/1.187.26 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Kai-Chuan Hsieh (kchsieh)]

Run 100 time reboot stress on WMVD5-DVT2-A1, no chip reset, wifi/bt works normally.

kernel: 5.14.0-1020-oem
bios: 1.2.0

[You-Sheng Yang (vicamo)]

verified linux-firmware/impish-proposed version 1.201.4.

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux-firmware - 1.201.4

---------------
linux-firmware (1.201.4) impish; urgency=medium

  * mt7921e: Failed to start WM firmware (LP: #1954300)
    - linux-firmware: update frimware for mediatek bluetooth chip (MT7921)
  * mt7921e 0000:02:00.0: PM: failed to resume async: error -110 (LP: #1958286)
    - linux-firmware: update frimware for mediatek bluetooth chip (MT7921)
    - linux-firmware: update firmware for mediatek bluetooth chip (MT7921)
    - linux-firmware: update firmware for MT7921 WiFi device
    - linux-firmware: update firmware for MT7921 WiFi device
    - linux-firmware: update firmware for mediatek bluetooth chip(MT7921)

 -- Juerg Haefliger <email address hidden> Fri, 28 Jan 2022 11:12:51 +0100

[Timo Aaltonen (tjaalton)]

The verification of the Stable Release Update for linux-firmware has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux-firmware - 1.187.26

---------------
linux-firmware (1.187.26) focal; urgency=medium

  * [SRU][F] Add firmware of MT7922 (LP: #1958153)
    - linux-firmware: add firmware for MT7922
    - linux-firmware: add frimware for mediatek bluetooth chip (MT7922)
  * linux-firmware missing renoir_ta (LP: #1958283)
    - amdgpu: update renoir firmware for 20.10
    - amdgpu: update renoir firmware from 20.20 release
    - amdgpu: update renoir firmware for 20.30
    - amdgpu: update renoir firmware for 20.40
    - amdgpu: update renoir firmware for 20.45
    - amdgpu: update renoir firmware for 20.50
    - amdgpu: update renoir firmware from 21.10
    - amdgpu: update renoir firmware from 21.20
    - amdgpu: update vcn firmware for renoir for 21.20
    - amdgpu: update renoir firmware from 21.30
    - amdgpu: update VCN firmware for renoir
  * mt7921e 0000:02:00.0: PM: failed to resume async: error -110 (LP: #1958286)
    - linux-firmware: update frimware for mediatek bluetooth chip (MT7921)
    - linux-firmware: update firmware for mediatek bluetooth chip (MT7921)
    - linux-firmware: update firmware for MT7921 WiFi device
    - linux-firmware: update firmware for MT7921 WiFi device
    - linux-firmware: update firmware for mediatek bluetooth chip(MT7921)
  * [SRU][F] Add mt8183 SCP firmware (LP: #1959064)
    - mediatek: Add mt8183 SCP firmware
  * AMD Yellow Carp DMCUB fw update for s0i3 B0 fixes (LP: #1957026)
    - amdgpu: update yellow carp dmcub firmware

 -- Juerg Haefliger <email address hidden> Fri, 28 Jan 2022 10:54:30 +0100

[David Soles (davidsoles)]

What is the proper place to report that this error persists on Ubuntu 22.04 Jammy? After resuming my suspended PC, the wifi becomes unresponsive. The only way to get it working again is to do a full reboot.

 kernel: mt7921e 0000:02:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x000a address=0x0 flags=0x0020]
 kernel: mt7921e 0000:02:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x000a address=0x0 flags=0x0020]
 kernel: mt7921e 0000:02:00.0: chip reset
 kernel: mt7921e 0000:02:00.0: Message 0000004e (seq 7) timeout
 kernel: mt7921e 0000:02:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x000a address=0x0 flags=0x0020]
 kernel: mt7921e 0000:02:00.0: chip reset failed
 kernel: mt7921e 0000:02:00.0: Failed to get patch semaphore
 kernel: mt7921e 0000:02:00.0: Message 80000010 (seq 4) timeout

Any advice?

[Kai-Heng Feng (kaihengfeng)]

Please file a new bug.