mstflint-dkms module bug causes UBSAN errors in dmesg

Bug #2061185 reported by dann frazier
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
mstflint (Ubuntu)
Fix Released
Undecided
Unassigned
Jammy
Fix Committed
Undecided
dann frazier
Mantic
Fix Released
Undecided
dann frazier

Bug Description

[Impact]
A bug in the kernel module source can cause error messages to show up in the kernel when an application closes the device when UBSAN is active. This is likely to cause unnecessary user worry:

From an NVIDIA DGX 1 system:
$ sudo mstflint -d 84:00.0 q
$ dmesg
[ 304.400217] ================================================================================
[ 304.400267] UBSAN: shift-out-of-bounds in /var/lib/dkms/mstflint/4.21.0-7/build/mst_main.c:601:21
[ 304.400283] shift exponent 4294967295 is too large for 32-bit type 'int'
[ 304.400295] CPU: 65 PID: 25268 Comm: mstflint Tainted: G OE 5.15.0-102-generic #112-Ubuntu
[ 304.400298] Hardware name: NVIDIA DGX-1 with V100-32/DGX-1 with V100-32, BIOS S2W_3A13 01/03/2023
[ 304.400299] Call Trace:
[ 304.400301] <TASK>
[ 304.400303] show_stack+0x52/0x5c
[ 304.400309] dump_stack_lvl+0x4a/0x63
[ 304.400315] dump_stack+0x10/0x16
[ 304.400318] ubsan_epilogue+0x9/0x36
[ 304.400320] __ubsan_handle_shift_out_of_bounds.cold+0x61/0xef
[ 304.400324] ? do_sigaction+0x226/0x290
[ 304.400329] mst_release.cold+0x14/0x26 [mstflint_access]
[ 304.400332] __fput+0x9f/0x280
[ 304.400338] ____fput+0xe/0x20
[ 304.400340] task_work_run+0x6d/0xb0
[ 304.400345] exit_to_user_mode_loop+0x157/0x160
[ 304.400352] exit_to_user_mode_prepare+0xa0/0xb0
[ 304.400354] syscall_exit_to_user_mode+0x27/0x50
[ 304.400372] ? __x64_sys_close+0x11/0x50
[ 304.400375] do_syscall_64+0x69/0xc0
[ 304.400380] ? do_syscall_64+0x69/0xc0
[ 304.400383] ? do_syscall_64+0x69/0xc0
[ 304.400385] entry_SYSCALL_64_after_hwframe+0x62/0xcc
[ 304.400389] RIP: 0033:0x7f0347edaf67
[ 304.400391] Code: ff e8 0d 16 02 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 03 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 41 c3 48 83 ec 18 89 7c 24 0c e8 73 ba f7 ff
[ 304.400393] RSP: 002b:00007ffebcd12c98 EFLAGS: 00000246 ORIG_RAX: 0000000000000003
[ 304.400396] RAX: 0000000000000000 RBX: 000055aa91e5fe10 RCX: 00007f0347edaf67
[ 304.400398] RDX: 0000000000000000 RSI: 000055aa913737a1 RDI: 0000000000000003
[ 304.400399] RBP: 000055aa91e5fe10 R08: 000055aa91e60cc0 R09: 000055aa91e61160
[ 304.400400] R10: 000055aa91e615b0 R11: 0000000000000246 R12: 000055aa91e60820
[ 304.400401] R13: 000055aa91e2b650 R14: 000055aa91e4b3f0 R15: 000055aa9136ac47
[ 304.400403] </TASK>
[ 304.400404] ================================================================================

[Test Case]
Load the mstflint_access module
open then close a /dev/*_mstconf file
Look for any errors in dmesg

[What Could Go Wrong]
A severe bug in the fix could cause the problem to escalate to an oops or panic.

dann frazier (dannf)
Changed in mstflint (Ubuntu):
status: New → Fix Released
Changed in mstflint (Ubuntu Mantic):
status: New → In Progress
Changed in mstflint (Ubuntu Jammy):
status: New → In Progress
assignee: nobody → dann frazier (dannf)
Changed in mstflint (Ubuntu Mantic):
assignee: nobody → dann frazier (dannf)
dann frazier (dannf)
description: updated
description: updated
Revision history for this message
Timo Aaltonen (tjaalton) wrote : Please test proposed package

Hello dann, or anyone else affected,

Accepted mstflint into mantic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/mstflint/4.21.0+1-1ubuntu0.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-mantic to verification-done-mantic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-mantic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in mstflint (Ubuntu Mantic):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-mantic
Revision history for this message
Timo Aaltonen (tjaalton) wrote :

Hello dann, or anyone else affected,

Accepted mstflint into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/mstflint/4.21.0+1-1ubuntu0.1~22.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in mstflint (Ubuntu Jammy):
status: In Progress → Fix Committed
tags: added: verification-needed-jammy
Revision history for this message
dann frazier (dannf) wrote :

# mantic verification

ubuntu@akis:~$ uname -a
Linux akis 6.5.0-28-generic #29-Ubuntu SMP PREEMPT_DYNAMIC Thu Mar 28 23:46:48 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
ubuntu@akis:~$ sudo dmesg -c > /dev/null
ubuntu@akis:~$ sudo mstflint -d 35:00.0 q
Image type: FS4
FW Version: 16.28.2040
FW Release Date: 6.10.2020
Product Version: 16.28.2040
Rom Info: type=UEFI version=14.21.17 cpu=AMD64
                       type=PXE version=3.6.102 cpu=AMD64
Description: UID GuidsNumber
Base GUID: 98039b0300638334 4
Base MAC: 98039b638334 4
Image VSD: N/A
Device VSD: N/A
PSID: MT_0000000010
Security Attributes: N/A
ubuntu@akis:~$ sudo dmesg
ubuntu@akis:~$

Revision history for this message
dann frazier (dannf) wrote :

# jammy verification
ubuntu@hinyari:~$ uname -a
Linux hinyari 6.5.0-28-generic-64k #29~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Fri Apr 5 16:55:05 UTC 2 aarch64 aarch64 aarch64 GNU/Linux
ubuntu@hinyari:~$ sudo dmesg -c > /dev/null
ubuntu@hinyari:~$ sudo mstflint -d 0006:01:00.0 q
Image type: FS4
FW Version: 26.33.1048
FW Release Date: 29.4.2022
Product Version: 26.33.1048
Rom Info: type=UEFI version=14.26.17 cpu=AMD64,AARCH64
                       type=PXE version=3.6.502 cpu=AMD64
Description: UID GuidsNumber
Base GUID: b83fd203001d3740 4
Base MAC: b83fd21d3740 4
Image VSD: N/A
Device VSD: N/A
PSID: MT_0000000531
Security Attributes: N/A
ubuntu@hinyari:~$ sudo dmesg
ubuntu@hinyari:~$

tags: added: verification-done verification-done-jammy verification-done-mantic
removed: verification-needed verification-needed-jammy verification-needed-mantic
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package mstflint - 4.21.0+1-1ubuntu0.1

---------------
mstflint (4.21.0+1-1ubuntu0.1) mantic; urgency=medium

  * Add mstflint-dkms package to avoid a regression from jammy.
    LP: #2061182. This backports the packaging changes from noble that
    introduce the mstflint-dkms binary, which depends on the following
    backport of the install-dkms target from upstream:
    - d/p/0001-debian-mstflint-mstflint-dkms-packages-generation.patch
  * A bug in the kernel module can lead to UBSAN errors in dmesg upon
    device closure. LP: #2061185. Backport changes to fix that:
    - d/p/0002-Fix-illegal-shift-in-mst_release.patch
    - d/p/0003-Fix-typo.patch (fix for previous patch)
  * Backport change to clarify the licensing terms for the mstflint_access
    kernel module. LP: #2061186.
    - d/p/0004-Add-missed-license.patch
  * The kernel modules source in mstflint 4.21 FTBFS w/ the 6.5 kernel.
    LP: #2043098. Backport changes to fix that:
    - d/p/0005-fix-kernel-module-build-on-kernel-6.4.patch
    - d/p/0006-kernel-module-fix-wrong-comparison-order.patch (fix for
      previous patch).
  * Since we have now backported every patch that touches the kernel module
    source between 4.21.0 and 4.26.0, use 4.26.0 as the upstream version for
    the mstflint-dkms deb. This requires a patch to let us override the
    version during the build:
    - d/p/kernel-module-config-version.patch

 -- dann frazier <email address hidden> Fri, 12 Apr 2024 14:26:39 -0600

Changed in mstflint (Ubuntu Mantic):
status: Fix Committed → Fix Released
Revision history for this message
Andreas Hasenack (ahasenack) wrote : Update Released

The verification of the Stable Release Update for mstflint has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.