| 2019-07-01 12:41:41 |
Mark Blakley |
description |
What I expected to happen: Loops can be successfully created and removed from multiple different threads without any adverse effects
What happened instead: Udev processes ended up in D state, which soft locked the system (unable to reboot without resorting to "echo b > /proc/sysrq-trigger", various other commands [losetup -a, mount -l] would hang)
The use case that this was originally reproduced on required frequent setup and teardown of loops in order to persist and access various snapshots of backup files. Through various methods of analysis, we were able to create a script that recreates the bad behavior relatively quickly (see attachment testudevLoopOnly.php).
We captured some stack traces from a locked up system, and found that many of the calls were originating from loop.c. Investigating that file (specifically lo_setup and lo_release functions) showed that the 4.4 usage of mutexes is different from earlier versions of the kernel package (4.4.0.142.148), which doesn't exhibit this same behavior. It's also different from the 4.15 version of the kernel package, which also does not exhibit the same (incorrect) behavior. |
What I expected to happen: Loops can be successfully created and removed from multiple different threads without any adverse effects
What happened instead: Udev processes ended up in D state, which soft locked the system (unable to reboot without resorting to "echo b > /proc/sysrq-trigger", various other commands [losetup -a, mount -l] would hang)
The use case that this was originally reproduced on required frequent setup and teardown of loops in order to persist and access various snapshots of backup files. Through various methods of analysis, we were able to create a script that recreates the bad behavior relatively quickly (see attachment testudevLoopOnly.php).
We captured some stack traces from a locked up system, and found that many of the calls were originating from loop.c. Investigating that file (specifically lo_setup and lo_release functions) showed that the 4.4 usage of mutexes is different from earlier versions of the kernel package (4.4.0.142.148), which doesn't exhibit this same behavior. It's also different from the 4.15 version of the kernel package, which also does not exhibit the same (incorrect) behavior.
---
AlsaVersion: Advanced Linux Sound Architecture Driver Version k4.4.0-148-generic.
AplayDevices: Error: [Errno 2] No such file or directory
ApportVersion: 2.20.1-0ubuntu2.18
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/by-path', '/dev/snd/hwC0D0', '/dev/snd/pcmC0D8p', '/dev/snd/pcmC0D7p', '/dev/snd/pcmC0D3p', '/dev/snd/controlC0', '/dev/snd/hwC1D2', '/dev/snd/pcmC1D2c', '/dev/snd/pcmC1D1p', '/dev/snd/pcmC1D0c', '/dev/snd/pcmC1D0p', '/dev/snd/controlC1', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
Card0.Amixer.info: Error: [Errno 2] No such file or directory
Card0.Amixer.values: Error: [Errno 2] No such file or directory
Card1.Amixer.info: Error: [Errno 2] No such file or directory
Card1.Amixer.values: Error: [Errno 2] No such file or directory
DistroRelease: Ubuntu 16.04
IwConfig: Error: [Errno 2] No such file or directory
MachineType: Gigabyte Technology Co., Ltd. H97N
NonfreeKernelModules: zfs zunicode icp zcommon znvpair zavl
Package: linux (not installed)
ProcEnviron:
TERM=xterm-256color
PATH=(custom, no user)
XDG_RUNTIME_DIR=<set>
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcFB: 0 inteldrmfb
ProcKernelCmdLine: BOOT_IMAGE=(loop)/vmlinuz root=UUID=02d26ec8-92ae-414a-9332-4e57f62f9bd3 rootdelay=120 rw quiet panic=5 vga=791 consoleblank=0 max_loop=256 splash loop=/images/704.0.img net.ifnames=0 nomdmonisw nomdmonddf bootdegraded=true audit=1
ProcVersionSignature: Ubuntu 4.4.0-148.174-generic 4.4.177
RelatedPackageVersions:
linux-restricted-modules-4.4.0-148-generic N/A
linux-backports-modules-4.4.0-148-generic N/A
linux-firmware 1.157.21
RfKill: Error: [Errno 2] No such file or directory
Tags: xenial
Uname: Linux 4.4.0-148-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups:
_MarkForUpload: True
dmi.bios.date: 06/10/2014
dmi.bios.vendor: American Megatrends Inc.
dmi.bios.version: F1
dmi.board.asset.tag: To be filled by O.E.M.
dmi.board.name: H97N
dmi.board.vendor: Gigabyte Technology Co., Ltd.
dmi.board.version: x.x
dmi.chassis.asset.tag: To Be Filled By O.E.M.
dmi.chassis.type: 3
dmi.chassis.vendor: Gigabyte Technology Co., Ltd.
dmi.chassis.version: To Be Filled By O.E.M.
dmi.modalias: dmi:bvnAmericanMegatrendsInc.:bvrF1:bd06/10/2014:svnGigabyteTechnologyCo.,Ltd.:pnH97N:pvrTobefilledbyO.E.M.:rvnGigabyteTechnologyCo.,Ltd.:rnH97N:rvrx.x:cvnGigabyteTechnologyCo.,Ltd.:ct3:cvrToBeFilledByO.E.M.:
dmi.product.name: H97N
dmi.product.version: To be filled by O.E.M.
dmi.sys.vendor: Gigabyte Technology Co., Ltd. |
|
