Simultaneous creation and teardown of separate loops causes udev softlock

Bug #1834693 reported by Mark Blakley
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
linux (Ubuntu)
Confirmed
Undecided
Unassigned

Bug Description

What I expected to happen: Loops can be successfully created and removed from multiple different threads without any adverse effects
What happened instead: Udev processes ended up in D state, which soft locked the system (unable to reboot without resorting to "echo b > /proc/sysrq-trigger", various other commands [losetup -a, mount -l] would hang)

The use case that this was originally reproduced on required frequent setup and teardown of loops in order to persist and access various snapshots of backup files. Through various methods of analysis, we were able to create a script that recreates the bad behavior relatively quickly (see attachment testudevLoopOnly.php).

We captured some stack traces from a locked up system, and found that many of the calls were originating from loop.c. Investigating that file (specifically lo_setup and lo_release functions) showed that the 4.4 usage of mutexes is different from earlier versions of the kernel package (4.4.0.142.148), which doesn't exhibit this same behavior. It's also different from the 4.15 version of the kernel package, which also does not exhibit the same (incorrect) behavior.
---
AlsaVersion: Advanced Linux Sound Architecture Driver Version k4.4.0-148-generic.
AplayDevices: Error: [Errno 2] No such file or directory
ApportVersion: 2.20.1-0ubuntu2.18
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/by-path', '/dev/snd/hwC0D0', '/dev/snd/pcmC0D8p', '/dev/snd/pcmC0D7p', '/dev/snd/pcmC0D3p', '/dev/snd/controlC0', '/dev/snd/hwC1D2', '/dev/snd/pcmC1D2c', '/dev/snd/pcmC1D1p', '/dev/snd/pcmC1D0c', '/dev/snd/pcmC1D0p', '/dev/snd/controlC1', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
Card0.Amixer.info: Error: [Errno 2] No such file or directory
Card0.Amixer.values: Error: [Errno 2] No such file or directory
Card1.Amixer.info: Error: [Errno 2] No such file or directory
Card1.Amixer.values: Error: [Errno 2] No such file or directory
DistroRelease: Ubuntu 16.04
IwConfig: Error: [Errno 2] No such file or directory
MachineType: Gigabyte Technology Co., Ltd. H97N
NonfreeKernelModules: zfs zunicode icp zcommon znvpair zavl
Package: linux (not installed)
ProcEnviron:
 TERM=xterm-256color
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=en_US.UTF-8
 SHELL=/bin/bash
ProcFB: 0 inteldrmfb
ProcKernelCmdLine: BOOT_IMAGE=(loop)/vmlinuz root=UUID=02d26ec8-92ae-414a-9332-4e57f62f9bd3 rootdelay=120 rw quiet panic=5 vga=791 consoleblank=0 max_loop=256 splash loop=/images/704.0.img net.ifnames=0 nomdmonisw nomdmonddf bootdegraded=true audit=1
ProcVersionSignature: Ubuntu 4.4.0-148.174-generic 4.4.177
RelatedPackageVersions:
 linux-restricted-modules-4.4.0-148-generic N/A
 linux-backports-modules-4.4.0-148-generic N/A
 linux-firmware 1.157.21
RfKill: Error: [Errno 2] No such file or directory
Tags: xenial
Uname: Linux 4.4.0-148-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups:

_MarkForUpload: True
dmi.bios.date: 06/10/2014
dmi.bios.vendor: American Megatrends Inc.
dmi.bios.version: F1
dmi.board.asset.tag: To be filled by O.E.M.
dmi.board.name: H97N
dmi.board.vendor: Gigabyte Technology Co., Ltd.
dmi.board.version: x.x
dmi.chassis.asset.tag: To Be Filled By O.E.M.
dmi.chassis.type: 3
dmi.chassis.vendor: Gigabyte Technology Co., Ltd.
dmi.chassis.version: To Be Filled By O.E.M.
dmi.modalias: dmi:bvnAmericanMegatrendsInc.:bvrF1:bd06/10/2014:svnGigabyteTechnologyCo.,Ltd.:pnH97N:pvrTobefilledbyO.E.M.:rvnGigabyteTechnologyCo.,Ltd.:rnH97N:rvrx.x:cvnGigabyteTechnologyCo.,Ltd.:ct3:cvrToBeFilledByO.E.M.:
dmi.product.name: H97N
dmi.product.version: To be filled by O.E.M.
dmi.sys.vendor: Gigabyte Technology Co., Ltd.

Revision history for this message
Mark Blakley (mblakley) wrote :
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote : Missing required logs.

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1834693

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

Changed in linux (Ubuntu):
status: New → Incomplete
Revision history for this message
Mark Blakley (mblakley) wrote :

I'm not able to run this command on the system where my attached logs were collected.

Changed in linux (Ubuntu):
status: Incomplete → Confirmed
Revision history for this message
Mark Blakley (mblakley) wrote : AlsaDevices.txt

apport information

tags: added: apport-collected xenial
description: updated
Revision history for this message
Mark Blakley (mblakley) wrote : CRDA.txt

apport information

Revision history for this message
Mark Blakley (mblakley) wrote : Card0.Codecs.codec.0.txt

apport information

Revision history for this message
Mark Blakley (mblakley) wrote : Card1.Codecs.codec.2.txt

apport information

Revision history for this message
Mark Blakley (mblakley) wrote : CurrentDmesg.txt

apport information

Revision history for this message
Mark Blakley (mblakley) wrote : Lspci.txt

apport information

Revision history for this message
Mark Blakley (mblakley) wrote : Lsusb.txt

apport information

Revision history for this message
Mark Blakley (mblakley) wrote : PciMultimedia.txt

apport information

Revision history for this message
Mark Blakley (mblakley) wrote : ProcCpuinfo.txt

apport information

Revision history for this message
Mark Blakley (mblakley) wrote : ProcCpuinfoMinimal.txt

apport information

Revision history for this message
Mark Blakley (mblakley) wrote : ProcInterrupts.txt

apport information

Revision history for this message
Mark Blakley (mblakley) wrote : ProcModules.txt

apport information

Revision history for this message
Mark Blakley (mblakley) wrote : UdevDb.txt

apport information

Revision history for this message
Mark Blakley (mblakley) wrote : WifiSyslog.txt

apport information

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.