Kernel Oops - unable to handle kernel NULL pointer dereference at 0000000000000910 in update_blocked_averages
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
linux (Ubuntu) |
Expired
|
Medium
|
Unassigned |
Bug Description
Running in Virtualbox as a guest in a Macbook Pro, I'm running into a reliably-
The stack trace isn't always the same, although the error always happens in the same `update_
Here' one oops:
```
[ 56.972407] BUG: unable to handle kernel NULL pointer dereference at 0000000000000910
[ 57.076894] IP: [<ffffffff810b4
[ 57.173751] PGD 0
[ 57.193537] Oops: 0000 [#1] SMP
[ 57.245472] Modules linked in: veth xt_addrtype xt_conntrack ipt_MASQUERADE nf_nat_
[ 57.937239] CPU: 1 PID: 269 Comm: systemd-journal Not tainted 4.4.0-17-generic #33-Ubuntu
[ 58.049743] Hardware name: innotek GmbH VirtualBox/
[ 58.189828] task: ffff8800d8f90dc0 ti: ffff8800da9a8000 task.ti: ffff8800da9a8000
[ 58.299087] RIP: 0010:[<
[ 58.320327] RSP: 0018:ffff8800da
[ 58.366603] RAX: 0000000000000000 RBX: ffff88020c3bae00 RCX: 0000000000000000
[ 58.514856] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000001
[ 58.612464] RBP: ffff8800da9abcc8 R08: afb504000afb5041 R09: 0000000000000000
[ 58.738629] R10: 0000000000000002 R11: 0000000000000000 R12: 0000000000000045
[ 58.885338] R13: 0000000000000000 R14: ffff8802198975c0 R15: 0000000000000001
[ 59.001563] FS: 00007f7db7f6e84
[ 59.160653] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 59.296218] CR2: 0000000000000910 CR3: 000000020c3fd000 CR4: 00000000000406e0
[ 59.418514] Stack:
[ 59.458306] 00000000000205f5 ffff880219896d00 0000000000000086 afb504000afb5041
[ 59.580604] 0000000000000000 0000ad60ffff01fe 0000000000000001 00000000ffff02f8
[ 59.720268] 0000000000016d00 ffff880219896d00 ffff8800d8f91310 ffff8800da9abd38
[ 59.848040] Call Trace:
[ 59.902635] [<ffffffff810bc
[ 59.992238] [<ffffffff8181e
[ 60.030794] [<ffffffff8181e
[ 60.074193] [<ffffffff81822
[ 60.134427] [<ffffffff81254
[ 60.195234] [<ffffffff810c9
[ 60.289354] [<ffffffff81822
[ 60.325540] [<ffffffff81254
[ 60.328490] [<ffffffff810aa
[ 60.387888] [<ffffffff81255
[ 60.502960] [<ffffffff81822
[ 60.610213] Code: 24 08 49 b8 41 50 fb 0a 00 04 b5 af 4d 89 fe 0f 1f 44 00 00 44 8b 9b 24 01 00 00 45 85 db 0f 85 a0 03 00 00 48 8b 83 c8 00 00 00 <48> 8b 80 10 09 00 00 48 2b 83 18 01 00 00 48 8b 93 a0 00 00 00
[ 60.941287] RIP [<ffffffff810b4
[ 61.111617] RSP <ffff8800da9abc70>
[ 61.201406] CR2: 0000000000000910
[ 61.258351] ---[ end trace fe3df8ee7b476828 ]---
```
Here's another one:
```
[ 154.454951] BUG: unable to handle kernel [ 154.455154] device vethe5e9554 entered promiscuous mode
[ 154.455202] IPv6: ADDRCONF(
[ 154.455203] docker0: port 1(vethe5e9554) entered forwarding state
[ 154.455210] docker0: port 1(vethe5e9554) entered forwarding state
[ 154.744305] NULL pointer dereference at 0000000000000910
[ 154.783574] IP: [<ffffffff810b4
[ 154.784877] PGD 0
[ 154.785372] Oops: 0000 [#1] SMP
[ 154.790654] Modules linked in: veth xt_addrtype xt_conntrack ipt_MASQUERADE nf_nat_
[ 155.387419] CPU: 0 PID: 2043 Comm: systemd-udevd Tainted: G OE 4.4.0-17-generic #33-Ubuntu
[ 155.546119] Hardware name: innotek GmbH VirtualBox/
[ 155.654775] task: ffff8800dbb3b700 ti: ffff88020da8c000 task.ti: ffff88020da8c000
[ 155.753906] RIP: 0010:[<
[ 155.775805] RSP: 0000:ffff880219
[ 155.811224] RAX: 0000000000000000 RBX: ffff8800db891200 RCX: 0000000000000001
[ 155.857524] RDX: 0000000000000001 RSI: ffffffffffffffff RDI: 0000000000000002
[ 155.861832] RBP: ffff880219803e50 R08: afb504000afb5041 R09: 0000000000000100
[ 155.980061] R10: 0000000000000003 R11: 0000000000000000 R12: 0000000000000291
[ 156.059221] R13: 0000000000000000 R14: ffff8802198175c0 R15: 0000000000000001
[ 156.167527] FS: 00007f06cde8c8c
[ 156.302318] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 156.386563] CR2: 0000000000000910 CR3: 000000020c6a7000 CR4: 00000000000406f0
[ 156.507153] Stack:
[ 156.559351] 0000000000445496 ffff880219816d00 0000000000000286 afb504000afb5041
[ 156.590109] 0000000000000000 0043430019816d00 0000000000016d00 0000000000000000
[ 156.684722] 00000000ffff61a3 0000000000000001 ffff880219816d00 ffff8800db5097f8
[ 156.798239] Call Trace:
[ 156.837119] <IRQ>
[ 156.863469] [<ffffffff810bc
[ 156.997302] [<ffffffff810ed
[ 157.111072] [<ffffffff810bd
[ 157.183338] [<ffffffff81051
[ 157.271222] [<ffffffff81084
[ 157.359428] [<ffffffff81084
[ 157.453079] [<ffffffff81825
[ 157.535290] [<ffffffff81823
[ 157.656699] <EOI>
[ 157.688002] [<ffffffff8118c
[ 157.752691] [<ffffffff810c9
[ 157.806798] [<ffffffff811be
[ 157.860940] [<ffffffff8122b
[ 157.904327] [<ffffffff8106a
[ 157.984122] [<ffffffff8106a
[ 158.076895] [<ffffffff81824
[ 158.138672] Code: 24 08 49 b8 41 50 fb 0a 00 04 b5 af 4d 89 fe 0f 1f 44 00 00 44 8b 9b 24 01 00 00 45 85 db 0f 85 a0 03 00 00 48 8b 83 c8 00 00 00 <48> 8b 80 10 09 00 00 48 2b 83 18 01 00 00 48 8b 93 a0 00 00 00
[ 158.290783] RIP [<ffffffff810b4
[ 158.383333] RSP <ffff880219803df0>
[ 158.414588] CR2: 0000000000000910
[ 158.467115] ---[ end trace 27ceb18bfd8d94ca ]---
[ 158.505862] Kernel panic - not syncing: Fatal exception in interrupt
[ 159.629144] Shutting down cpus with NMI
[ 159.745790] Kernel Offset: disabled
[ 159.859365] ---[ end Kernel panic - not syncing: Fatal exception in interrupt
```
And a third one:
```
[ 139.661389] BUG: unable to handle kernel NULL pointer dereference at 0000000000000910
[ 139.807547] IP: [<ffffffff810b4
[ 139.946882] PGD 0
[ 139.984609] Oops: 0000 [#1] SMP
[ 140.048650] Modules linked in: veth xt_addrtype xt_conntrack ipt_MASQUERADE nf_nat_
[ 141.115576] CPU: 0 PID: 2038 Comm: systemd-udevd Not tainted 4.4.0-17-generic #33-Ubuntu
[ 141.310652] Hardware name: innotek GmbH VirtualBox/
[ 141.415336] task: ffff88020c2b9b80 ti: ffff8800dadf0000 task.ti: ffff8800dadf0000
[ 141.519708] RIP: 0010:[<
[ 141.707701] RSP: 0018:ffff8800da
[ 141.824270] RAX: 0000000000000000 RBX: ffff88020c1a9000 RCX: 0000000000000001
[ 141.999630] RDX: 0000000000000485 RSI: 00000000cd4629c5 RDI: ffff880219bf1000
[ 142.180519] RBP: ffff8800dadf3cc8 R08: afb504000afb5041 R09: 0000000000000000
[ 142.369893] R10: 00000000ffff518b R11: 0000000000000000 R12: 00000000ffff5285
[ 142.525548] R13: 0000000000016d00 R14: ffff8802198175c0 R15: ffff8802198175c0
[ 142.675206] FS: 00007ff7980cf8c
[ 142.871842] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 143.019994] CR2: 0000000000000910 CR3: 00000000d8e0e000 CR4: 00000000000406f0
[ 143.181043] Stack:
[ 143.225953] ffffffff813fcd45 ffff880219816d00 0000000000000086 ffffffff810c9591
[ 143.308932] ffff88020ee31c00 00000000ffff518b 0000000000000000 00000000ffff5285
[ 143.470616] 0000000000016d00 ffff880219816d00 ffff88020c2ba0d0 ffff8800dadf3d38
[ 143.593228] Call Trace:
[ 143.695835] [<ffffffff813fc
[ 143.812410] [<ffffffff810c9
[ 143.999872] [<ffffffff810bc
[ 144.104135] [<ffffffff8181e
[ 144.120508] [<ffffffff8181e
[ 144.226139] [<ffffffff81821
[ 144.372190] [<ffffffff810ed
[ 144.491899] [<ffffffff81821
[ 144.646067] [<ffffffff81822
[ 144.687266] [<ffffffff81254
[ 144.689511] [<ffffffff810aa
[ 144.785095] [<ffffffff81255
[ 144.843879] [<ffffffff81822
[ 144.859757] Code: 24 08 49 b8 41 50 fb 0a 00 04 b5 af 4d 89 fe 0f 1f 44 00 00 44 8b 9b 24 01 00 00 45 85 db 0f 85 a0 03 00 00 48 8b 83 c8 00 00 00 <48> 8b 80 10 09 00 00 48 2b 83 18 01 00 00 48 8b 93 a0 00 00 00
[ 144.982177] RIP [<ffffffff810b4
[ 145.024132] RSP <ffff8800dadf3c70>
[ 145.069952] CR2: 0000000000000910
[ 145.118873] ---[ end trace 5324671710cba237 ]---
```
I can reproduce this reliably using the following script (which relies on Docker). This starts a container, then adds another task into its namespace via `docker exec`, and then kills PID 1 in the container, which tears down the PID namespace and should kill both tasks. If I remove the `docker exec` line, there is no crash.
```
#!/bin/bash
set -exu
cid=$(docker run -it -d ubuntu sleep 1000)
docker exec "$cid" sleep 1000 &
sleep 1
docker kill -s KILL "$cid"
docker run -it --rm ubuntu echo "Alive 1?"
docker run -it --rm ubuntu echo "Alive 2?"
docker run -it --rm ubuntu echo "Alive 3?"
```
In the vast majority of cases, the crash will happen before "Alive 1?" is printed to the screen. In some cases, it'll happen at "Alive 2?".
I tried reproducing the issue on an AWS instance, with no success so far.
ProblemType: Bug
DistroRelease: Ubuntu 16.04
Package: linux-image-
ProcVersionSign
Uname: Linux 4.4.0-17-generic x86_64
AlsaVersion: Advanced Linux Sound Architecture Driver Version k4.4.0-17-generic.
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.1-0ubuntu1
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/by-path', '/dev/snd/
Card0.Amixer.info: Error: [Errno 2] No such file or directory: 'amixer'
Card0.Amixer.
Date: Thu Apr 7 20:53:43 2016
HibernationDevice: RESUME=
InstallationDate: Installed on 2016-02-03 (63 days ago)
InstallationMedia: Ubuntu-Server 14.04.3 LTS "Trusty Tahr" - Beta amd64 (20150805)
IwConfig:
eth0 no wireless extensions.
eth1 no wireless extensions.
lo no wireless extensions.
Lsusb:
Bus 001 Device 002: ID 80ee:0021 VirtualBox USB Tablet
Bus 001 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
MachineType: innotek GmbH VirtualBox
ProcFB:
ProcKernelCmdLine: BOOT_IMAGE=
RelatedPackageV
linux-
linux-
linux-firmware 1.157
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
SourcePackage: linux
UpgradeStatus: Upgraded to xenial on 2016-04-06 (1 days ago)
dmi.bios.date: 12/01/2006
dmi.bios.vendor: innotek GmbH
dmi.bios.version: VirtualBox
dmi.board.name: VirtualBox
dmi.board.vendor: Oracle Corporation
dmi.board.version: 1.2
dmi.chassis.type: 1
dmi.chassis.vendor: Oracle Corporation
dmi.modalias: dmi:bvninnotekG
dmi.product.name: VirtualBox
dmi.product.
dmi.sys.vendor: innotek GmbH
Changed in linux (Ubuntu): | |
importance: | Undecided → Medium |
status: | Confirmed → Incomplete |
I should mention that I had the Virtualbox Kernel modules installed in the two latter oops, but that uninstalling them (which is why they are not in first oops) does not make a difference.