| 2019-05-24 19:52:50 |
Mathieu Desnoyers |
bug |
|
|
added bug |
| 2019-05-24 19:52:50 |
Mathieu Desnoyers |
attachment added |
|
kernel BUG screenshot https://bugs.launchpad.net/bugs/1830433/+attachment/5266352/+files/Screenshot_ubuntu-bionic-32bit_2019-05-23_10%3A20%3A08.png |
|
| 2019-05-24 19:53:57 |
Mathieu Desnoyers |
attachment added |
|
version.log https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1830433/+attachment/5266353/+files/version.log |
|
| 2019-05-24 19:54:11 |
Mathieu Desnoyers |
attachment added |
|
uname-a.log https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1830433/+attachment/5266354/+files/uname-a.log |
|
| 2019-05-24 19:54:40 |
Mathieu Desnoyers |
attachment added |
|
dmesg.log https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1830433/+attachment/5266355/+files/dmesg.log |
|
| 2019-05-24 19:54:50 |
Mathieu Desnoyers |
attachment added |
|
lspci-vvnn.log https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1830433/+attachment/5266356/+files/lspci-vvnn.log |
|
| 2019-05-24 20:00:08 |
Ubuntu Kernel Bot |
linux (Ubuntu): status |
New |
Confirmed |
|
| 2019-05-24 20:00:09 |
Ubuntu Kernel Bot |
tags |
|
bionic |
|
| 2019-05-30 15:19:45 |
Terry Rudd |
bug |
|
|
added subscriber Terry Rudd |
| 2019-06-13 14:38:16 |
Andrea Righi |
linux (Ubuntu): importance |
Undecided |
Medium |
|
| 2019-06-13 14:38:43 |
Andrea Righi |
nominated for series |
|
Ubuntu Bionic |
|
| 2019-06-13 14:38:43 |
Andrea Righi |
bug task added |
|
linux (Ubuntu Bionic) |
|
| 2019-06-13 14:38:49 |
Andrea Righi |
linux (Ubuntu Bionic): status |
New |
Confirmed |
|
| 2019-06-13 14:38:52 |
Andrea Righi |
linux (Ubuntu Bionic): importance |
Undecided |
Medium |
|
| 2019-06-13 21:04:52 |
Andrea Righi |
attachment added |
|
vmalloc_sync_all() stress test https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1830433/+attachment/5270656/+files/vmalloc-stress-test.c |
|
| 2019-06-13 21:14:27 |
Andrea Righi |
description |
Hi,
I'm reproducing a kernel bug in vmalloc_sync_all() with a 32-bit x86 kernel.
The problem appears in
Linux ubuntu 4.15.0-50-generic #54-Ubuntu SMP Mon May 6 18:45:45 UTC 2019 i686 i686 i686 GNU/Linux
Kernels 4.15.0-49 and prior work fine.
The kernel 4.18.0-20-generic works fine.
This problem has not been experienced with upstream Linux kernels.
It appears that invoking vmalloc_sync_all() a few times end up triggering this issue. This can be triggered by restarting the lttng-sessiond service with lttng-modules-dkms installed (sometimes a few restarts are needed to trigger the bug). This ends up unloading and reloading those modules, which issues a few vmalloc_sync_all() as side-effect.
I'm not reporting this issue with the "ubuntu-bug linux" command because it crashes the system on that kernel (system hangs, no console output).
My test system runs within a kvm virtual machine on a 64-bit host.
lsb release:
Description: Ubuntu 18.04.2 LTS
Release: 18.04
Information about my kernel:
linux-image-4.15.0-50-generic:
Installed: 4.15.0-50.54
Candidate: 4.15.0-50.54
Version table:
*** 4.15.0-50.54 500
500 http://ca.archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages
500 http://security.ubuntu.com/ubuntu bionic-security/main i386 Packages
100 /var/lib/dpkg/status
Information about lttng-modules-dkms:
lttng-modules-dkms:
Installed: 2.10.5-1ubuntu1.2
Candidate: 2.10.5-1ubuntu1.2
Version table:
*** 2.10.5-1ubuntu1.2 500
500 http://ca.archive.ubuntu.com/ubuntu bionic-updates/universe i386 Packages
100 /var/lib/dpkg/status
2.10.5-1ubuntu1 500
500 http://ca.archive.ubuntu.com/ubuntu bionic/universe i386 Packages |
[Impact]
Commit d653420532d580156c8486686899ea6a9eeb7bf0 in bionic enabled kernel page table isolation for x86_32, but also introduced a kernel bug (the BUG_ON() condition in vmalloc_sync_one()) that seems to happen when vmalloc_sync_all() is called multiple times (e.g., in a busy loop).
The real problem seems to be a race condition with page-table entries' initialization that can be fixed applying the upstream commit 9bc4f28af75a91aea0ae383f50b0a430c4509303 ("x86/mm: Use WRITE_ONCE() when setting PTEs").
[Test Case]
The bug can be easily triggered by rebooting the system a couple of times and loading this module:
https://launchpadlibrarian.net/428142172/vmalloc-stress-test.c
[Fix]
The following upstream fix seems to resolve the problem:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=9bc4f28af75a91aea0ae383f50b0a430c4509303
In addition to that the following other upstream fixes are required (all clean cherry picks) to do a cleaner backport of 9bc4f28af75a91aea0ae383f50b0a430c4509303:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=86fa949b050184ffc53688516a6a83ae5f98d08a
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=792adb90fa724ce07c0171cbc96b9215af4b1045
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=5e0fb5df2ee871b841f96f9cb6a7f2784e96aa4e
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=785a19f9d1dd8a4ab2d0633be4656653bd3de1fc
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=f967db0b9ed44ec3057a28f3b28efc51df51b835
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=ba6f508d0ec4adb09f0a939af6d5e19cdfa8667d
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=f77084d96355f5fba8e2c1fb3a51a393b1570de7
[Regression Potential]
All upstream fixes, tested on the affected platform, backport changes are minimal.
[Original bug report]
Hi,
I'm reproducing a kernel bug in vmalloc_sync_all() with a 32-bit x86 kernel.
The problem appears in
Linux ubuntu 4.15.0-50-generic #54-Ubuntu SMP Mon May 6 18:45:45 UTC 2019 i686 i686 i686 GNU/Linux
Kernels 4.15.0-49 and prior work fine.
The kernel 4.18.0-20-generic works fine.
This problem has not been experienced with upstream Linux kernels.
It appears that invoking vmalloc_sync_all() a few times end up triggering this issue. This can be triggered by restarting the lttng-sessiond service with lttng-modules-dkms installed (sometimes a few restarts are needed to trigger the bug). This ends up unloading and reloading those modules, which issues a few vmalloc_sync_all() as side-effect.
I'm not reporting this issue with the "ubuntu-bug linux" command because it crashes the system on that kernel (system hangs, no console output).
My test system runs within a kvm virtual machine on a 64-bit host.
lsb release:
Description: Ubuntu 18.04.2 LTS
Release: 18.04
Information about my kernel:
linux-image-4.15.0-50-generic:
Installed: 4.15.0-50.54
Candidate: 4.15.0-50.54
Version table:
*** 4.15.0-50.54 500
500 http://ca.archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages
500 http://security.ubuntu.com/ubuntu bionic-security/main i386 Packages
100 /var/lib/dpkg/status
Information about lttng-modules-dkms:
lttng-modules-dkms:
Installed: 2.10.5-1ubuntu1.2
Candidate: 2.10.5-1ubuntu1.2
Version table:
*** 2.10.5-1ubuntu1.2 500
500 http://ca.archive.ubuntu.com/ubuntu bionic-updates/universe i386 Packages
100 /var/lib/dpkg/status
2.10.5-1ubuntu1 500
500 http://ca.archive.ubuntu.com/ubuntu bionic/universe i386 Packages |
|
| 2019-07-01 03:13:48 |
Khaled El Mously |
linux (Ubuntu Bionic): status |
Confirmed |
Fix Committed |
|
| 2019-07-03 13:07:06 |
Ubuntu Kernel Bot |
tags |
bionic |
bionic verification-needed-bionic |
|
| 2019-07-19 12:12:47 |
Kleber Sacilotto de Souza |
tags |
bionic verification-needed-bionic |
bionic verification-done-bionic |
|
| 2019-07-22 10:53:34 |
Launchpad Janitor |
linux (Ubuntu Bionic): status |
Fix Committed |
Fix Released |
|
| 2019-07-22 10:53:34 |
Launchpad Janitor |
cve linked |
|
2018-12126 |
|
| 2019-07-22 10:53:34 |
Launchpad Janitor |
cve linked |
|
2018-12127 |
|
| 2019-07-22 10:53:34 |
Launchpad Janitor |
cve linked |
|
2018-12130 |
|
| 2019-07-22 10:53:34 |
Launchpad Janitor |
cve linked |
|
2019-11085 |
|
| 2019-07-22 10:53:34 |
Launchpad Janitor |
cve linked |
|
2019-11091 |
|
| 2019-07-22 10:53:34 |
Launchpad Janitor |
cve linked |
|
2019-11815 |
|
| 2019-07-22 10:53:34 |
Launchpad Janitor |
cve linked |
|
2019-11833 |
|
| 2019-07-22 10:53:34 |
Launchpad Janitor |
cve linked |
|
2019-11884 |
|
| 2019-07-24 21:18:57 |
Brad Figg |
tags |
bionic verification-done-bionic |
bionic cscc verification-done-bionic |
|
| 2019-08-22 16:16:50 |
Ubuntu Kernel Bot |
tags |
bionic cscc verification-done-bionic |
bionic cscc verification-done-bionic verification-needed-xenial |
|
| 2019-11-14 04:26:33 |
Po-Hsu Lin |
linux (Ubuntu): status |
Confirmed |
Fix Released |
|
