ibrs/ibpb fixes result in excessive kernel logging
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
linux (Ubuntu) |
Fix Released
|
Medium
|
Kamal Mostafa | ||
Trusty |
Fix Released
|
Medium
|
Kamal Mostafa | ||
Xenial |
Fix Released
|
Medium
|
Kamal Mostafa | ||
Artful |
Fix Released
|
Undecided
|
Kamal Mostafa |
Bug Description
Since at least kernel 4.4.0-116, every invocation of `sysctl -a` results in kernel logs similar to the following:
% sysctl -a &>/dev/null; dmesg -T | tail -8
[Wed Mar 14 00:06:36 2018] sysctl_ibrs_enabled = 0, sysctl_ibpb_enabled = 0
[Wed Mar 14 00:06:36 2018] use_ibrs = 4, use_ibpb = 4
[Wed Mar 14 00:06:36 2018] read cpu 0 ibrs val 0
[Wed Mar 14 00:06:36 2018] read cpu 1 ibrs val 0
[Wed Mar 14 00:06:36 2018] sysctl_ibrs_enabled = 0, sysctl_ibpb_enabled = 0
[Wed Mar 14 00:06:36 2018] use_ibrs = 4, use_ibpb = 4
[Wed Mar 14 00:06:36 2018] read cpu 0 ibrs val 0
[Wed Mar 14 00:06:36 2018] read cpu 1 ibrs val 0
The output varies with the number of CPUs.
After digging a bit, it turns out this is triggered upon every read of `kernel.ibrs_dump`:
% for i in {1..3}; do sysctl kernel.ibrs_dump; dmesg -T | tail -8; echo; sleep 1; done
kernel.ibrs_dump = 0
[Wed Mar 14 00:08:48 2018] sysctl_ibrs_enabled = 0, sysctl_ibpb_enabled = 0
[Wed Mar 14 00:08:48 2018] use_ibrs = 4, use_ibpb = 4
[Wed Mar 14 00:08:48 2018] read cpu 0 ibrs val 0
[Wed Mar 14 00:08:48 2018] read cpu 1 ibrs val 0
[Wed Mar 14 00:08:48 2018] sysctl_ibrs_enabled = 0, sysctl_ibpb_enabled = 0
[Wed Mar 14 00:08:48 2018] use_ibrs = 4, use_ibpb = 4
[Wed Mar 14 00:08:48 2018] read cpu 0 ibrs val 0
[Wed Mar 14 00:08:48 2018] read cpu 1 ibrs val 0
kernel.ibrs_dump = 0
[Wed Mar 14 00:08:49 2018] sysctl_ibrs_enabled = 0, sysctl_ibpb_enabled = 0
[Wed Mar 14 00:08:49 2018] use_ibrs = 4, use_ibpb = 4
[Wed Mar 14 00:08:49 2018] read cpu 0 ibrs val 0
[Wed Mar 14 00:08:49 2018] read cpu 1 ibrs val 0
[Wed Mar 14 00:08:49 2018] sysctl_ibrs_enabled = 0, sysctl_ibpb_enabled = 0
[Wed Mar 14 00:08:49 2018] use_ibrs = 4, use_ibpb = 4
[Wed Mar 14 00:08:49 2018] read cpu 0 ibrs val 0
[Wed Mar 14 00:08:49 2018] read cpu 1 ibrs val 0
kernel.ibrs_dump = 0
[Wed Mar 14 00:08:50 2018] sysctl_ibrs_enabled = 0, sysctl_ibpb_enabled = 0
[Wed Mar 14 00:08:50 2018] use_ibrs = 4, use_ibpb = 4
[Wed Mar 14 00:08:50 2018] read cpu 0 ibrs val 0
[Wed Mar 14 00:08:50 2018] read cpu 1 ibrs val 0
[Wed Mar 14 00:08:50 2018] sysctl_ibrs_enabled = 0, sysctl_ibpb_enabled = 0
[Wed Mar 14 00:08:50 2018] use_ibrs = 4, use_ibpb = 4
[Wed Mar 14 00:08:50 2018] read cpu 0 ibrs val 0
[Wed Mar 14 00:08:50 2018] read cpu 1 ibrs val 0
Those tests were against an EC2 instance running Ubuntu 4.4.0-116.
Normally this would not be the biggest concern but we have tooling that gathers instance info on a schedule, including sysctl output, thus resulting in the kernel ring buffer being full of nothing but said output in most cases and hindering live troubleshooting as a result.
Changed in linux (Ubuntu Trusty): | |
status: | New → Triaged |
importance: | Undecided → Medium |
assignee: | nobody → Canonical Kernel Team (canonical-kernel-team) |
Changed in linux (Ubuntu Xenial): | |
status: | Confirmed → Triaged |
assignee: | nobody → Canonical Kernel Team (canonical-kernel-team) |
Changed in linux (Ubuntu Trusty): | |
assignee: | Canonical Kernel Team (canonical-kernel-team) → Kamal Mostafa (kamalmostafa) |
Changed in linux (Ubuntu Xenial): | |
assignee: | Canonical Kernel Team (canonical-kernel-team) → Kamal Mostafa (kamalmostafa) |
Changed in linux (Ubuntu Trusty): | |
status: | Triaged → In Progress |
Changed in linux (Ubuntu Xenial): | |
status: | Triaged → In Progress |
Changed in linux (Ubuntu Artful): | |
status: | New → In Progress |
assignee: | nobody → Kamal Mostafa (kamalmostafa) |
Changed in linux (Ubuntu): | |
status: | Fix Committed → Fix Released |
This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:
apport-collect 1755627
and then change the status of the bug to 'Confirmed'.
If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.
This change has been made by an automated script, maintained by the Ubuntu Kernel Team.