powerpc/livepatch: Implement reliable stack tracing for the consistency model

Bug #1771844 reported by bugproxy on 2018-05-17
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
The Ubuntu-power-systems project
High
Canonical Kernel Team
linux (Ubuntu)
Status tracked in Cosmic
Bionic
High
Joseph Salisbury
Cosmic
High
Joseph Salisbury

Bug Description

== SRU Justification ==
Livepatch has a consistency model which is a hybrid of kGraft and kpatch:
it uses kGraft's per-task consistency and syscall barrier switching
combined with kpatch's stack trace switching. The current approach is
stack checking of sleeping tasks. If no affected functions are on the
stack of a given task, the task is patched. In most cases this will patch
most or all of the tasks on the first try. Otherwise, it'll keep trying
periodically. This patch implements the reliable stack tracing for
consistency model a.k.a HAVE_RELIABLE_STACKTRACE.

This will help in switching livepatching implementation to basic per-task
consistency model. It is the foundation, which will help us enable
security patches changing function or data semantics. This is the biggest
remaining piece needed on ppc64le to make livepatch more generally useful.

== Fix ==
df78d3f61480 ("powerpc/livepatch: Implement reliable stack tracing for the consistency model")

== Regression Potential ==
Low. Limited to powerpc.

== Test Case ==
A test kernel was built with this patch and tested by the original bug reporter.
The bug reporter states the test kernel resolved the bug.

https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git/commit/?id=df78d3f6148092d33a9a24c7a9cfac

bugproxy (bugproxy) on 2018-05-17
tags: added: architecture-ppc64le bugnameltc-167879 severity-critical targetmilestone-inin1804
Changed in ubuntu:
assignee: nobody → Ubuntu on IBM Power Systems Bug Triage (ubuntu-power-triage)
affects: ubuntu → linux (Ubuntu)
Changed in ubuntu-power-systems:
status: New → Triaged
tags: added: triage-g
Changed in ubuntu-power-systems:
importance: Undecided → High
assignee: nobody → Canonical Kernel Team (canonical-kernel-team)
bugproxy (bugproxy) on 2018-05-17
tags: added: targetmilestone-inin18041
removed: targetmilestone-inin1804
bugproxy (bugproxy) on 2018-05-17
tags: added: targetmilestone-inin1804
removed: targetmilestone-inin18041
Changed in linux (Ubuntu):
importance: Undecided → High
status: New → Triaged
Changed in linux (Ubuntu Bionic):
status: New → In Progress
Changed in linux (Ubuntu Cosmic):
status: Triaged → In Progress
Changed in linux (Ubuntu Bionic):
assignee: nobody → Joseph Salisbury (jsalisbury)
Changed in linux (Ubuntu Cosmic):
assignee: Ubuntu on IBM Power Systems Bug Triage (ubuntu-power-triage) → Joseph Salisbury (jsalisbury)
Changed in linux (Ubuntu Bionic):
importance: Undecided → High
Joseph Salisbury (jsalisbury) wrote :

I built a test kernel with commit df78d3f6148092d33a9a24c7a9cfac3d0220b484. The test kernel can be downloaded from:
http://kernel.ubuntu.com/~jsalisbury/lp1771844

Can you test this kernel and see if it resolves this bug?

Note about installing test kernels:
• If the test kernel is prior to 4.15(Bionic) you need to install the linux-image and linux-image-extra .deb packages.
• If the test kernel is 4.15(Bionic) or newer, you need to install the linux-image-unsigned, linux-modules and linux-modules-extra .deb packages.

Thanks in advance!

------- Comment From <email address hidden> 2018-05-22 00:13 EDT-------
(In reply to comment #5)
> I built a test kernel with commit df78d3f6148092d33a9a24c7a9cfac3d0220b484.
> The test kernel can be downloaded from:
> http://kernel.ubuntu.com/~jsalisbury/lp1771844
>
> Can you test this kernel and see if it resolves this bug?
>
> Note about installing test kernels:
> ? If the test kernel is prior to 4.15(Bionic) you need to install the
> linux-image and linux-image-extra .deb packages.
> ? If the test kernel is 4.15(Bionic) or newer, you need to install the
> linux-image-unsigned, linux-modules and linux-modules-extra .deb packages.
>
> Thanks in advance!

Thanks for building the kernel with the patch. Can you share the kernel sources, so that can I build the livepatch modules based on the sources. Standard .config, does not build the samples/livepatch to do the basic testing.
cat /boot/config-4.15.0-20-generic |grep -i livepatch
CONFIG_HAVE_LIVEPATCH=y
CONFIG_LIVEPATCH=y

I would also try to build livepatch modules other than the sample modules in the kernel sources to verify save_stack_trace functionality.

Joseph Salisbury (jsalisbury) wrote :

I built a new test kernel and included the source package.

The test kernel can be downloaded from:
http://kernel.ubuntu.com/~jsalisbury/lp1771844

bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2018-05-23 06:27 EDT-------
(In reply to comment #7)
> I built a new test kernel and included the source package.
>
> The test kernel can be downloaded from:
> http://kernel.ubuntu.com/~jsalisbury/lp1771844

Thanks for sharing the kernel sources. I was able to verify the patch with livepatch-sample.ko :
[ 4772.567633] livepatch_sample: tainting kernel with TAINT_LIVEPATCH
[ 4772.569747] livepatch_sample: module verification failed: signature and/or required key missing - tainting kernel
[ 4772.579440] livepatch: enabling patch 'livepatch_sample'
[ 4772.580686] livepatch: 'livepatch_sample': starting patching transition
[ 4777.032952] livepatch: 'livepatch_sample': patching complete
[ 4800.732461] livepatch: 'livepatch_sample': starting unpatching transition
[ 4804.969121] livepatch: 'livepatch_sample': unpatching complete

transition (un)patching works as expected.

Changed in ubuntu-power-systems:
status: Triaged → In Progress
Joseph Salisbury (jsalisbury) wrote :
description: updated
Changed in linux (Ubuntu Bionic):
status: In Progress → Fix Committed
Manoj Iyer (manjo) on 2018-06-11
Changed in ubuntu-power-systems:
status: In Progress → Fix Committed
Changed in linux (Ubuntu Cosmic):
status: In Progress → Fix Committed
Brad Figg (brad-figg) wrote :

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-bionic' to 'verification-done-bionic'. If the problem still exists, change the tag 'verification-needed-bionic' to 'verification-failed-bionic'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: verification-needed-bionic
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2018-06-15 04:33 EDT-------
The proposed Bionic kernel enables the 'Implement reliable stack tracing for the consistency model' patch.

In brief:
------------
After upgrading the system by enabling bionic-proposed to sources.list:
deb http://archive.ubuntu.com/ubuntu/ bionic-proposed restricted main multiverse universe

system booted into kernel version 4.15.0-24-generic, using 4.15.0-24-generic sources (apt source linux) to build livepatch modules:
# insmod ./livepatch-0001-mm-thp-Do-not-make-page-table-dir.ko
# dmesg
.....
[ 1827.037453] livepatch_0001_mm_thp_Do_not_make_page_table_dir: loading out-of-tree module taints kernel.
[ 1827.037458] livepatch_0001_mm_thp_Do_not_make_page_table_dir: tainting kernel with TAINT_LIVEPATCH
[ 1827.043106] livepatch_0001_mm_thp_Do_not_make_page_table_dir: module verification failed: signature and/or required key missing - tainting kernel
[ 1827.176933] livepatch: enabling patch 'livepatch_0001_mm_thp_Do_not_make_page_table_dir'
[ 1827.179784] livepatch: 'livepatch_0001_mm_thp_Do_not_make_page_table_dir': starting patching transition
[ 1827.179785] livepatch: 'livepatch_0001_mm_thp_Do_not_make_page_table_dir': patching complete
[ 2526.521348] livepatch: 'livepatch_0001_mm_thp_Do_not_make_page_table_dir': starting unpatching transition
[ 2526.525554] livepatch: 'livepatch_0001_mm_thp_Do_not_make_page_table_dir': unpatching complete

tags: added: verification-done-bionic
removed: verification-needed-bionic
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers