ubuntu_boot test failures on b/linux-gcp-5.4 on c3a-standard instance
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
ubuntu-kernel-tests |
New
|
Undecided
|
Unassigned | ||
linux (Ubuntu) |
Fix Released
|
Undecided
|
Unassigned | ||
Focal |
Confirmed
|
Undecided
|
Unassigned |
Bug Description
Boot testing fails for b/gcp-5.4 on new c3a-standard instance as below:
Causes WARNING at: arch/arm64/
[ 0.060607] WARNING: CPU: 3 PID: 29 at /build/
[ 0.060612] WARNING: CPU: 2 PID: 23 at /build/
[ 0.060613] Modules linked in:
[ 0.060615] Modules linked in:
[ 0.060621] CPU: 3 PID: 29 Comm: migration/3 Not tainted 5.4.0-1122-gcp #131~18.04.1-Ubuntu
[ 0.060622] CPU: 2 PID: 23 Comm: migration/2 Not tainted 5.4.0-1122-gcp #131~18.04.1-Ubuntu
[ 0.060628] WARNING: CPU: 1 PID: 17 at /build/
[ 0.060629] pstate: 20400085 (nzCv daIf +PAN -UAO)
[ 0.060630] Modules linked in:
[ 0.060632] pc : spectre_
[ 0.060635] lr : spectre_
[ 0.060636] CPU: 1 PID: 17 Comm: migration/1 Not tainted 5.4.0-1122-gcp #131~18.04.1-Ubuntu
[ 0.060637] sp : ffff80001012bce0
[ 0.060639] pstate: 20400085 (nzCv daIf +PAN -UAO)
[ 0.060639] x29: ffff80001012bce0 x28: 0000000000000002
[ 0.060642] pc : spectre_
[ 0.060645] lr : spectre_
[ 0.060646] x27: ffff80001003bd10
[ 0.060647] sp : ffff8000100fbce0
[ 0.060647] x26: 0000000000000000
[ 0.060648] x29: ffff8000100fbce0
[ 0.060650] x28: 0000000000000002
[ 0.060652] x25: 0000000000000060
[ 0.060653] x24: 0000000000000000
[ 0.060655] x27: ffff80001003bd10 x26: 0000000000000000
[ 0.060657] x23: 0000000000000001
[ 0.060661] x22: ffffdca01ea9a3c0
[ 0.060663] x25: 0000000000000060 x24: 0000000000000000
[ 0.060665] x21: ffffdca01dda4970
[ 0.060668] x20: 0000000000000001
[ 0.060669] x23: 0000000000000001 x22: ffffdca01ea9a3c0
[ 0.060671] x19: ffffdca01ecca000
[ 0.060675] x18: ffffdca01dc56a10
[ 0.060680] x21: ffffdca01dda4970 x20: 0000000000000001
[ 0.060684] x17: 00000000bc8dcec3
[ 0.060687] x16: 00000000b1e1b9b5
[ 0.060688] x19: ffffdca01ecca000 x18: ffffdca01dc56a10
[ 0.060694] x17: 000000007e50fa5f
[ 0.060694] x15: 0000000000000001
[ 0.060695] x16: 000000009cd7320e
[ 0.060696] x14: ffff74e6bd9c01c8
[ 0.060699] x13: 0000000000000004
[ 0.060701] x15: 0000000000000001
[ 0.060702] x12: 0000000000000228
[ 0.060703] x14: ffff74e6bd99f1c8
[ 0.060705] x11: 0000000000000000
[ 0.060711] x13: 0000000000000004
[ 0.060712] x10: 0000000000000b00
[ 0.060713] x12: 0000000000000228
[ 0.060716] x11: 0000000000000000
[ 0.060722] x9 : ffff80001012bd20
[ 0.060723] x10: 0000000000000b00
[ 0.060724] x8 : ffff74e6ba3c66e0
[ 0.060726] x7 : 0000000000000000 x6 : ffffdca01d822000
[ 0.060727] x9 : ffff8000100fbd20
[ 0.060729] x8 : ffff74e6ba33a9e0
[ 0.060729] x5 : 00000000c00fac30
[ 0.060730] x4 : 0000000000000000
[ 0.060731] x7 : 0000000000000000
[ 0.060734] x6 : ffffdca01d822000
[ 0.060735] x3 : 000000000000000b
[ 0.060736] x2 : ffffdca01ecca508
[ 0.060738] x5 : 00000000c00fac30 x4 : 0000000000000000
[ 0.060740] x1 : 000000000000000b
[ 0.060742] x0 : 000000000000000b
[ 0.060746] x3 : 000000000000000b x2 : ffffdca01ecca508
[ 0.060749] Call trace:
[ 0.060751] x1 : 000000000000000b
[ 0.060753] spectre_
[ 0.060755] x0 : 000000000000000b
[ 0.060757] cpu_enable_
[ 0.060761] multi_cpu_
[ 0.060762] Call trace:
[ 0.060763] cpu_stopper_
[ 0.060766] spectre_
[ 0.060769] smpboot_
[ 0.060771] cpu_enable_
[ 0.060773] kthread+0x114/0x118
[ 0.060775] multi_cpu_
[ 0.060778] ret_from_
[ 0.060779] cpu_stopper_
[ 0.060780] ---[ end trace 82b1638f0eea221b ]---
[ 0.060783] smpboot_
[ 0.060785] kthread+0x114/0x118
[ 0.060787] ret_from_
[ 0.060789] ---[ end trace 82b1638f0eea221c ]---
[ 0.060797] pstate: 20400085 (nzCv daIf +PAN -UAO)
[ 0.060799] pc : spectre_
[ 0.060800] lr : spectre_
[ 0.060801] sp : ffff80001015bce0
[ 0.060802] x29: ffff80001015bce0 x28: 0000000000000002
[ 0.060804] x27: ffff80001003bd10 x26: 0000000000000000
[ 0.060805] x25: 0000000000000060 x24: 0000000000000000
[ 0.060806] x23: 0000000000000001 x22: ffffdca01ea9a3c0
[ 0.060807] x21: ffffdca01dda4970 x20: 0000000000000001
[ 0.060808] x19: ffffdca01ecca000 x18: ffffdca01dc56a10
[ 0.060809] x17: 000000002041ff62 x16: 0000000057377f7c
[ 0.060811] x15: 0000000000000001 x14: ffff74e6bd9e11c8
[ 0.060812] x13: 0000000000000004 x12: 0000000000000228
[ 0.060813] x11: 0000000000000000 x10: 0000000000000000
[ 0.060814] x9 : 0000000000000000 x8 : 0000000000000000
[ 0.060815] x7 : ffff74e6bd9f5bc0 x6 : ffffdca01d822000
[ 0.060816] x5 : 00000000c00fac30 x4 : 0000000000000000
[ 0.060817] x3 : 000000000000000b x2 : ffffdca01ecca508
[ 0.060818] x1 : 000000000000000b x0 : 000000000000000b
[ 0.060820] Call trace:
[ 0.060822] spectre_
[ 0.060823] cpu_enable_
[ 0.060826] multi_cpu_
[ 0.060827] cpu_stopper_
[ 0.060829] smpboot_
[ 0.060830] kthread+0x114/0x118
[ 0.060832] ret_from_
[ 0.060833] ---[ end trace 82b1638f0eea221d ]---
description: | updated |
tags: | added: 5.4 bionic gcp sru-20240108 ubuntu-boot |
Changed in linux (Ubuntu): | |
status: | New → Fix Released |
Here is my investigation when discussing this issue with Jesse:
This warning is triggered in spectre_ bhb_enable_ mitigation( ) from arch/arm64/ kernel/ cpu_errata. c, inside this if statement: bhb_loop_ affected( SCOPE_LOCAL_ CPU)) { bhb_loop_ affected( SCOPE_SYSTEM) ) {
/ *
* A57/A72-r0 will already have selected the
* spectre-indirect vector, which is sufficient
* for BHB too.
*/
if (!__this_ cpu_read( bp_hardening_ data.fn) )
kvm_setup_ bhb_slot( __spectre_ bhb_loop_ k8_start) ;
break;
kvm_ setup_bhb_ slot(__ spectre_ bhb_loop_ k24_start) ;
break;
kvm_ setup_bhb_ slot(__ spectre_ bhb_loop_ k32_start) ;
break;
default:
WARN_ ON_ONCE( 1);
this_ cpu_set_ vectors( EL1_VECTOR_ BHB_LOOP) ;
} else if (spectre_
switch (spectre_
case 8:
case 24:
case 32:
}
This is because the case value (spectre_ bhb_loop_ affected( SCOPE_SYSTEM) ) does not match, and falls into the "default" case.
This piece of code came from a SAUCE patch 14d45ef5 (UBUNTU: SAUCE: arm64: Mitigate spectre style branch history side channels), in which it moves the code that exist in arch/arm64/ kernel/ proton- pack.c to cpu_errata.c as mentioned in the commit message:
[ modified for stable, moved code to cpu_errata.c removed bitmap of
mitigations, use kvm template infrastructure ]
This issue does not exist in 5.15 kernel, and there is a new commit that added a new "case 11" to the switch-case, 0e5d5ae8 (arm64: Add AMPERE1 to the Spectre-BHB affected list).
According to Google [1]:
> C3A powered by AmpereOne processors
So I think this 0e5d5ae8 is very likely the fix we need for 5.4.
[1] https:/ /cloud. google. com/blog/ products/ compute/ whats-new- in-googles- workload- optimized- computing