Kernel crash vmcore controller reboot related to BIOS/Kernel CVE
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
StarlingX |
Fix Released
|
Medium
|
Jiping Ma |
Bug Description
Brief Description
Standby controller crashes with vmcore generated.
vmcore generated on controller-1 of Radio site 44895 running 21.12 Patch6 and Samsung R2.
Might be related to Spectre/Meltdown correction BIOS/Kernel CVE level of BIOS and Kernel. It happens when the pod/ containers using RT cores.
Information requested from Dell to get the BIOS/Kernel CVE level for Dell XR11 servers.
Interestingly after the incident we saw an excessive etc slowdown on controller-0 from the smu-eru database there is a steep drop in the free memory after 23:29:06.
Severity
Major: Service impacting reboot.
Steps to Reproduce
Experienced on Radio site running starlingx on controller-1.
More sites might be affected, to be updated in this JIRA as soon as this is discovered.
Expected Behavior
System should protect itself from rebooting and crash the application process that is causing the issue on the Kernel.
Actual Behavior
System crashes and vmcore generated.
Reproducibility
such vmcore seen 5 times
Free memory consumed at 2 Mega Bytes per seconds seen on another site on 88807 worker-1 and 96090 controller-1
etcd slow downs seen on almost all sites.
kernel BUG at kernel/
invalid opcode: 0000 [#1] PREEMPT_RT SMP NOPTI
......
Call Trace:
rt_spin_
? update_
rt_spin_
? update_
rt_spin_
free_unref_
__vunmap+
put_task_
__put_
rt_mutex_
task_blocks_
rt_spin_
rt_spin_
rt_spin_
free_unref_
release_
tlb_flush_
tlb_finish_
zap_page_
? find_vma+0x16/0x70
do_madvise+
? do_epoll_
? __x64_sys_
__x64_
do_syscall_
entry_
Changed in starlingx: | |
assignee: | nobody → Jiping Ma (jma11) |
Changed in starlingx: | |
importance: | Undecided → Medium |
tags: | added: stx.9.0 stx.distro.other |
Fix proposed to branch: master /review. opendev. org/c/starlingx /kernel/ +/891652
Review: https:/