Memory leak in net/xfrm/xfrm_state.c - 8 pages per ipsec connection
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
linux (Ubuntu) |
Invalid
|
Undecided
|
Unassigned | ||
Bionic |
Fix Released
|
High
|
Stefan Bader | ||
Disco |
Fix Released
|
High
|
Stefan Bader | ||
Eoan |
Fix Released
|
High
|
Stefan Bader |
Bug Description
[SRU Justification]
== Impact ==
An upstream change in v4.11 made xfrm loose memory (8 pages per ipsec connection). This was fixed in v5.4 by:
commit 86c6739eda7d "xfrm: Fix memleak on xfrm state destroy"
== Fix ==
Pick the upstream fix into all affected series.
== Testcase ==
see below
== Risk of Regression ==
Low, the change adds a single memory release case in one driver. The effect can be verified.
---
Ubuntu linux distro, 4.15.0-62 kernel, server platform.
This OS is used as an IPSec VPN gateway. It serves up to several hundred concurrent connections
In an attempt to upgrade from the 4.4 kernel to 4.15, the team noticed that VPN gateway VMs were running out of physical memory after 12-48 hours, depending on load.
Attachments from a server machine in this state in attached leakinfo.txt
output of free -t
output of /proc/meminfo in out of memory condition
output of /slabtop -o -sc
/sys/kernel/
Patches for 4.15 and 5.4
Highlight from page_owner, we can see the leak is a buffer associated with the ipsec impelementation. Each connection leaks 32k of memory via alloc_page with order=3
100960 times:
Page allocated via order 3, mask 0x1085220(
get_page_
__alloc_
alloc_
skb_page_
esp_output_
esp_output+
xfrm_output_
xfrm_output+
xfrm4_
__xfrm4_
xfrm4_
ip_forward_
ip_forward+
ip_rcv_
ip_rcv+0x292/0x360
__netif_
Patch to fix this issue in 4.15 (tested and verified on same server exhibiting above leak):
diff --git a/net/xfrm/
index 728272f..7842f83 100644
--- a/net/xfrm/
+++ b/net/xfrm/
@@ -451,6 +451,10 @@ static void xfrm_state_
}
+
+ if(x->xfrag.page)
+ put_page(
+
kfree(x);
}
Patch for master branch (5.4 I believe) from Paul Wouters (<email address hidden>)
diff --git a/net/xfrm/
index c6f3c4a1bd99.
--- a/net/xfrm/
+++ b/net/xfrm/
@@ -495,6 +495,8 @@ static void ___xfrm_
}
+ if (x->xfrag.page)
+ put_page(
Severity: Critical - we are unable to use any kernel later than 4.11, and are sticking with 4.4 in production.
CVE References
description: | updated |
description: | updated |
Changed in linux (Ubuntu Eoan): | |
assignee: | nobody → Stefan Bader (smb) |
Changed in linux (Ubuntu Disco): | |
assignee: | nobody → Stefan Bader (smb) |
Changed in linux (Ubuntu Bionic): | |
assignee: | nobody → Stefan Bader (smb) |
Changed in linux (Ubuntu Bionic): | |
status: | Triaged → Fix Committed |
Changed in linux (Ubuntu Disco): | |
status: | Triaged → Fix Committed |
Changed in linux (Ubuntu Eoan): | |
status: | Triaged → Fix Committed |
Changed in linux (Ubuntu Bionic): | |
status: | Confirmed → Fix Committed |
tags: |
added: verification-done-bionic removed: verification-needed-bionic |
tags: |
added: verification-done-eoan removed: verification-needed-eoan |
This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:
apport-collect 1853197
and then change the status of the bug to 'Confirmed'.
If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.
This change has been made by an automated script, maintained by the Ubuntu Kernel Team.