On boot excessive number of kworker threads are running
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
linux (Ubuntu) |
Fix Released
|
Medium
|
Colin Ian King | ||
Yakkety |
Fix Released
|
Undecided
|
Colin Ian King | ||
Zesty |
Fix Released
|
Medium
|
Colin Ian King |
Bug Description
[SRU REQUEST, Yakkety]
Ubuntu Yakkety 4.8 kernels have an excessive amount of kworker threads running, this is especially noticeable on boot, one can easily have > 1000 kworker threads on a 4 CPU box.
Bisected this down to:
commit 81ae6d03952c1bf
Author: Vladimir Davydov <email address hidden>
Date: Thu May 19 17:10:34 2016 -0700
mm/slub.c: replace kick_all_
kmem_cache_shrink()
[FIX]
The synchronize_sched calls seem to create all these excessive kworker threads. This is fixed with upstream commit:
commit 89e364db71fb5e7
Author: Vladimir Davydov <email address hidden>
Date: Mon Dec 12 16:41:32 2016 -0800
slub: move synchronize_sched out of slab_mutex on shrink
synchronize
owned by a memory cgroup being destroyed may take quite some time. What
is worse, it's currently called under the slab_mutex, stalling all works
doing cache creation/
Actually, there isn't much point in calling synchronize_sched() for each
cache - it's enough to call it just once - after setting cpu_partial for
all caches and before shrinking them. This way, we can also move it out
of the slab_mutex, which we have to hold for iterating over the slab
cache list.
[TEST CASE]
Without the fix, boot a Yakkety and count the number of kthreads:
ps -ef | grep kworker | wc -l
1034
With the fix, boot count the number kthreads and it will be dramatically less:
ps -ef | grep kworker | wc -l
32
Since this touches the slub allocator and cgroups too, I have regression tested this against the kernel-team autotest regression tests to sanity check this fix. All seems OK.
Note: this only affects kernels from 4.7-rc1 through to 4.8
Changed in linux (Ubuntu): | |
status: | New → In Progress |
importance: | Undecided → Medium |
assignee: | nobody → Colin Ian King (colin-king) |
Changed in linux (Ubuntu Yakkety): | |
assignee: | nobody → Colin Ian King (colin-king) |
status: | New → In Progress |
Changed in linux (Ubuntu Zesty): | |
status: | In Progress → Fix Released |
Changed in linux (Ubuntu Yakkety): | |
status: | In Progress → Fix Committed |
I didn't see the third commit mentioned which I thought was also required:
mm: memcontrol: use special workqueue for creating per-memcg caches
Anyway, I'll test also, once kernel 4.10-rc1 is ready.
The upstream bug report is this one: /bugzilla. kernel. org/show_ bug.cgi? id=172991
https:/