GC runtime is O (heap_size * number_of_threads)

Bug #1339924 reported by Ilya Perminov on 2014-07-09
16
This bug affects 3 people
Affects Status Importance Assigned to Milestone
SBCL
Undecided
Unassigned

Bug Description

The garbage collector becomes very slow as the number of threads grows. The problem is in scavenge_newspace_generation (in gencgc.c). It invokes gc_alloc_update_all_page_tables in loop over sets of new areas. gc_alloc_update_all_page_tables, in its turn, iterates over all threads.
There seems to be an easy fix to the problem: GC is single-threaded and all memory allocations during a collection happen in the GC thread, so it is sufficient to update all the the page tables once at the beginning of a collection and update only page tables of the GC thread during the collection.

Ilya Perminov (iperminov) wrote :

Here is a test case.

Ilya Perminov (iperminov) wrote :
description: updated
Ilya Perminov (iperminov) wrote :

My patch seems to work well. We've been running a patched SBCL on a dozen of servers since I reported the bug.

Mark W Johnson (mwjohnson0) wrote :

I just wanted to comment that we have been using Ilya's patch on
all of our systems for more than two years now. This includes
3-5 production systems and more than a dozen development systems.
Each system includes two heavily used sbcl images (usually 24/7).
No sign of trouble from this fix. We could not effectively
operate without it.

I do see that no one else appears to have been bothered by this
problem, and understand this may thus not appear to merit
attention. But the problem is fairly generic.

For what its worth, I can say that the patch still applies with
no conflict with current sbcl sources.

Mark W Johnson
D-Wave Systems Inc

Stas Boukarev (stassats) on 2016-10-06
Changed in sbcl:
assignee: nobody → Stas Boukarev (stassats)
Stas Boukarev (stassats) wrote :

In 55174a78dc1e062547c8d7b539e2c6b07c061b6c

Changed in sbcl:
assignee: Stas Boukarev (stassats) → nobody
status: New → Fix Committed
Changed in sbcl:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Patches

Bug attachments