wanted: *before-gc-hooks*

Bug #310117 reported by James Y Knight
4
Affects Status Importance Assigned to Milestone
SBCL
Confirmed
Wishlist
Unassigned

Bug Description

This adds a *before-gc-hooks* (analogous to *after-gc-hooks*) that was lost somewhere along the lines from cmucl to sbcl. It is a useful feature, although you have to be careful, of course, to not shoot yourself in the foot and try to cons yourself out of memory before the gc gets to run.

I'm using this to keep real-time statistics from the app's execution, without mixing in GC time, and for getting performance counter data from the app execution separate from GC.

Some comments from the sbcl-devel thread on this subject starting November 13, 2006 5:56:37 PM EST.

Nathan Froyd writes:
> I seem to remember that there were good reasons for removing it (the
> above warning about not consing yourself to death being one). Indeed,
> Juho's recent commit to sb-sprof removed this comment:
>
> ;; Disabled for now, since this was causing some problems with the
> ;; sampling getting turned off completely. --JES, 2004-06-19
> ;;
> ;; BEFORE-GC-HOOKS have exceedingly bad interactions with
> ;; threads. -- CSR, 2004-06-21
> ;;
> ;; (pushnew 'turn-off-sampling *before-gc-hooks*)

Nikodemus Siivola writes:
> Juho Snellman writes:
>
> > Hmm... What makes a before-gc-hook more problematic when run in a
> > separate thread?
>
> For SBCL the only issue is added cost (and consing) of signalling the
> housekeeping thread and waiting for it to finish running those hooks.
> I don't know if this would be significant or not.
>
> The user, however, loses the dynamic countour he was interested in.
> If before-hooks are run in a separate thread, the only thing they can
> inspect is global state.
>
> Moreover, pretty much the only bits of (local and global) state that
> should change during GC are weak pointers and GC statistics. These
> things should also change _only_ during GC, so after-hooks can easily
> remember their old values and use those to generate the statistics.
>
> Either I am missing something or before-hooks aren't really necessary
> if the user has access to the GC statistics he needs.
>
> > It seems to me that a general-purpose hook (even one that's documented
> > as "handle with care") would be preferrable to trying to directly
> > provide more GC statistics.
>
> Dunno, really. I don't think the amount of statistics that we'd need
> to give is overwhelming, but maybe there are funky uses for before-hooks
> that I am not seeing.
>
> One possibility would be to just keep *before-gc-hooks* and
> *after-gc-hooks* in the current thread, both marked "use with care",
> while ensuring that user-code never runs in a dead thread. Then add
> safe & sane *gc-hooks* and put them in the (future) housekeeping
> thread.

Revision history for this message
James Y Knight (foom) wrote :
Changed in sbcl:
importance: Undecided → Wishlist
status: New → Confirmed
Revision history for this message
Nikodemus Siivola (nikodemus) wrote :

Is there a specific need to have the lock held and interrupts disabled when running these hooks? They seem less likely to screw things up when executed before the lock is grabbed, inside a WITH-INTERRUPTS.

Additionally, if you want to maintain a sane pairing between before and after hooks, it seems sensible to ensure that GC's potentially triggered by the hooks or finalizers don't cause nested execution of GC hooks.

Alternative patch attached.

Changed in sbcl:
assignee: nobody → nikodemus
Revision history for this message
Nikodemus Siivola (nikodemus) wrote :
Revision history for this message
James Y Knight (foom) wrote :

I've no need to have locks held, since it's only for statistics gathering. Having the hooks always come paired up is certainly desirable.

Changed in sbcl:
assignee: Nikodemus Siivola (nikodemus) → nobody
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.