Very slow compile due to method definitions

Bug #1946786 reported by Paul F. Dietz
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
SBCL
Undecided
Unassigned

Bug Description

I am noticing a situation where a program begins to compile very slowly. The program has some generic functions with a large number of methods on many classes. When I profile the compile, 95% of the time is being spent below these functions (each 95+% of time in the profile graph):

SB-PCL::LOAD-DEFMETHOD-INTERNAL
SB-PCL::REAL-ADD-NAMED-METHOD
SB-PCL::REAL-ENSURE-GF-USING-CLASS--GENERIC-FUNCTION
SB-PCL::NOTE-GF-SIGNATURE
SB-PCL::FTYPE-DECLARATION-FROM-LAMBDA-LIST
SB-INT:GLOBAL-FTYPE
SB-PCL::COMPUTE-GF-FTYPE
(SB-PCL::FAST-METHOD SB-PCL::GENERIC-FUNCTION-PRETTY-ARGLIST (STANDARD-GENERIC-FUNCTION))

(each of those calls the next, each with about the same number of calls) followed by

SB-PCL::UNPARSE-SPECIALIZERS (73%)
SB-PCL::REAL-UNPARSE-SPECIALIZER-USING-CLASS (45%)

None of this happens when the fasl file is loaded. A workaround is to compile these files separately then quickload the fasl files, but that's a stopgap measure, not a solution.

It seems likely to me that a poorly scaling algorithm is being used to compute something that isn't used for anything. Can this be sped up by computing and caching the gf signature lazily, when needed?

The program where this is coming up is a real program, not an artificial test case.

description: updated
Revision history for this message
Stas Boukarev (stassats) wrote :

Would be nice to have a test case.

Revision history for this message
Paul F. Dietz (paul-f-dietz) wrote :

I wanted to see if that was enough to figure out what was going on. It seems the time spent to create a large number of methods (including accessor methods for slots) is nonlinear in the number of methods for the generic function. By the end of the compile it's spending up to half a second just handling a single defmethod for certain generic functions.

When I have the time I'll see if I can put together a simplified test case to confirm this.

Revision history for this message
Stas Boukarev (stassats) wrote :

I know what's going on, but I want to measure the effect of changes.

Revision history for this message
Paul F. Dietz (paul-f-dietz) wrote :

If you can commit something on a branch I can measure.

Revision history for this message
Stas Boukarev (stassats) wrote :
Revision history for this message
Paul F. Dietz (paul-f-dietz) wrote :

This caused only a slight reduction in runtime, if any.

Revision history for this message
Stas Boukarev (stassats) wrote :

Can you try the latest commit?

Revision history for this message
Paul F. Dietz (paul-f-dietz) wrote :

Vastly better. Compile time reduced from first run by a factor of 20.

Stas Boukarev (stassats)
Changed in sbcl:
status: New → Fix Committed
Revision history for this message
Paul F. Dietz (paul-f-dietz) wrote :

Thanks! This deserves an entry in the next release note.

Revision history for this message
Stas Boukarev (stassats) wrote :

That was a regression, actually.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers