Performance degradation with test loading in 2.5

[Jim Ivey]

A performance degradation was introduced in the 2.5 timeframe that greatly increases the amount of time required to load a test assembly containing a large number of fixtures that inherit from a base class with a large number of assertion "helper" methods.

An example assembly was created with 1000 fixtures each containing 10 tests. The GUI was tested for both 2.4.6 and 2.5, and each loaded the test assembly in roughly 15 seconds. Then 400 methods were added to the base class that all of the fixtures inherited from, and while the time for 2.4.6 increased to roughly a minute, the time for 2.5 increased to 18 minutes.

Initial belief is that this is due to some caching that was introduced in the 2.5 timeframe, with cache keys that are based on the reflected type, meaning that each of the subclasses would have its own 400 cached key/values.

[Charlie Poole (charlie.poole)]

Thanks.

Charlie

> -----Original Message-----
> From: <email address hidden> [mailto:<email address hidden>] On
> Behalf Of Jim Ivey
> Sent: Friday, August 28, 2009 7:33 PM
> To: <email address hidden>
> Subject: [Bug 420897] [NEW] Performance degradation with test
> loading in 2.5
>
> Public bug reported:
>
> A performance degradation was introduced in the 2.5 timeframe
> that greatly increases the amount of time required to load a
> test assembly containing a large number of fixtures that
> inherit from a base class with a large number of assertion
> "helper" methods.
>
> An example assembly was created with 1000 fixtures each
> containing 10 tests. The GUI was tested for both 2.4.6 and
> 2.5, and each loaded the test assembly in roughly 15 seconds.
> Then 400 methods were added to the base class that all of
> the fixtures inherited from, and while the time for 2.4.6
> increased to roughly a minute, the time for 2.5 increased to
> 18 minutes.
>
> Initial belief is that this is due to some caching that was
> introduced in the 2.5 timeframe, with cache keys that are
> based on the reflected type, meaning that each of the
> subclasses would have its own 400 cached key/values.
>
> ** Affects: nunitv2
> Importance: Undecided
> Status: New
>
> --
> Performance degradation with test loading in 2.5
> https://bugs.launchpad.net/bugs/420897
> You received this bug notification because you are the
> registrant for NUnit V2.
>