zope.testrunner should only profile actual tests, not setup

[Daniel Nouri]

I'm trying to use the testrunner to profile my application. For this, it appears that layer setup code isn't really interesting. Thus, I've written this patch to defer calls to profile.enable() and profile.disable() to when the actual testing happens.

[Roman Joost (romanofski)]

I've had a look at this. Unfortunately the submitted patch doesn't apply anymore, because the code has already been refactored a lot.

Currently the profiler is enabled before the layer is setup:

    zope.testing.testrunner.profiling.py:125 (late_setup)

and stopped after all tests ran, but before the layer is teared down:

    zope.testing.testrunner.profiling.py:126 (early_teardown)

Please also check zope.testing.testrunner.feature.Feature for all the stages.

Now here's the thing: it looks like your suggested solution is only half implemented. To avoid the layer setup showing up, the profiler needs to be started/stopped multiple times (after/before each test_setup, test_teardown to be precise), but apparently this doesn't work with hotshot. I checked the python doc for hotshot and it appears to me, that it doesn't provide some kind of cumulative feature.

Did you intend with your patch to start/stop hotshot multiple times? If so, how is that supposed to work? If not, the profiler disable method should be set to "layer_setup".

[Colin Watson (cjwatson)]

The zope.testing project on Launchpad has been archived at the request of the Zope developers (see https://answers.launchpad.net/launchpad/+question/683589 and https://answers.launchpad.net/launchpad/+question/685285). If this bug is still relevant, please refile it at https://github.com/zopefoundation/zope.testing.

[Colin Watson (cjwatson)]

The zope.testrunner project on Launchpad has been archived at the request of the Zope developers (see https://answers.launchpad.net/launchpad/+question/683589 and https://answers.launchpad.net/launchpad/+question/685285). If this bug is still relevant, please refile it at https://github.com/zopefoundation/zope.testrunner.