Details added in fixture cleanup are not gathered

[Jonathan Lange]

useFixture does this::
        try:
            fixture.setUp()
        except:
            gather_details(fixture.getDetails(), self.getDetails())
            raise
        else:
            self.addCleanup(fixture.cleanUp)
            self.addCleanup(
                gather_details, fixture.getDetails(), self.getDetails())
            return fixture

(python-fixtures has a useFixture that's very similar).

Note that the details gathering will run *before* the call to cleanup. This means any details added in cleanup will not be included in the gathered details.

Here's a test that could go into testtools's test_fixturesupport::

    def test_useFixture_details_captured_from_cleanups(self):
        # Details added during fixture clean up are gathered.
        class CleanupDetail(fixtures.Fixture):
            def setUp(self):
                fixtures.Fixture.setUp(self)
                self.addDetail('content', content.text_content("foobar"))
                self.addCleanup(
                    self.addDetail, 'cleanup', content.text_content("foobar"))
        fixture = CleanupDetail()
        class SimpleTest(TestCase):
            def test_foo(self):
                self.useFixture(fixture)
        result = ExtendedTestResult()
        SimpleTest('test_foo').run(result)
        self.assertEqual('addSuccess', result._events[-2][0])
        details = result._events[-2][2]
        self.assertEqual(['cleanup', 'content'], sorted(details))
        self.assertEqual('foobar', ''.join(details['cleanup'].iter_text()))

It's a pretty easy fix (just swap the addCleanup calls around), but I'm not sure if that introduce different bugs.

[Jonathan Lange (jml)]

Not being able to add details during cleanup removes a valuable channel for providing debugging information.

[Robert Collins (lifeless)]

Details can be registered before their content is known, quite trivially. The issue with gathering details after cleanup is that cleanup tends to free and toss away state. So for most well written fixtures its impossible to get details *after* cleanup has happened.

I think the current code is entirely correct but we may want to document this behaviour, and why.

[Jonathan Lange (jml)]

Fair enough. IIRC, the particular context where this came up was debugging fixtures that weren't able to clean up properly. I do think handling fixtures that are *not* well-written is an important use case.

[Robert Collins (lifeless)]

I'm going to propose that we attach details to an exception and raise that when setUp fails.

https://rbtcollins.wordpress.com/2015/06/22/revisiting-the-fixture-api-handling-leaky-resources/ is my analysis of why.

[Jonathan Lange (jml)]

Good analysis.

Only point I'd add is that this makes the `setUp` behaviour of fixtures
different from the `setUp` of `TestCase`. If `TestCase.setUp` raises an
error, `TestCase.tearDown` is not run. This difference will probably
confuse people.

On Mon, 22 Jun 2015 at 03:20 Robert Collins <email address hidden>
wrote:

> I'm going to propose that we attach details to an exception and raise
> that when setUp fails.
>
> https://rbtcollins.wordpress.com/2015/06/22/revisiting-the-fixture-api-
> handling-leaky-resources/
> <https://rbtcollins.wordpress.com/2015/06/22/revisiting-the-fixture-api-handling-leaky-resources/>
> is my analysis of why.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/897190
>
> Title:
> Details added in fixture cleanup are not gathered
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/python-fixtures/+bug/897190/+subscriptions
>

[Robert Collins (lifeless)]

Note to future self: passing something to hand fixtures too into cleanUp would be a way past this, and can be done backward compatibly I think.

[Jonathan Lange (jml)]

I guess we've decided that this bug isn't actually a bug, but in fact desired behaviour.