test documentation bug

[Giampaolo Rodolà]

In:
http://grok.zope.org/documentation/how-to/tests-with-grok-testing
...at a certain point grok.testing.register_all_tests function is mentioned but this does not exist:

>>> import grok
>>> test_suite = grok.testing.register_all_tests('sample')

    test_suite = grok.testing.register_all_tests('bgb')
AttributeError: 'module' object has no attribute 'register_all_tests'

[Ben Caddis (nebuloustwo)]

It appears that a lot of the current documentation is in need of updating.

Currently (as of Grok 1.2.1), we need to use z3c.testsetup. So src/myapp/tests.py will typically look like this:

-----
import os.path
import z3c.testsetup
from zope.app.wsgi.testlayer import BrowserLayer

import myapp

browser_layer = BrowserLayer(myapp)

test_suite = z3c.testsetup.register_all_tests(
    'myapp', globs={'getRootFolder': browser_layer.getRootFolder})
-----

[Giampaolo Rodolà (g-rodola)]

My personal opinion is that the user should not deal with this kind of stuff at all, if possible.
grokproject should automatically generate all the necessary parts to have a ready-to-use unit and functional test suite for the app.

Ideally, by using the current naming convention (which I personally don't like) the structure should look like this:

tests.py
app.txt
app_tests/test_app.py (within TestUnit and TestFunctional classes)

...by using a naming convention I like more:

tests/config.py
tests/test_doctest.txt
tests/test_app.py

[Giampaolo Rodolà (g-rodola)]

In case it can be useful, the one below is the test structure that I use in my app (app name is "bgb").
I do not use/like doctests, hence only a unit and functional test classes are provided.
Do you think it would be acceptable for grokproject to generate something similar?

=== tests/config.py ===

import os

import z3c.testsetup
from zope.app.testing.functional import ZCMLLayer

import bgb

ftesting_zcml = os.path.join(os.path.dirname(bgb.__file__), 'ftesting.zcml')
FunctionalLayer = ZCMLLayer(ftesting_zcml, __name__, 'FunctionalLayer', allow_teardown=True)

test_suite = z3c.testsetup.register_all_tests('bgb')

=== tests/test_app.py ===

"""
Needed in order to make this executable.

:Test-Layer: python
"""

import unittest

from zope.publisher.interfaces import NotFound

# --- unittest

class TestUnit(unittest.TestCase):
    # put unit tests for your app here

    def test_some_test(self):
        self.assertEqual(1 + 1, 2)

# --- functional

from zope.app.testing.functional import FunctionalTestCase
from zope.testbrowser.testing import Browser

from bgb.tests.config import FunctionalLayer
from bgb.app import Bgb

class TestFunctional(FunctionalTestCase):
    # put functional tests for your app here

    layer = FunctionalLayer

    def setUp(self):
        # creates an app/site which can be navigated
        # http://grok.zope.org/doc/current/reference/testing.html#functional-test-layer
        FunctionalTestCase.setUp(self)
        self.browser = Browser()
        self.browser.handleErrors = False
        root = self.getRootFolder()
        root['bgb'] = Bgb()

    def test_home(self):
        self.browser.open('http://localhost/bgb')
        #self.assertEqual(self.browser.title, 'Welcome')

    # test other views here
    #def test_search(self):
    # self.browser.open('http://localhost/bgb/search')

[Uli Fouquet (uli-gnufix)]

Thanks everybody for the lots of comments!

After a quick glance my impression is that both, the reference documentation and the way tests are created in grokproject, are terribly outdated/broken.

For the documentation: grok does not provide any z3c.testsetup wrapper anymore. Therefore the mentioned docs shouldn't exist (or at least tell where to find up-to-date docs (which is for non-grok apps the z3c.testsetup PyPI page, BTW).

For grokproject and test setups created by it: there also have been many changes to the way most core developers do testing in grok environments, including the fact, that zope.app.testing is normally not used any more. Instead one should use zope.app.wsgi as described by Ben above (thanks, Ben!). This, I think, in turn is not mentioned/documented yet.

So both have to be fixed.

Concerning the 'plain' unit tests and a new directory layout for tests: yes, we should consider a complete new test layout including a doctest and a plain unittest, both maybe functional. But I'd rather like to stick with the convention used in most grokcore packages, which provide a filesystem layout similar to this:

sample/app.txt (functional doctest for app.py)
sample/testing.py (test setup, layer definitions and the like, and helper stuff)
sample/ftesting.zcml (a _working_ ZCML config, the current one seems to be outdated as well)
sample/tests/
sample/tests/test_app.py (functional unittests for app.py

There is a reason for placing general setup stuff in the main package itself and naming it 'testing.py' instead of tests/config.py:
Often packages need to do extended setup stuff that can be reused by other packages. For example they provide implementations of fake objects that speed-up tests and things like that. Putting that stuff in the 'main' package dir (and not in a tests subdirectory) we indicate that the stuff in there might be reusable for others.

That's what could be done shortly (and I'll try to do so).

In the long run we might consider to switch to another setup helper than z3c.testsetup. Maybe nose or something like this. This way we would be more compatible with other on-going developments in the Python world.

Please tell, what you think!

--
Uli

[Martijn Faassen (faassen)]

Note that I have code (that can I make public) that integrates grok with py.test. I'm using this in real world projects. This means that at least unit tests should become a lot easier to write, as the py.test test collector is quite good at finding tests. If people are interested in working on this, please drop me an email and I'll get code and more info to you.

[Jan-Jaap Driessen (janjaapdriessen)]

I discussed with Uli and have replaced z3c.testsetup with manual test setup in the trunk of grokproject. This hopefully leads to less confusion with new users while we work on zope.pytest in the mean time.

[Martijn Faassen (faassen)]

2011/1/3 Jan-Jaap Driessen <email address hidden>:
> I discussed with Uli and have replaced z3c.testsetup with manual test
> setup in the trunk of grokproject. This hopefully leads to less
> confusion with new users while we work on zope.pytest in the mean time.

Cool!

I just checked zope.pytest into svn.zope.org

[Jan Wijbrand Kolman (janwijbrand)]

For the record: we wil need upgrade documentation for people who want to upgrade from an earlier grok version to the upcoming grok version explaining how to register their tests.