grokproject's test take a very long time

[Jan Wijbrand Kolman]

Running grokproject's test will download all grok's dependecy eggs at least two times. This can take a long time. Let's find a way to speeds this up.

[Uli Fouquet (uli-gnufix)]

I am to blame for the long test runs, so I will have to think about improvements.

[Uli Fouquet (uli-gnufix)]

The tests need too much time, that's unfortunately true.

As you already said, the long runs are mainly caused by downloading and installing all the eggs. The really time-consuming thing here is the installation (compiling and all that) of the eggs. Downloading the eggbasket takes some time too, but much less than installing everything. So I would propose to shorten the installation needs where appropriate.

Currently I see two approaches, that could improve the situation:

1) We setup some local fake-repository from where we get the eggbasket.

    The problem with this approach is, that it stops us from testing the real-world connectivity and the installation would
    still need much time. Furthermore we would have to provide ready to use eggbaskets with the distribution which would
    blow up the package enormously. Overall I would tend to skip this approach.

2) We introduce a constant in the test setup, which shortens some tests when set, but does the whole thing if set to
    False.

   Most time of the testruns is spent by filling the eggs directory. If you have already a local well-filled eggs-repository,
   then you could use this to shorten the test runtime dramatically. For me the difference is as follows:

         > 17 minutes when I start with an empty eggs directory

        < 1.5 minutes when I use a prepopulated eggs directory.

   I'd consider this an improvement.

   The shorter runtime can be archieved, if the removal/creation of the used eggs-directory is skipped depending
   on the setting of the constant mentioned above.

   We should make sure, however, that before commits or at least releases the whole testruns including, downloading
   all eggs, are executed. This is the only way to make sure, that tests do not fail because of strange packages contained
   in a local egg-repository (we already had such cases).

   The drawbacks would be:

      - the first test-run would last very long; only following test-runs would be faster.

      - the testers have to make sure, they run the complete suite once before releases. We could, however, remind them
        to do so by issuing a warning, when tests are shortened.

What do you think?

[Uli Fouquet (uli-gnufix)]

When creating a file 'shorttests' in the root of the checked out sources, egg directories are not cleared between tests and testruns. This makes testruns (except on first run) about 10 times faster.