Spurious failures in test_regionservice

[Gavin Panella]

Asynchronous tests in maasserver.rpc.tests.test_regionservice spuriously
fail, often (always?) with log messages from other tests, i.e. there is
an isolation issue. I suspect this has been brought out into the light
since we started using the new twisted.logger package, though I have not
been able to identify a root cause.

Isolation between asynchronous tests has been a bit of a lingering
problem in the region tests. In the rack tests we use testtools'
AsynchronousDeferredRunTest, by way of MAASTwistedRunTest. This starts
and stops the Twisted reactor, checks for junk, and cleans up between
tests, but in the region that hasn't been possible because of Django's
TransactionTestCase and TestCase. These make too many assumptions to be
used alongside testtools' asynchronous test support.

We need to run the region's asynchronous tests with testtools' help, or
with Twisted's test runner, trial, ideally the former... or either.

In the rack we run tests with only a kimono-thin wrapper around
nosetests. We can, right now, run nearly all the rack tests with trial
(there's a bug preventing a completely clean run) for example, because
of the composition of tests.

In the region, however, we must invoke Django's test command, which then
loads the maas.development settings in which the django_nose app is
installed and TEST_RUNNER is set to something custom; the net result is
that the test command eventually invokes nosetests. In other words,
set-up and tear-down is a distinctly "as grafted-on by Victor F." deal.

The region can no longer be called just a "Django application". We do
use Django's ORM widely, forms too, and its request/response machinery
is used in the Web API (though we hope to remove that in the fullness of
time). As such, Django's assumption of sovereignty is a throwback.

Recasting Django's test database set-up and tear-down as a test fixture
and/or test resource would allow us to run with much better support for
asynchronous tests, and allow us to bypass Django's smothering and
clunky command-line tools.

It ought to allow us to also speed up the test suite because we would
have finer control over what exactly to set-up or tear-down, and because
testresources (the library) can optimise the order in which tests are
run to minimise overall set-up/tear-down costs.