[2.1] Deferred being returned from thread

[Gavin Panella]

Seen in a log file somewhere:

  Unhandled Error
  Traceback (most recent call last):
    File "/usr/lib/python3/dist-packages/twisted/application/app.py",
    line 364, in startReactor
      self.config, oldstdout, oldstderr, self.profiler, reactor)
    File "/usr/lib/python3/dist-packages/twisted/application/app.py",
    line 285, in runReactorWithLogging
      reactor.run()
    File "/usr/lib/python3/dist-packages/twisted/internet/base.py", line
    1194, in run
      self.mainLoop()
    File "/usr/lib/python3/dist-packages/twisted/internet/base.py", line
    1203, in mainLoop
      self.runUntilCurrent()
  --- <exception caught here> ---
    File "/usr/lib/python3/dist-packages/twisted/internet/base.py", line
    798, in runUntilCurrent
      f(*a, **kw)
    File "/usr/lib/python3/dist-packages/twisted/internet/defer.py",
    line 392, in callback
      assert not isinstance(result, Deferred)
  builtins.AssertionError:

This:

    File "/usr/lib/python3/dist-packages/twisted/internet/base.py", line
    798, in runUntilCurrent
      f(*a, **kw)

is calling a function that's been put into the reactor via
callFromThread. I've inspected all uses of this in MAAS and they're
fine. However, this:

    File "/usr/lib/python3/dist-packages/twisted/internet/defer.py",
    line 392, in callback
      assert not isinstance(result, Deferred)

tells us that the `f` earlier is a bound Deferred.callback method.
Looking at Twisted's source code there is exactly one function that does
callFromThread(d.callback, ...): deferToThreadPool. There's only one use
of that in MAAS! Do we have a winner?

No. Unfortunately, Twisted's deferToThread calls deferToThreadPool, and
there are 32 uses of that in MAAS, so more investigation is necessary.

It seems very likely that somewhere in MAAS a Deferred is being returned
from a non-reactor thread. That's almost certainly broken code in its
own right, never mind that it's crashing on the way back into the
reactor.

The effect of this bug may be to permanently leave a Deferred unfired.
It can be reproduced with a simple script that will not exit until you
kill it:

  from twisted.internet.defer import Deferred
  from twisted.internet.task import react
  from twisted.internet.threads import deferToThread

  def run(reactor):
      return deferToThread(Deferred)

  react(run)

[Andres Rodriguez (andreserl)]

Dear user,

This is an automated message.

We believe this bug report is no longer an issue in the latest version of MAAS. For such reason, we are making this issue as Won't Fix. If you believe this issue is still present in the latest version of MAAS, please re-open this bug report.