MAAS deploy command hangs due to twisted errors

[Lee Trager]

While testing MAAS 2.9.0b1 I noticed that the MAAS deploy command will hang due to twisted errors. This isn't notifiable in the UI as the as the deployment does continue but if you use

maas $PROFILE machine deploy $SYSTEM_ID

it will hang for several minutes. This may be due to twisted errors seen in the MAAS logs

==> /var/log/maas/regiond.log <==
2020-09-09 00:31:00 maasserver.dhcp: [warn] No DNS servers found, DHCP defaulting to region IP.
2020-09-09 00:31:01 maasserver.dhcp: [info] Successfully configured DHCPv4 on rack controller 'maas-focal (bwpxqf)'.
2020-09-09 00:31:01 maasserver.dhcp: [info] Successfully configured DHCPv6 on rack controller 'maas-focal (bwpxqf)'.
2020-09-09 00:31:09 regiond: [info] 10.0.0.1 GET /MAAS/api/2.0/scripts/?include_script=true HTTP/1.1 --> 200 OK (referrer: http://10.0.0.2:5240/MAAS/r/machines; agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:80.0) Gecko/20100101 Firefox/80.0)
2020-09-09 00:31:12 regiond: [info] 10.0.0.1 GET /MAAS/api/2.0/account/?op=list_authorisation_tokens HTTP/1.1 --> 200 OK (referrer: http://10.0.0.2:5240/MAAS/l/machine/kgntx8; agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:80.0) Gecko/20100101 Firefox/80.0)
2020-09-09 00:31:18 regiond: [info] 127.0.0.1 GET /MAAS/rpc/ HTTP/1.1 --> 200 OK (referrer: -; agent: provisioningserver.rpc.clusterservice.ClusterClientService)
2020-09-09 00:31:21 maasserver.models.node: [critical] Performing IP address checking failed, will be retried.
 Traceback (most recent call last):
 --- <exception caught here> ---
   File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 654, in _runCallbacks
     current.result = callback(current.result, *args, **kw)
   File "/usr/lib/python3/dist-packages/twisted/protocols/amp.py", line 1994, in _massageError
     error.trap(RemoteAmpError)
   File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 439, in trap
     self.raiseException()
   File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 467, in raiseException
     raise self.value.with_traceback(self.tb)
 twisted.internet.defer.CancelledError:

2020-09-09 00:31:22 maasserver.dhcp: [warn] No DNS servers found, DHCP defaulting to region IP.
2020-09-09 00:31:22 maasserver.dhcp: [info] Successfully configured DHCPv4 on rack controller 'maas-focal (bwpxqf)'.
2020-09-09 00:31:22 maasserver.dhcp: [info] Successfully configured DHCPv6 on rack controller 'maas-focal (bwpxqf)'.

==> /var/log/maas/maas.log <==
2020-09-09T00:31:25.179069+00:00 maas-focal maas.power: [info] Changing power state (on) of node: maas-test-1 (dken44)

==> /var/log/maas/regiond.log <==
2020-09-09 00:31:25 twisted.web.wsgi: [critical] WSGI application error
 Traceback (most recent call last):
   File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 122, in callWithContext
     return self.currentContext().callWithContext(ctx, func, *args, **kw)
   File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 85, in callWithContext
     return func(*args,**kw)
   File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 886, in callInContext
     return func(*args, **kwargs)
   File "/usr/lib/python3/dist-packages/twisted/web/wsgi.py", line 533, in run
     self.started = True
 --- <exception caught here> ---
   File "/usr/lib/python3/dist-packages/twisted/web/wsgi.py", line 508, in run
     self.write(elem)
   File "/usr/lib/python3/dist-packages/twisted/web/wsgi.py", line 462, in write
     return blockingCallFromThread(
   File "/usr/lib/python3/dist-packages/twisted/internet/threads.py", line 122, in blockingCallFromThread
     result.raiseException()
   File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 467, in raiseException
     raise self.value.with_traceback(self.tb)
 builtins.AttributeError: 'NoneType' object has no attribute 'writeHeaders'

2020-09-09 00:31:25 asyncio: [error] Exception in callback <function AsyncioSelectorReactor.callLater.<locals>.run at 0x7fb7091aeca0>
handle: <Handle AsyncioSelectorReactor.callLater.<locals>.run>
Traceback (most recent call last):
  File "uvloop/cbhandles.pyx", line 68, in uvloop.loop.Handle._run
  File "/usr/lib/python3/dist-packages/twisted/internet/asyncioreactor.py", line 290, in run
    f(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/twisted/web/wsgi.py", line 521, in wsgiError
    self.request.loseConnection()
  File "/usr/lib/python3/dist-packages/twisted/web/http.py", line 1550, in loseConnection
    self.channel.loseConnection()
AttributeError: 'NoneType' object has no attribute 'loseConnection'
2020-09-09 00:31:25 maasserver.dhcp: [warn] No DNS servers found, DHCP defaulting to region IP.
2020-09-09 00:31:26 maasserver.dhcp: [info] Successfully configured DHCPv4 on rack controller 'maas-focal (bwpxqf)'.
2020-09-09 00:31:26 maasserver.dhcp: [info] Successfully configured DHCPv6 on rack controller 'maas-focal (bwpxqf)'.
2020-09-09 00:31:26 maasserver.region_controller: [info] Reloaded DNS configuration; ip 10.0.0.100 released