[2.0b4] twisted.internet.error.ConnectionDone: Connection was closed cleanly.

[Andres Rodriguez]

1. updated the ephemeral image repo to use daily
2. maas webui lagged for a sec.
3. controller status showed yellow (warning)
4. saw this error.

I don't know whether this is reproduceable.

==> /var/log/maas/rackd.log <==
2016-05-12 08:00:22+0800 [ClusterClient,client] ClusterClient connection lost (HOST:IPv4Address(TCP, '192.168.20.104', 57846) PEER:IPv4Address(T
CP, '192.168.20.104', 5252))
2016-05-12 08:00:22+0800 [ClusterClient,client] Querying node power states.
        Traceback (most recent call last):
          File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 434, in errback
            self._startRunCallbacks(fail)
          File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 501, in _startRunCallbacks
            self._runCallbacks()
          File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 588, in _runCallbacks
            current.result = callback(current.result, *args, **kw)
          File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1184, in gotResult
            _inlineCallbacks(r, g, deferred)
        --- <exception caught here> ---
          File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1126, in _inlineCallbacks
            result = result.throwExceptionIntoGenerator(g)
          File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator
            return g.throw(self.type, self.value, self.tb)
          File "/usr/lib/python3/dist-packages/provisioningserver/pserv_services/node_power_monitor_service.py", line 64, in query_nodes
            ListNodePowerParameters, uuid=client.localIdent)
        twisted.internet.error.ConnectionDone: Connection was closed cleanly.

==> /var/log/maas/maas.log <==
May 12 08:00:22 trusty-maas9 maas.power_monitor_service: [ERROR] Failed to query nodes' power status: Connection was closed cleanly.

[Lee Trager (ltrager)]

I wasn't able to reproduce this. MAAS periodically checks the power status of all nodes a rack controller has access to. When you switched boot sources from releases to daily this triggered new images to be downloaded. This action may of starved some resource on the system enough that the RPC action timed out and the connection was closed.

[Gavin Panella (allenap)]

I think this happens when the rack loses its connection to the region while looping in NodePowerMonitorService.query_nodes(). It's not critical at all, but I'll improve the error message.