Comment 6 for bug 1789426

I tried releasing 5 machines for which power management wasn't working (e.g. MAAS just couldn't contact the BMC's). 1 machine failed to release immediately, while the other 4 got stuck in 'Releasing':

=> /var/log/maas/maas.log <==
Aug 28 20:35:42 maas00 maas.node: [info] node01: Releasing node
Aug 28 20:35:42 maas00 maas.node: [info] node03: Releasing node
Aug 28 20:35:42 maas00 maas.node: [info] node01: Status transition from DEPLOYED to RELEASING
Aug 28 20:35:42 maas00 maas.node: [info] node03: Status transition from DEPLOYED to RELEASING
Aug 28 20:35:42 maas00 maas.node: [info] node02: Releasing node
Aug 28 20:35:42 maas00 maas.node: [info] node04: Releasing node
Aug 28 20:35:42 maas00 maas.node: [info] node02: Status transition from DEPLOYED to RELEASING
Aug 28 20:35:42 maas00 maas.node: [info] node04: Status transition from FAILED_RELEASING to RELEASING
Aug 28 20:35:43 maas00 maas.power: [info] Changing power state (off) of node: node03 (ncyhd6)
Aug 28 20:35:43 maas00 maas.power: [info] Changing power state (off) of node: node01 (8r76bg)
Aug 28 20:35:43 maas00 maas.power: [info] Changing power state (off) of node: node04 (mnwpg8)
Aug 28 20:35:43 maas00 maas.power: [info] Changing power state (off) of node: node02 (6y8g6m)
Aug 28 20:35:43 maas00 maas.node: [info] node05: Releasing node
Aug 28 20:35:43 maas00 maas.node: [info] node05: Status transition from FAILED_DEPLOYMENT to RELEASING
Aug 28 20:35:43 maas00 maas.power: [info] Changing power state (off) of node: node05 (d4wqsc)

==> /var/log/maas/regiond.log <==
2018-08-28 20:35:45 regiond: [info] 10.90.90.1 GET /MAAS/rpc/ HTTP/1.1 --> 200 OK (referrer: -; agent: provisioningserver.rpc.clusterservice.ClusterClientService)

==> /var/log/maas/maas.log <==
Aug 28 20:35:54 maas00 maas.power: [error] node01: Power state could not be queried: Could not connect to BMC. Check BMC configuration and try again.
Aug 28 20:35:54 maas00 maas.power: [error] node01: Could not query power state: Could not connect to BMC. Check BMC configuration and try again..

==> /var/log/maas/regiond.log <==
2018-08-28 20:36:15 regiond: [info] 10.90.90.1 GET /MAAS/rpc/ HTTP/1.1 --> 200 OK (referrer: -; agent: provisioningserver.rpc.clusterservice.ClusterClientService)

==> /var/log/maas/maas.log <==
Aug 28 20:36:36 maas00 maas.power: [error] Error changing power state (off) of node: node01 (8r76bg)
Aug 28 20:36:36 maas00 maas.node: [info] node01: Status transition from RELEASING to FAILED_RELEASING
Aug 28 20:36:36 maas00 maas.node: [error] node01: Marking node failed: Power off for the node failed: Could not contact node's BMC: Could not connect to BMC. Check BMC configuration and try again.

==> /var/log/maas/rackd.log <==
2018-08-28 20:36:36 provisioningserver.rpc.power: [critical] node01: Power off failed.
        Traceback (most recent call last):
          File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 459, in callback
            self._startRunCallbacks(result)
          File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 567, in _startRunCallbacks
            self._runCallbacks()
          File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 653, in _runCallbacks
            current.result = callback(current.result, *args, **kw)
          File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1442, in gotResult
            _inlineCallbacks(r, g, deferred)
        --- <exception caught here> ---
          File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1384, in _inlineCallbacks
            result = result.throwExceptionIntoGenerator(g)
          File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 408, in throwExceptionIntoGenerator
            return g.throw(self.type, self.value, self.tb)
          File "/usr/lib/python3/dist-packages/provisioningserver/rpc/power.py", line 290, in change_power_state
            system_id, hostname, power_type, power_change, context)
          File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1386, in _inlineCallbacks
            result = g.send(result)
          File "/usr/lib/python3/dist-packages/provisioningserver/drivers/power/__init__.py", line 422, in perform_power
            raise exc_info[0](exc_info[1]).with_traceback(exc_info[2])
          File "/usr/lib/python3/dist-packages/provisioningserver/drivers/power/__init__.py", line 379, in perform_power
            power_func, system_id, context)
          File "/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 250, in inContext
            result = inContext.theWork()
          File "/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 266, in <lambda>
            inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
          File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 122, in callWithContext
            return self.currentContext().callWithContext(ctx, func, *args, **kw)
          File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 85, in callWithContext
            return func(*args,**kw)
          File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 233, in wrapper
            result = func(*args, **kwargs)
          File "/usr/lib/python3/dist-packages/provisioningserver/drivers/power/amt.py", line 428, in power_off
            amt_command = self._get_amt_command(ip_address, power_pass)
          File "/usr/lib/python3/dist-packages/provisioningserver/drivers/power/amt.py", line 363, in _get_amt_command
            'exception')(error_info.get('message'))
        provisioningserver.drivers.power.PowerConnError: Could not connect to BMC. Check BMC configuration and try again.

==> /var/log/maas/regiond.log <==
2018-08-28 20:36:45 regiond: [info] 10.90.90.1 GET /MAAS/rpc/ HTTP/1.1 --> 200 OK (referrer: -; agent: provisioningserver.rpc.clusterservice.ClusterClientService)
2018-08-28 20:37:15 regiond: [info] 10.90.90.1 GET /MAAS/rpc/ HTTP/1.1 --> 200 OK (referrer: -; agent: provisioningserver.rpc.clusterservice.ClusterClientService)