[2.5] Updating the Power IP of a (AMT) machine updates all other machines

[Andres Rodriguez]

I changed the BMC IP of the machine while it was 'Deploying'. After the change, I was allowed to release the machine even with the changed IP.

The error seen in the UI is:

Error: Unable to retrieve AMT version: Connection failed. response code = 0 Couldn't connect to server

On the machine details page, power configuration section.

2018-09-19 15:35:46 provisioningserver.rpc.common: [debug] [RPC <- recieved] AmpBox({b'_answer': b'83'})
2018-09-19 15:35:46 provisioningserver.rpc.common: [debug] [RPC -> sent] AmpBox({b'system_id': b'eynamb', b'type_name': b'NODE_POWER_OFF_FAILED', b'description': b"Power off for the node failed: Could not contact node's BMC: Unable to retrieve AMT version: Connection failed. response code = 0\nCouldn't connect to server\n", b'_command': b'SendEvent'})
2018-09-19 15:35:46 provisioningserver.rpc.common: [debug] [RPC <- recieved] AmpBox({b'_answer': b'7b'})
2018-09-19 15:35:46 provisioningserver.rpc.power: [critical] node04: Power off failed.
 Traceback (most recent call last):
   File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 459, in callback
     self._startRunCallbacks(result)
   File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 567, in _startRunCallbacks
     self._runCallbacks()
   File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 653, in _runCallbacks
     current.result = callback(current.result, *args, **kw)
   File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1442, in gotResult
     _inlineCallbacks(r, g, deferred)
 --- <exception caught here> ---
   File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1384, in _inlineCallbacks
     result = result.throwExceptionIntoGenerator(g)
   File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 408, in throwExceptionIntoGenerator
     return g.throw(self.type, self.value, self.tb)
   File "/usr/lib/python3/dist-packages/provisioningserver/rpc/power.py", line 290, in change_power_state
     system_id, hostname, power_type, power_change, context)
   File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1386, in _inlineCallbacks
     result = g.send(result)
   File "/usr/lib/python3/dist-packages/provisioningserver/drivers/power/__init__.py", line 422, in perform_power
     raise exc_info[0](exc_info[1]).with_traceback(exc_info[2])
   File "/usr/lib/python3/dist-packages/provisioningserver/drivers/power/__init__.py", line 379, in perform_power
     power_func, system_id, context)
   File "/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 250, in inContext
     result = inContext.theWork()
   File "/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 266, in <lambda>
     inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
   File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 122, in callWithContext
     return self.currentContext().callWithContext(ctx, func, *args, **kw)
   File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 85, in callWithContext
     return func(*args,**kw)
   File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 233, in wrapper
     result = func(*args, **kwargs)
   File "/usr/lib/python3/dist-packages/provisioningserver/drivers/power/amt.py", line 430, in power_off
     amt_command = self._get_amt_command(ip_address, power_pass)
   File "/usr/lib/python3/dist-packages/provisioningserver/drivers/power/amt.py", line 367, in _get_amt_command
     "Unable to retrieve AMT version: %s" % stderr)
 provisioningserver.drivers.power.PowerConnError: Unable to retrieve AMT version: Connection failed. response code = 0
 Couldn't connect to server

[Andres Rodriguez (andreserl)]

It seems this is due to AMT power driver, since it parses errors. The error we would need to parse would be:

ubuntu@maas00:/usr/lib/python3/dist-packages/provisioningserver/drivers/power$ wsman identify --port 16992 --hostname 10.90.90.14 --username admin --password Password1+
Connection failed. response code = 0
Couldn't connect to server

[Andres Rodriguez (andreserl)]

After a bit of investigation, it seems that the error I was seeing was because I was updating the IP of an AMT machine, and the UI would update the IP for *all* machines.

[Andres Rodriguez (andreserl)]

Re-opening this as the migration added fails to upgrade:

https://pastebin.ubuntu.com/p/WdxPZr9kjJ/