Comment 1 for bug 1171418

Revision history for this message
Raphaƫl Badin (rvb) wrote : Re: MAAS fails to power up machines when trying to install a node.

After investigating the issue, we found that the fix landed by https://code.launchpad.net/~vanhoof/maas/ipmi-state-fix_lp1086160/+merge/159714 (fix for bug 1086160) is responsible for the problem: the fix landed in this branch uncovered a bug in how MAAS deals with ipmi.

Before bug 1086160 was fixed, the ipmi template was *always* issuing the power command (because get_power_state() was broken). Now that we check the state of the node before powering it up, it the node is being brought down but is still up when get_power_state() is called, the ipmipower command won't be issued.

This is an example of what happens: right after "--off" is issued, the node is still up and thus "--stat" returns "on":
ubuntu@lenovo-RD230-01:~$ ipmipower -h 192.168.22.33 -u root -p ubuntu --off && ipmipower -h 192.168.22.33 -u root -p ubuntu --stat
192.168.22.33: ok <- this is the result of the "--off" command
192.168.22.33: on <- this is the result of the "--stat" command

ipmipower is clever enough to understand that, if "--on" is issued while the node is being powered down, the node needs to be powered up after it has gone down:
ubuntu@lenovo-RD230-01:~$ ipmipower -h 192.168.22.33 -u root -p ubuntu --off && ipmipower -h 192.168.22.33 -u root -p --on
=> the node is powered down *then up*.

In conclusion, we should probably revert to the old behavior and not check the return value of "--stat" at all, just issue the --on/--off command. (Note that MAAS executes these ipmi commands asynchronously [using celery] so we cannot use ipmipower's --wait-until-on/--wait-until-off commands to solve this problem).