Comment 13 for bug 1516065

Revision history for this message
Keesjan Karsten (keesjank) wrote : Re: [1.9] Unable to power manage IPMI if BMC does not support changing boot order

The patch in comment #11 doesn't seem to work for me. I'm using MAAS Version 2.0.0+bzr5189-0ubuntu1 (16.04.1) which seems to have the patch applied.

I have a couple of ProLiant BL460c G6, iLO 2 Firmware Version: 2.29 07/16/2015, which won't commission. I get the error in /var/log/maas/rackd.log ("provisioningserver.drivers.power.PowerConnError: The IPMI session has timed out. MAAS performed several retries. Check BMC configuration and connectivity and try again.").

The /var/log/maas/maas.log part:
[...]
Oct 6 08:07:44 maas maas.node: [INFO] blade10: Status transition from FAILED_COMMISSIONING to COMMISSIONING
Oct 6 08:07:44 maas maas.power: [INFO] Changing power state (on) of node: blade10 (4y3h8c)
Oct 6 08:07:44 maas maas.node: [INFO] blade10: Commissioning started
Oct 6 08:08:04 maas maas.drivers.power.ipmi: [WARNING] Failed to change the boot order to PXE 10.10.1.20: ipmi_cmd_get_system_boot_options_boot_f
lags: session timeout
Oct 6 08:08:29 maas maas.drivers.power.ipmi: [WARNING] Failed to change the boot order to PXE 10.10.1.20: ipmi_cmd_get_system_boot_options_boot_f
lags: session timeout
Oct 6 08:08:57 maas maas.drivers.power.ipmi: [WARNING] Failed to change the boot order to PXE 10.10.1.20: ipmi_cmd_get_system_boot_options_boot_f
lags: session timeout
Oct 6 08:09:09 maas maas.power: [ERROR] Error changing power state (on) of node: blade10 (4y3h8c)
Oct 6 08:09:09 maas maas.node: [INFO] blade10: Status transition from COMMISSIONING to FAILED_COMMISSIONING
Oct 6 08:09:09 maas maas.node: [ERROR] blade10: Marking node failed: Node could not be powered on: Could not contact node's BMC: The IPMI session
 has timed out. MAAS performed several retries. Check BMC configuration and connectivity and try again.
[...]

When testing with ipmi-chassis-config, I get the same result as in comment #12 and debug output looking similar as in previous comments.

The G7's I have commission ok though (iLO 3, firmware version 1.85 May 13 2015).

The BMC in the G6 seems to be flakey and doesn't respond well to the request to change the boot order to boot from PXE.

Since I manually set the boot order to PXE boot in all machines via the iLO web administration, I figured I don't really need this ipmi-chassis-config command anyway. I commented it out from /usr/lib/python3/dist-packages/provisioningserver/drivers/power/ipmi.py (see below), restarted rackd ("sudo systemctl restart maas-rackd.service") and the commissioning works fine now for all machines.

ubuntu@maas:~$ sudo vi /usr/lib/python3/dist-packages/provisioningserver/drivers/power/ipmi.py
[...]
        # Update the chassis config and power commands.
        ipmi_chassis_config_command.extend(common_args)
        ipmi_chassis_config_command.append('--commit')
        ipmipower_command.extend(common_args)

        # Before changing state run the chassis config command.
# if power_change in ("on", "off"):
# self._issue_ipmi_chassis_config_command(
# ipmi_chassis_config_command, power_change, power_address)

        # Additional arguments for the power command.
        if power_change == 'on':
            ipmipower_command.append('--cycle')
            ipmipower_command.append('--on-if-off')
        elif power_change == 'off':
            if power_off_mode == 'soft':
                ipmipower_command.append('--soft')
            else:
                ipmipower_command.append('--off')
        elif power_change == 'query':
            ipmipower_command.append('--stat')
[...]
ubuntu@maas:~$

I like the idea of setting the boot device to PXE prior to powering on for devices that deal with it properly (i.e. G7), but I'm not sure how to make a workaround so the commission of the G6's works without the setting of the boot device on it. For some reason the patch doesn't work for me.