Failed to change the boot order to PXE - session timeout

Bug #1704243 reported by Kayode Odeyemi on 2017-07-14
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
MAAS
Undecided
Unassigned

Bug Description

I get this in the logs while deploying a MaaS Node

Jul 14 01:59:26 maas-dev maas.node: [info] boss-kodiak: Status transition from READY to ALLOCATED
Jul 14 01:59:26 maas-dev maas.node: [info] boss-kodiak: allocated to user admin
Jul 14 01:59:27 maas-dev maas.interface: [info] Allocated automatic IP address 192.168.12.6 for eth1 (physical) on boss-kodiak.
Jul 14 01:59:27 maas-dev maas.node: [info] boss-kodiak: Status transition from ALLOCATED to DEPLOYING
Jul 14 01:59:27 maas-dev maas.power: [info] Changing power state (on) of node: boss-kodiak (sdamye)
Jul 14 02:00:07 maas-dev maas.drivers.power.ipmi: [warn] Failed to change the boot order to PXE 192.168.13.132: ipmi_cmd_set_system_boot_options_boot_flags: session timeout
Jul 14 02:00:38 maas-dev maas.drivers.power.ipmi: [warn] Failed to change the boot order to PXE 192.168.13.132: ipmi_cmd_get_system_boot_options_boot_flags: session timeout
Jul 14 02:00:50 maas-dev maas.power: [error] Error changing power state (on) of node: boss-kodiak (sdamye)
Jul 14 02:00:50 maas-dev maas.node: [info] boss-kodiak: Status transition from DEPLOYING to FAILED_DEPLOYMENT
Jul 14 02:00:50 maas-dev maas.node: [error] boss-kodiak: Marking node failed: Power on for the node failed: Failed talking to node's BMC: Failed to power sdamye. BMC never transitioned from off to on.
Jul 14 02:00:51 maas-dev maas.node: [info] boss-kodiak: Releasing node
Jul 14 02:00:51 maas-dev maas.node: [info] boss-kodiak: Status transition from FAILED_DEPLOYMENT to RELEASING
Jul 14 02:00:51 maas-dev maas.power: [info] Changing power state (off) of node: boss-kodiak (sdamye)
Jul 14 02:01:11 maas-dev maas.drivers.power.ipmi: [warn] Failed to change the boot order to PXE 192.168.13.132: ipmi_cmd_set_system_boot_options_boot_flags: session timeout
Jul 14 02:01:27 maas-dev maas.node: [info] boss-kodiak: Status transition from RELEASING to READY
Jul 14 02:01:27 maas-dev maas.power: [info] Changed power state (off) of node: boss-kodiak (sdamye)

So literally, it moves from READY -> ALLOCATED -> DEPLOYING -> FAILED DEPLOYMENT -> RELEASING -> READY. I think the catch phrase is "Failed to change the boot order to PXE 192.168.13.132: ipmi_cmd_get_system_boot_options_boot_flags: session timeout".

We're running maas 2.1.5. To replicate, we launched a "node deploy" using "conjure-up --boostrap-to boss-kodiak".

Any ideas please?

Kayode Odeyemi (dreyemi) on 2017-07-14
summary: - Failed to change the boot order to PXE
+ Failed to change the boot order to PXE - session timeout
description: updated
Andres Rodriguez (andreserl) wrote :

Failed to changing boot order is a non-failure status. Meaning if you see that, MAAS failed to change the boot order, but will continue to try to power on the machine.

When this is failing is when MAAS is trying to power on the machine. That would typically happen because:

1. Buggy BMC.
2. Issues when contacting the BMC.

That said, the fact that it fails to change to boot order is an indifcation that soething is wrong. So:

1. Have you tried to deploy this machine manually?
2. If you do: ipmipower -D LAN_2_0 -h <ip> -u <maas ipmi user> -p <maas ipmi password> --cycle --on-if-off, does that work ?

Changed in maas:
status: New → Incomplete
milestone: none → 2.3.0
Andres Rodriguez (andreserl) wrote :

Also, you may wanna try https://code.launchpad.net/~andreserl/maas/+git/maas/+merge/327339 , which may address your issue.

Kayode Odeyemi (dreyemi) wrote :

Thanks. But sorry. Noob here. How do I get the IPMI power password?

Kayode Odeyemi (dreyemi) wrote :

Patch successful.

"boss-kodiak: Status transition from DEPLOYING to DEPLOYED"

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers