Get/Set power state fails
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
python-dracclient |
Fix Released
|
Undecided
|
Christopher Dearborn |
Bug Description
The periodic job to retrieve node power state fails on specific nodes resulting in the nodes being moved into maintenance mode and an unknown power state.
or
A command to set the power state on specific nodes fails during the initial query to get the power state, resulting in the set power state command not being executed against those nodes and so the nodes fail to change power state.
Investigation:
Investigation has shown that it is possible for iDRACs to get into a state where the "Name" element is completely missing from the DCIM_ComputerSystem table. This results in the following query issued from get_power_state resulting in 0 records, and the resulting attempt to retrieve the power state fails and an exception is thrown:
select EnabledState from DCIM_ComputerSystem where Name="srv:system"
Resetting the iDRAC causes the "Name" element to be reported once again on a query of the DCIM_ComputerSystem table, which causes get_power_state to now succeed, where before it failed.
This has been seen on multiple servers that are running the most recent firmware (lifecycle controller: 2.41.40.40, bios: 2.5.4).
Note that the DCIM_ComputerSystem table appears to always contain only 1 record, making selecting on Name unnecessary. I tried removing the select on Name from the get_power_state select and it worked both on nodes that were in this state and on nodes that were not.
It should be noted that the command to set the power state uses "'Name': 'srv:system'" in the selector. Experimentation showed that this command works both on iDRACs that are in the state where no "Name" element is returned when querying DCIM_ComputerSy
I have contacted the iDRAC team and they agree that this is a firmware bug. It will probably be a considerable period of time before it is fixed in the firmware though.
Changed in python-dracclient: | |
assignee: | nobody → Christopher Dearborn (cdearbor) |
Fix proposed to branch: master /review. openstack. org/437132
Review: https:/