DRAC driver fails to detect job completed with errors

Bug #1462513 reported by Christopher Dearborn
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ironic
Fix Released
Medium
Christopher Dearborn

Bug Description

The DRAC driver considers configuration jobs that have completed with errors as still running, and throws an exception when a new job is submitted.

To reproduce:
Submit a job to a drac that completes with errors. Submit another job. The server is powered off and deployment is marked as failed.

From ironic conductor log:
<ns4:DCIM_LifecycleJob>
<ns4:ElapsedTimeSinceCompletion>35799</ns4:ElapsedTimeSinceCompletion>
<ns4:InstanceID>JID_313750130793</ns4:InstanceID>
<ns4:JobStartTime>TIME_NOW</ns4:JobStartTime>
<ns4:JobStatus>Completed with Errors</ns4:JobStatus>
<ns4:JobUntilTime>TIME_NA</ns4:JobUntilTime>
<ns4:Message>Job completed with errors.</ns4:Message>
<ns4:MessageArguments>NA</ns4:MessageArguments>
<ns4:MessageID>PR31</ns4:MessageID>
<ns4:Name>ConfigBIOS:BIOS.Setup.1-1</ns4:Name>
<ns4:PercentComplete>100</ns4:PercentComplete>
</ns4:DCIM_LifecycleJob>

2015-06-05 14:22:37.360 26174 ERROR ironic.drivers.modules.pxe [-] Deploy failed for instance 15ef74a3-dbc9-4223-ad50-897ac95f8fac. Error: Another job with ID JID_313750130793 is already created to configure BIOS.Setup.1-1. Wait until existing job is completed or is canceled
2015-06-05 14:22:37.361 26174 DEBUG ironic.common.states [-] Exiting old state 'deploying' in response to event 'fail' on_exit /usr/lib/python2.7/site-packages/ironic/common/states.py:177
2015-06-05 14:22:37.361 26174 DEBUG ironic.common.states [-] Entering new state 'deploy failed' in response to event 'fail' on_enter /usr/lib/python2.7/site-packages/ironic/common/states.py:183

Tags: drac
Changed in ironic:
assignee: nobody → Christopher Dearborn (cdearbor)
Dmitry Tantsur (divius)
Changed in ironic:
status: New → Triaged
importance: Undecided → Medium
tags: added: drac
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to ironic (master)

Fix proposed to branch: master
Review: https://review.openstack.org/189409

Changed in ironic:
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to ironic (master)

Reviewed: https://review.openstack.org/189409
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=5704227cf0c1adf6c1d8a321b60eb6c5e8461ced
Submitter: Jenkins
Branch: master

commit 5704227cf0c1adf6c1d8a321b60eb6c5e8461ced
Author: Chris Dearborn <email address hidden>
Date: Mon Jun 8 15:17:46 2015 -0400

    Fix DRAC driver job completion detection

    The DRAC driver code determined job completion by comparing the
    returned JobStatus to "completed" and "failed". Testing with hardware
    showed that JobStatus can also return "Completed with Errors".

    This change updates the DRAC driver so that it recognizes a job with
    "Completed with Errors" for the JobStatus as a completed job.

    Change-Id: I051d6703194231712f0234f31b0b49208bf0e6ec
    Closes-Bug: 1462513

Changed in ironic:
status: In Progress → Fix Committed
Changed in ironic:
milestone: none → 4.0.0
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.