OC nodes stuck in BUILD state on Ironic timeouts

Bug #1417026 reported by Giulio Fidente
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ironic
Fix Released
Critical
Jim Rollenhagen
tripleo
Fix Released
Critical
Unassigned

Bug Description

I am seeing the OC nodes stuck in BUILD state with the following in nova-compute log:

Feb 02 01:28:09 host-192-168-1-30 nova-compute[4714]: 2015-02-02 01:28:09,282 ERROR Error deploying instance 257eed8c-3aa4-401d-b32c-c061648e2a1c on baremetal node d4719eeb-de3e-4cce-b28f-7b711ae11dca.

and this in ironic-conductor log:

Feb 02 01:28:08 host-192-168-1-30 ironic-conductor[4016]: 2015-02-02 01:28:08,286 ERROR Timeout reached while waiting for callback for node d4719eeb-de3e-4cce-b28f-7b711ae11dca

Interestingly this is happening for all the three OC nodes, a symptomatic job: https://review.openstack.org/#/c/137028/11

Tags: ci
Changed in tripleo:
status: New → Triaged
importance: Undecided → Medium
description: updated
Derek Higgins (derekh)
Changed in tripleo:
importance: Medium → Critical
Revision history for this message
Derek Higgins (derekh) wrote :

Reproducing this locally,

I'm seeing tftp usage problems, will attach a screen shot if I can, in the meantime I'm testing a revert to a ironic patch that recently touched the boot options

https://review.openstack.org/#/c/141148/6

Revision history for this message
Derek Higgins (derekh) wrote :
tags: added: ci
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to ironic (master)

Fix proposed to branch: master
Review: https://review.openstack.org/152129

Changed in ironic:
assignee: nobody → Jim Rollenhagen (jim-rollenhagen)
status: New → In Progress
Revision history for this message
Derek Higgins (derekh) wrote :

Just confirming a revert of https://review.openstack.org/#/c/141148/6 worked

both locally and in CI

Dmitry Tantsur (divius)
Changed in ironic:
importance: Undecided → Critical
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to ironic (master)

Reviewed: https://review.openstack.org/152129
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=16b8b5628991ca83964ea78e099dd1929df86189
Submitter: Jenkins
Branch: master

commit 16b8b5628991ca83964ea78e099dd1929df86189
Author: Jim Rollenhagen <email address hidden>
Date: Mon Feb 2 06:17:03 2015 -0800

    Revert "Do not pass PXE net config from bootloader to ramdisk"

    This reverts commit cb82dabdca5c37d9ac54e4c207a896a32f18525b.

    Change-Id: Ie623483d01db434241bfb00a84c5e6b7321e04d5
    Closes-Bug: #1417026

Changed in ironic:
status: In Progress → Fix Committed
Derek Higgins (derekh)
Changed in tripleo:
status: Triaged → Fix Released
Thierry Carrez (ttx)
Changed in ironic:
milestone: none → kilo-2
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in ironic:
milestone: kilo-2 → 2015.1.0
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.