reboot timeout way too short in fuel 11 stable

Bug #1671793 reported by Luca Cervigni on 2017-03-10
38
This bug affects 6 people
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
High
Stanislaw Bogatkin
Nominated for Ocata by Michael Polenchuk
Newton
Critical
Michael Polenchuk

Bug Description

During the provisioning of servers, the timeout of node_reboot is set to 3 minutes.
This is way too short for bare metal servers. In our environment with UEFI enabled our IBM servers take at least 10 minutes for the fastest, up to 15 minutes for the slowest.

The timeout should be increased to at least 15 minutes from 3 or the provisioning fails with:

All nodes are finished. Failed tasks: Task[node_reboot/10], Task[node_reboot/26], Task[node_reboot/32], Task[node_reboot/34] Stopping the deployment process!

Expected result:
provisioning success

Actual result:
provisioning failed.

Impact: not able to provision servers (very high)

Changed in fuel:
milestone: none → 11.0
assignee: nobody → Stanislaw Bogatkin (sbogatkin)
importance: Undecided → High
status: New → Confirmed

Fix proposed to branch: master
Review: https://review.openstack.org/444826

Changed in fuel:
status: Confirmed → In Progress
Sergey Yezhkov (yezhkov) wrote :

Hello, I have the same problem.
I already try to change timeout in graphs/provision/tasks.yaml, but I can`t find how to sync this changes.
For plugins I usualy use "fuel2 plugins sync". But it not work for graph/provision.

Sergey Yezhkov (yezhkov) wrote :

I found how to sync changes:
fuel2 graph upload -r {releaseID} -t provision -f /etc/fuel/graphs/provision/tasks.yaml
or, for enviroment:
fuel2 graph upload -e {envID} -t provision -f /etc/fuel/graphs/provision/tasks.yaml

Reviewed: https://review.openstack.org/444826
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=12e9b93a9d123f56c1500714fd97bdd35cf4bca0
Submitter: Jenkins
Branch: master

commit 12e9b93a9d123f56c1500714fd97bdd35cf4bca0
Author: Stanislaw Bogatkin <email address hidden>
Date: Mon Mar 13 14:13:34 2017 +0300

    Increase reboot task timeout

    To avoid broken provision on baremetal servers, increase reboot
    task timeout to 15 minutes.

    Change-Id: Ia7b9aa96407dae5819420146f5e1e934031b662c
    Closes-Bug: #1671793

Changed in fuel:
status: In Progress → Fix Committed

Reviewed: https://review.openstack.org/445962
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=74acfc23f3bd69f93903fe257eb6c43bcbc2167d
Submitter: Jenkins
Branch: stable/newton

commit 74acfc23f3bd69f93903fe257eb6c43bcbc2167d
Author: Stanislaw Bogatkin <email address hidden>
Date: Mon Mar 13 14:13:34 2017 +0300

    Increase reboot task timeout

    To avoid broken provision on baremetal servers, increase reboot
    task timeout to 15 minutes.

    Change-Id: Ia7b9aa96407dae5819420146f5e1e934031b662c
    Closes-Bug: #1671793

Changed in fuel:
milestone: 11.0 → 12.0
Kyrylo Romanenko (kromanenko) wrote :

It looks that this bug also affects 10.0 community build BVT:
https://ci.fuel-infra.org/job/10.0-community.main.ubuntu.bvt_2/1337/

Reviewed: https://review.openstack.org/446357
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=f4eda96d45469a42b3cc2f69466ef24eae0f4008
Submitter: Jenkins
Branch: stable/ocata

commit f4eda96d45469a42b3cc2f69466ef24eae0f4008
Author: Stanislaw Bogatkin <email address hidden>
Date: Mon Mar 13 14:13:34 2017 +0300

    Increase reboot task timeout

    To avoid broken provision on baremetal servers, increase reboot
    task timeout to 15 minutes.

    Change-Id: Ia7b9aa96407dae5819420146f5e1e934031b662c
    Closes-Bug: #1671793

tags: added: in-stable-ocata
Kyrylo Romanenko (kromanenko) wrote :

Please check this, it looks it is also subject for this bug: https://ci.fuel-infra.org/job/11.0-community.main.ubuntu.bvt_2/580/

Ekaterina Shutova (eshutova) wrote :

Verified for 10.0 #1512.
Latest runs are passed, problem is fixed.
https://ci.fuel-infra.org/job/10.0-community.main.ubuntu.bvt_2/1398/

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers