Bifrost CI job is occasionally hitting the job timeout

Bug #1460052 reported by Julia Kreger
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Bifrost
Fix Released
Medium
Julia Kreger

Bug Description

The Bifrost CI job is occasionally hitting the default CI job timeout of thirty minutes.

Above and beyond extending the timeout which is a logical first step as performance can vary, we need to look at utilizing pre-caching and potentially look at leveraging a pre-built qcow2 file.

Additionally, the largest amount of time spent during a job is the time cloud-init is spending attempting to retrieve network information. This may be solved in other ways, but should be considered as it is ultimately a core reason the job takes such a long time.

Revision history for this message
Julia Kreger (juliaashleykreger) wrote :
Changed in bifrost:
status: Triaged → In Progress
assignee: nobody → Julia Kreger (juliaashleykreger)
Revision history for this message
Julia Kreger (juliaashleykreger) wrote :

The proposed fix, by changing the default guest OS to cirros, reduced the overall testing time by approximately 50% which has largely resolved this issue. We will continue to monitor and potentially re-open this ticket if we begin to encounter this issue again.

Changed in bifrost:
importance: High → Medium
status: In Progress → Fix Committed
Revision history for this message
Julia Kreger (juliaashleykreger) wrote :

The remaining step will be to extend the timeout, proposed https://review.openstack.org/191104 to extend the timeout to 60 minutes.

Changed in bifrost:
milestone: none → 0.0.1
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.