hard-coded 300s max_wait is not long enough, and is not configurable

Bug #1666746 reported by Ryan Beisner
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Mojo: Continuous Delivery for Juju
Fix Released
Critical
Paul Collins
OpenStack Charm Test Infra
Fix Released
Critical
Ryan Beisner

Bug Description

hard-coded 300s max_wait is not long enough, and is not configurable

UOSCI cannot use Mojo's built-in wait=True feature after rev 384 of lp:mojo and will need to revert to use of a separate phase which calls juju wait with the necessary cli parameters.

Mojo should always expose any waits or timeouts that are added as configurable options.

http://bazaar.launchpad.net/~mojo-maintainers/mojo/trunk/revision/384

Tags: uosci

Related branches

Revision history for this message
Ryan Beisner (1chb1n) wrote :

Flagging crit for charm-test-infra (OSCI) as we are experiencing widespread false failures of pre-release scenario testing with the current version of Mojo.

Changed in charm-test-infra:
status: New → Confirmed
importance: Undecided → Critical
assignee: nobody → Ryan Beisner (1chb1n)
Revision history for this message
Ryan Beisner (1chb1n) wrote :

2017-02-21 23:28:59 [INFO] Waiting for environment to reach steady state
2017-02-21 23:34:03 [ERROR] Not ready in 300s (max_wait)

Revision history for this message
Paul Collins (pjdc) wrote :

You can adjust this with max-wait=N on the deploy phase, although it looks like this is not documented. Please try it out and let us know if you encounter and problems. I'll work on documenting this.

Revision history for this message
Paul Collins (pjdc) wrote :

"on the deploy phase" <-- actually I mean the juju-check-wait phase there; and in fact it's not wired up to "deploy" at all. I'll fix this.

Tom Haddon (mthaddon)
Changed in mojo:
status: New → Confirmed
importance: Undecided → Critical
assignee: nobody → Paul Collins (pjdc)
Revision history for this message
Ryan Beisner (1chb1n) wrote :

Current trunk, L790, sets 300s:
            max_wait = 300 # 5 minutes

http://bazaar.launchpad.net/~mojo-maintainers/mojo/trunk/view/head:/mojo/phase.py#L790

Revision history for this message
Paul Collins (pjdc) wrote :

See linked MP for proposed fix.

Changed in mojo:
status: Confirmed → In Progress
Paul Collins (pjdc)
Changed in mojo:
status: In Progress → Fix Committed
Revision history for this message
Ryan Beisner (1chb1n) wrote :

Worked around in OSCI:

Rather than modify all of our manifests to hard-code a max_wait value in multiple places across all of the deploy phases, we've opted to declare wait=False (not use Mojo's built-in wait logic) and continue to use our existing approach.

We continue to use our classic approach, which is a separate phase which calls juju-wait after the deploy phase.

As an aside, we now consume Mojo via a python venv, which allows us to optionally pin or track trunk or branches of lp:mojo, depending on the need at the time.

Changed in charm-test-infra:
status: Confirmed → Fix Committed
James Page (james-page)
Changed in charm-test-infra:
milestone: none → 17.02
James Page (james-page)
Changed in charm-test-infra:
status: Fix Committed → Fix Released
Paul Collins (pjdc)
Changed in mojo:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.