bay create is slow in gates

Bug #1536739 reported by Dimitry Ushakov
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Magnum
In Progress
Medium
Corey O'Brien

Bug Description

Bay create in devstack gate takes about 25 minutes. More data can be found in this gist: https://gist.github.com/dimtruck/1d59b7b8d2869190ccba#file-output-sh (and surrounding log files)

High level break down: heat workflow takes about 25 minutes. Most of the time is spent in swarm_masters and swarm_nodes.

In swarm_masters and swarm_nodes:
~5 minutes in resource creation (nova, neutron, cinder)
~5 minutes in cloud-init (both cloud-init modules and heat fragments we run as part of cloud-init)
~3 minutes unaccounted for

The challenge is that due to this slowness, our test suite is limited since we hit the time constraint of 2 hours after only 2 bay-creates (accounting for devstack spin-up time)

One of the ways to expedite this is to remove the dependency for swarm_masters for swarm_nodes to make these resources execute in parallel.

Dimitry Ushakov (dimalg)
description: updated
Changed in magnum:
assignee: nobody → Corey O'Brien (coreypobrien)
status: New → In Progress
hongbin (hongbin034)
Changed in magnum:
importance: Undecided → Medium
Changed in magnum:
assignee: Corey O'Brien (coreypobrien) → Jay Lau (jay-lau-513)
Changed in magnum:
assignee: Jay Lau (jay-lau-513) → Dimitry Ushakov (dimalg)
Changed in magnum:
assignee: Dimitry Ushakov (dimalg) → Corey O'Brien (coreypobrien)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to magnum (master)

Reviewed: https://review.openstack.org/275405
Committed: https://git.openstack.org/cgit/openstack/magnum/commit/?id=08ac9a6cbac5b324747c178c4cab36296c16b567
Submitter: Jenkins
Branch: master

commit 08ac9a6cbac5b324747c178c4cab36296c16b567
Author: Corey O'Brien <email address hidden>
Date: Tue Feb 2 15:57:40 2016 -0500

    Remove minion dependency on master

    Make thhe minions work correctly without depending on the master.
    Removing this dependency decreases provisioning time by provisioning
    nodes in parallel.

    Change-Id: I025dbf54ef0cdafc4fe26acc33a9faed8cacd1ca
    Partial-Bug: 1536739
    Partial-Bug: 1551824

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.