tox runs single threaded and can't be overridden

Bug #1459852 reported by John Griffith
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Cinder
Fix Released
Undecided
John Griffith

Bug Description

I noticed that I couldn't get tox to run more than a single thread; turns out that we are setting "--no-parallel" in the pretty_tox.sh
file. Removing that makes the default run such that it consumes as many threads as you have available.

This is great and speeds things up significantly; one problem however is that once you get above around 6 parallel test processes things start to get a bit unstable. It's possibly an issue with some of our GIANT test classes that share some fakes/mocks but not sure. Could be we just can't run that many jobs in parallel.

In the meantime, I think lifting the single process limit is useful, and just like we did with run_tests you can specify concurrency if needed:
"tox -epy27 -- --concurrency=6"

NOTE: 6 seems to be the threshold for now.

Revision history for this message
John Griffith (john-griffith) wrote :

Note, bug regarding stability here: https://bugs.launchpad.net/cinder/+bug/1459854

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to cinder (master)

Fix proposed to branch: master
Review: https://review.openstack.org/186586

Changed in cinder:
assignee: nobody → John Griffith (john-griffith)
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on cinder (master)

Change abandoned by John Griffith (<email address hidden>) on branch: master
Review: https://review.openstack.org/186586
Reason: this will hopefully be made obsolete with the move to using os-testr and removing pretty_tox altogether.

https://review.openstack.org/#/c/186879/

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to cinder (master)

Reviewed: https://review.openstack.org/186879
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=68b4aad405c019d660b86f62617394e8b170c2ea
Submitter: Jenkins
Branch: master

commit 68b4aad405c019d660b86f62617394e8b170c2ea
Author: John Griffith <email address hidden>
Date: Fri May 29 13:02:13 2015 -0600

    Remove pretty_tox and use ostestr

    This patch dumps pretty_tox.sh and gets Cinder on board
    with using ostestr. We're setting a concurrency value
    of 6 (instances in the gate are 6 VCPU's), and things
    seem to be stable at this value.

    For folks running on 'smaller' system, it's still possible
    to run in parallell; > ncpu's but if there are problems you
    can still specify concurrency in your tox argument. Tox will
    take the last occurence of the conncurrency argument; so for
    example if you specify "tox -epy27 -- --concurrency=4" the
    interpretted value for concurrency in the run will be 4.

    Closes-Bug: #1459852

    Change-Id: Iae12b42e915da22c863781015512df4f30aa95ca

Changed in cinder:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in cinder:
milestone: none → liberty-1
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in cinder:
milestone: liberty-1 → 7.0.0
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.