stop running exercise.sh in devstack-gate

[James E. Blair]

Now that we're running tempest smoke tests, we should be able to stop running exercise.sh, which should shorten run-time, and reduce a potential source of non-determinism (not to mention, actually achieve our goal of centralizing integration testing into one project -- tempest).

Need to do a gap analysis and see if devstack-gate is testing anything via exercise.sh that tempest is not.

Note that devstack-gate does not enable all projects, and skips some tests:

https://github.com/openstack-infra/devstack-gate/blob/master/devstack-vm-gate.sh#L47

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/42082

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/42082
Committed: http://github.com/openstack-infra/devstack-gate/commit/f70b9d6aa7584e75fd975c0f382713a7084eb062
Submitter: Jenkins
Branch: master

commit f70b9d6aa7584e75fd975c0f382713a7084eb062
Author: Attila Fazekas <email address hidden>
Date: Thu Aug 15 10:47:37 2013 +0200

    Skip devstack/exercises by default

    In order to avoid maintaining test cases in two location,
    the devstack/exercises should be disabled when tempest is enabled.

    The exercises is preserved for grande and stable branch jobs.

    A smaller/faster 'smoke' test case set can be selected from the tempest
    test cases if it is needed.

    Fixing bug #1023131

    Change-Id: Icac3998fb2c18274d909afcff8708af149d68197

[Matthew Treinish (treinish)]

We stopped running devstack exercises in devstack-gate by default on master. However the smoke test attr doesn't have the same coverage as the devstack exercises. (see the aggregates tests for an example) So the tempest part of this bug should remain open until everything that was tested with devstack exercises is tagged as smoke in tempest too.

[Matthew Treinish (treinish)]

The tempest scenario tests are still missing equivalent tests for the the aggregates and swift from the devstack exercises. Once that is finished the tempest portion can be closed.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/62618

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/61845
Committed: https://git.openstack.org/cgit/openstack/tempest/commit/?id=e86ed04965290b175d46bedacedf5d92c1f58704
Submitter: Jenkins
Branch: master

commit e86ed04965290b175d46bedacedf5d92c1f58704
Author: Mauro S. M. Rodrigues <email address hidden>
Date: Thu Dec 12 18:56:00 2013 +0000

    Add swift scenario tests

    This tests swift as exercises do, trying common operations as:
     - create/delete a container;
     - add/delete object to a container;
     - list objects in the container;

    Partial-Bug: #1023131
    Change-Id: Id4b27ee70f420f7536b4c698eb1f3bba15dbd1e8

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/62618
Committed: https://git.openstack.org/cgit/openstack/tempest/commit/?id=92ed9b8db905fddd6ce54ae87a2c54f2c8618bf3
Submitter: Jenkins
Branch: master

commit 92ed9b8db905fddd6ce54ae87a2c54f2c8618bf3
Author: Mauro S. M. Rodrigues <email address hidden>
Date: Tue Dec 17 07:21:48 2013 -0500

    Add aggregates scenario test

    The basic scenario for aggregates consists in the follow steps:
     - Creates an aggregate within an availability zone
     - Adds a host to the aggregate
     - Checks aggregate details
     - Updates aggregate's name
     - Removes host from aggregate
     - Deletes aggregate

    Change-Id: I7a11b11468a5bbe85da4f6154308f7a5336c6919
    Partial-bug: #1023131

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/havana
Review: https://review.openstack.org/71951

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/71009
Committed: https://git.openstack.org/cgit/openstack-dev/grenade/commit/?id=a6c7d4b4dbd98eeba4d3c33f51d893b7cac021d8
Submitter: Jenkins
Branch: master

commit a6c7d4b4dbd98eeba4d3c33f51d893b7cac021d8
Author: Mauro S. M. Rodrigues <email address hidden>
Date: Tue Feb 4 15:26:19 2014 +0000

    Remove exercises and run tempest smoke tests instead

    Smoke and Scenarios tests can offer a better coverage and also it's good to
    centralize all tests in tempest.

    Change-Id: Iacb60ab011d2c7d90885b339b3c8454bd3de8219
    Partial-bug: #1023131

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/71010
Committed: https://git.openstack.org/cgit/openstack-infra/devstack-gate/commit/?id=dc2d737c72881af5cf34b9ac366f22025236e618
Submitter: Jenkins
Branch: master

commit dc2d737c72881af5cf34b9ac366f22025236e618
Author: Mauro S. M. Rodrigues <email address hidden>
Date: Tue Feb 4 12:24:41 2014 +0000

    Remove exercises tests from grenade runs

    For now on we're going to use smoke + scenarios tests.

    Only devstack still uses exercises, so many references to it, and to exercises
    to be skipped were kept.
    Depends on: Iacb60ab011d2c7d90885b339b3c8454bd3de8219

    Change-Id: I76305fae46a662177c6edcc9eca237c1d64be137
    Closes-bug: #1023131

[Alan Pevec (apevec)]

Please backport to stable!

[Openstack Gerrit (openstack-gerrit)]

Reviewed: https://review.openstack.org/71951
Committed: https://git.openstack.org/cgit/openstack/tempest/commit/?id=b57373012e8fe9ea8ce443d541f94fea1374345e
Submitter: Jenkins
Branch: stable/havana

commit b57373012e8fe9ea8ce443d541f94fea1374345e
Author: Mauro S. M. Rodrigues <email address hidden>
Date: Thu Dec 12 18:56:00 2013 +0000

    Add swift scenario tests

    This tests swift as exercises do, trying common operations as:
     - create/delete a container;
     - add/delete object to a container;
     - list objects in the container;

    Partial-Bug: #1023131
    Cherry-picked from Id4b27ee70f420f7536b4c698eb1f3bba15dbd1e8

    Change-Id: I6b95651684644344cd688d1ca1d6b3bd6bc70f01