Improve tempest tests coverage

[Antoine Eiche]

Currently, there are two tempest tests which are
- test_create_bgpvpn
- test_create_bgpvpn_as_non_admin_fail

We propose to add the following tempest tests:
- delete_bgpvpn_as_non_admin_fail

- associate_virtual_network_to_bgpvpn
- disassociate_virtual_network_to_bgpvpn
- associate_multiple_virtual_network_to_bgpvpn

- upgrade_bgpvpn_route_target
- upgrade_bgpvpn_route_target_as_non_admin_fail

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/348771

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/348913

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/351791

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/348771
Committed: https://git.openstack.org/cgit/openstack/networking-bgpvpn/commit/?id=c35805421fe51181ea6f47af61c9e82484b4b19b
Submitter: Jenkins
Branch: master

commit c35805421fe51181ea6f47af61c9e82484b4b19b
Author: Antoine Eiche <email address hidden>
Date: Fri Jul 29 09:57:25 2016 +0200

    Add test delete_bgpvpn_as_non_admin_fail

    The admin creates a bgpvpn resource by specifying a normal user tenant
    id. Test checks this user fails to delete the bgpvpn resource.

    Change-Id: I2d3f91b9d7ab86187e842f6dbecfa51ef9afbe8e
    Partial-Bug: #1607664

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/348913
Committed: https://git.openstack.org/cgit/openstack/networking-bgpvpn/commit/?id=e2f276a02fffd116e3019f89321f435fb106aabe
Submitter: Jenkins
Branch: master

commit e2f276a02fffd116e3019f89321f435fb106aabe
Author: Antoine Eiche <email address hidden>
Date: Fri Jul 29 16:54:35 2016 +0200

    Add tempest test associate_disassociate_network

    Network assocation and disassociation client functions have also been
    added.

    Change-Id: I5494c3f7c85105bc22e508e09d2c32a09d664477
    Partial-Bug: #1607664

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/353272

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/351791
Committed: https://git.openstack.org/cgit/openstack/networking-bgpvpn/commit/?id=28dcfbecc28d7f6b6c4c8236ec87ee7a1b7546f2
Submitter: Jenkins
Branch: master

commit 28dcfbecc28d7f6b6c4c8236ec87ee7a1b7546f2
Author: Jean-Philippe Braun <email address hidden>
Date: Fri Aug 5 17:13:29 2016 +0200

    Add tempest tests on route-target update

    Change-Id: Ibd2f50797b4d6c6b34cda6bf3f219644af0ffe43
    Partial-Bug: #1607664

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/353272
Committed: https://git.openstack.org/cgit/openstack/networking-bgpvpn/commit/?id=c1ab12636c6d98a5a94bd18fc4b08735dea834ab
Submitter: Jenkins
Branch: master

commit c1ab12636c6d98a5a94bd18fc4b08735dea834ab
Author: suresh kumar <email address hidden>
Date: Wed Aug 10 08:25:51 2016 +0200

    Added the negative cases

    Test checks that invalid route targets are rejected by the create and update api.
    Test checks that invalid network,bgpvpn id are rejected by the associate and disassociate api.

    Change-Id: I181a1876f654ce5821c6bc212696d7be2801acde
    Partial-Bug: 1607664

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/359227

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/359227
Committed: https://git.openstack.org/cgit/openstack/networking-bgpvpn/commit/?id=304a7da868fced1b252a16dd4e381b3173b704fa
Submitter: Jenkins
Branch: master

commit 304a7da868fced1b252a16dd4e381b3173b704fa
Author: bfernando <email address hidden>
Date: Tue Aug 23 16:08:50 2016 +0200

    Add a tempest test on read permission with bgpvpn_list

    * Configure the second openstack user by providing the "alt"
      client_manager.
    * Add the test function which checks that a user cannot read
      bgpvpn resources that it does not own.
    * Add some comments in the BgpvpnTest description.

    Change-Id: Ia1edea9f414384a59b0568d0d733e3413944bd05
    Partial-Bug: #1607664

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/366229

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/366229
Committed: https://git.openstack.org/cgit/openstack/networking-bgpvpn/commit/?id=b5279bf723d5cb32e6f7c363b7dfa727003944c2
Submitter: Jenkins
Branch: master

commit b5279bf723d5cb32e6f7c363b7dfa727003944c2
Author: bfernando <email address hidden>
Date: Tue Sep 6 17:00:10 2016 +0200

    Add more tempest tests for read permissions

    * Improve the set of tests to be more exhaustive. The goal of these tests is
      to check that the reading restrictions between user of different tenants
      are preserved.
    * The tests added are:
      * show bgpvpn object made by a non owner fail.
      * show network association made by a non owner fail.
      * list network associations made by a non owner fail.
    * Add new methods to the BgpvpnClient (list, show).
    * Rename some methods/calls to be more standard.

    Change-Id: I7a3d773b30aa4ceaf13022f200f6fc6a49934ec2
    Partial-Bug: #1607664

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/367904

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/367904
Committed: https://git.openstack.org/cgit/openstack/networking-bgpvpn/commit/?id=396aee4d0ab65f786db93c22fd3f1b9d2fd6f887
Submitter: Jenkins
Branch: master

commit 396aee4d0ab65f786db93c22fd3f1b9d2fd6f887
Author: Mathieu Rohon <email address hidden>
Date: Fri Sep 9 09:15:14 2016 +0000

    Add tempest tests for router association

    Change-Id: I3dbd54364165b489871214b308a18c7b4a71f91d
    Partial-Bug: #1607664

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/381106

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/381107

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/381109

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/381110

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/381111

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/381112

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/381113

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/381106
Committed: https://git.openstack.org/cgit/openstack/networking-bgpvpn/commit/?id=3e7f1fe2acdea3f0657bbe24ebd20109244d381e
Submitter: Jenkins
Branch: stable/mitaka

commit 3e7f1fe2acdea3f0657bbe24ebd20109244d381e
Author: Antoine Eiche <email address hidden>
Date: Fri Jul 29 09:57:25 2016 +0200

    Add test delete_bgpvpn_as_non_admin_fail

    The admin creates a bgpvpn resource by specifying a normal user tenant
    id. Test checks this user fails to delete the bgpvpn resource.

    Change-Id: I2d3f91b9d7ab86187e842f6dbecfa51ef9afbe8e
    Partial-Bug: #1607664
    (cherry picked from commit c35805421fe51181ea6f47af61c9e82484b4b19b)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/381107
Committed: https://git.openstack.org/cgit/openstack/networking-bgpvpn/commit/?id=f6841b928938227ef429b9ccbdfc14f95514858a
Submitter: Jenkins
Branch: stable/mitaka

commit f6841b928938227ef429b9ccbdfc14f95514858a
Author: Antoine Eiche <email address hidden>
Date: Fri Jul 29 16:54:35 2016 +0200

    Add tempest test associate_disassociate_network

    Network assocation and disassociation client functions have also been
    added.

    Change-Id: I5494c3f7c85105bc22e508e09d2c32a09d664477
    Partial-Bug: #1607664
    (cherry picked from commit e2f276a02fffd116e3019f89321f435fb106aabe)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/381109
Committed: https://git.openstack.org/cgit/openstack/networking-bgpvpn/commit/?id=77f061bb60a9ba2e869de1849ad0ab3aba4f2d4a
Submitter: Jenkins
Branch: stable/mitaka

commit 77f061bb60a9ba2e869de1849ad0ab3aba4f2d4a
Author: Jean-Philippe Braun <email address hidden>
Date: Fri Aug 5 17:13:29 2016 +0200

    Add tempest tests on route-target update

    Change-Id: Ibd2f50797b4d6c6b34cda6bf3f219644af0ffe43
    Partial-Bug: #1607664
    (cherry picked from commit 28dcfbecc28d7f6b6c4c8236ec87ee7a1b7546f2)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/381110
Committed: https://git.openstack.org/cgit/openstack/networking-bgpvpn/commit/?id=b2897aeaf59828b89fa0ee01eba42603551852af
Submitter: Jenkins
Branch: stable/mitaka

commit b2897aeaf59828b89fa0ee01eba42603551852af
Author: suresh kumar <email address hidden>
Date: Wed Aug 10 08:25:51 2016 +0200

    Added the negative cases

    Test checks that invalid route targets are rejected by the create and update api.
    Test checks that invalid network,bgpvpn id are rejected by the associate and disassociate api.

    Change-Id: I181a1876f654ce5821c6bc212696d7be2801acde
    Partial-Bug: 1607664
    (cherry picked from commit c1ab12636c6d98a5a94bd18fc4b08735dea834ab)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/381111
Committed: https://git.openstack.org/cgit/openstack/networking-bgpvpn/commit/?id=df4e5262e39434bdd51d66b31c5f0bcaf5abc5f5
Submitter: Jenkins
Branch: stable/mitaka

commit df4e5262e39434bdd51d66b31c5f0bcaf5abc5f5
Author: bfernando <email address hidden>
Date: Tue Aug 23 16:08:50 2016 +0200

    Add a tempest test on read permission with bgpvpn_list

    * Configure the second openstack user by providing the "alt"
      client_manager.
    * Add the test function which checks that a user cannot read
      bgpvpn resources that it does not own.
    * Add some comments in the BgpvpnTest description.

    Change-Id: Ia1edea9f414384a59b0568d0d733e3413944bd05
    Partial-Bug: #1607664
    (cherry picked from commit 304a7da868fced1b252a16dd4e381b3173b704fa)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/381112
Committed: https://git.openstack.org/cgit/openstack/networking-bgpvpn/commit/?id=dbddaf6aef2f8a524c99a8b83e5a5959d2d32d79
Submitter: Jenkins
Branch: stable/mitaka

commit dbddaf6aef2f8a524c99a8b83e5a5959d2d32d79
Author: bfernando <email address hidden>
Date: Tue Sep 6 17:00:10 2016 +0200

    Add more tempest tests for read permissions

    * Improve the set of tests to be more exhaustive. The goal of these tests is
      to check that the reading restrictions between user of different tenants
      are preserved.
    * The tests added are:
      * show bgpvpn object made by a non owner fail.
      * show network association made by a non owner fail.
      * list network associations made by a non owner fail.
    * Add new methods to the BgpvpnClient (list, show).
    * Rename some methods/calls to be more standard.

    Change-Id: I7a3d773b30aa4ceaf13022f200f6fc6a49934ec2
    Partial-Bug: #1607664
    (cherry picked from commit b5279bf723d5cb32e6f7c363b7dfa727003944c2)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/381113
Committed: https://git.openstack.org/cgit/openstack/networking-bgpvpn/commit/?id=70e80aea2b5dfccb156f29c71d3075544d0df05f
Submitter: Jenkins
Branch: stable/mitaka

commit 70e80aea2b5dfccb156f29c71d3075544d0df05f
Author: Mathieu Rohon <email address hidden>
Date: Fri Sep 9 09:15:14 2016 +0000

    Add tempest tests for router association

    Change-Id: I3dbd54364165b489871214b308a18c7b4a71f91d
    Partial-Bug: #1607664
    (cherry picked from commit 396aee4d0ab65f786db93c22fd3f1b9d2fd6f887)

[Thomas Morin (tmmorin-orange)]

I'm closing this bug, now that we have a reasonable tempest API and scenario coverage, the incremental improvements will be better managed each in their own bug or blueprint.