| 2017-10-27 21:47:37 |
Trevor McCasland |
description |
Given a whitelist of the following format
$ cat whitelist.txt
tempest.api.network.admin.test_floating_ips_admin_actions.FloatingIPAdminTestJSON.test_create_list_show_floating_ip_with_tenant_id_by_admin
tempest.api.network.admin.test_floating_ips_admin_actions.FloatingIPAdminTestJSON.test_list_floating_ips_from_admin_and_nonadmin
tempest.api.network.admin.test_negative_quotas.QuotasNegativeTest.test_network_quota_exceeding
tempest.api.network.test_networks.NetworksIpV6Test.test_external_network_visibility
tempest.api.network.test_networks.NetworksTest.test_external_network_visibility
ostestr --whitelist-file whitelist.txt --list
will not match the list provided.
Perhaps it couldn't find any tests because it does not use regex when listing files from a whitelist and defaults to listing all tests if the result set is empty but I would like to have the output to include the test metadata too, like
tempest.scenario.test_volume_migrate_attached.TestVolumeMigrateRetypeAttached.test_volume_migrate_attached[compute,id-deadd2c2-beef-4dce-98be-f86765ff311b,slow,volume]
When running with a whitelist it does run the correct files (I don't have services running at the moment).
$ ostestr --whitelist-file whitelist.txt
2017-10-26 12:32:35.648 10441 INFO tempest [-] Using tempest config file /etc/tempest/tempest.conf
2017-10-26 12:32:35.676 10443 INFO tempest [-] Using tempest config file /etc/tempest/tempest.conf
2017-10-26 12:32:35.696 10445 INFO tempest [-] Using tempest config file /etc/tempest/tempest.conf
2017-10-26 12:32:35.699 10447 INFO tempest [-] Using tempest config file /etc/tempest/tempest.conf
{0} setUpClass (tempest.api.network.test_networks.NetworksIpV6Test) ... SKIPPED: Neutron support is required
{1} setUpClass (tempest.api.network.admin.test_floating_ips_admin_actions.FloatingIPAdminTestJSON) ... SKIPPED: Neutron support is required
{3} setUpClass (tempest.api.network.admin.test_negative_quotas.QuotasNegativeTest) ... SKIPPED: Neutron support is required
{2} setUpClass (tempest.api.network.test_networks.NetworksTest) ... SKIPPED: Neutron support is required
as advertised, the list option states:
--list, -l List all the tests which will be run.
but we can clearly see the --list option print all the tests, I did not include them for brevity but you can try it out.
stestr works fine though so we can skip this if we plan on replacing ostestr with stestr :)
$ stestr list --whitelist-file whitelist.txt
tempest.api.network.admin.test_floating_ips_admin_actions.FloatingIPAdminTestJSON.test_list_floating_ips_from_admin_and_nonadmin[id-64f2100b-5471-4ded-b46c-ddeeeb4f231b]
tempest.api.network.test_networks.NetworksTest.test_external_network_visibility[id-af774677-42a9-4e4b-bb58-16fe6a5bc1ec,smoke]
tempest.api.network.test_networks.NetworksIpV6Test.test_external_network_visibility[id-af774677-42a9-4e4b-bb58-16fe6a5bc1ec,smoke]
tempest.api.network.admin.test_negative_quotas.QuotasNegativeTest.test_network_quota_exceeding[id-644f4e1b-1bf9-4af0-9fd8-eb56ac0f51cf,negative]
tempest.api.network.admin.test_floating_ips_admin_actions.FloatingIPAdminTestJSON.test_create_list_show_floating_ip_with_tenant_id_by_admin[id-32727cc3-abe2-4485-a16e-48f2d54c14f2] |
Steps to reproduce
1. clone tempest
2. install requirements
3. ostestr --list --whitelist-file whitelist.txt # lists all tests
4. stestr list --whitelist-file whitelist.txt # lists whitelisted tests
Given a whitelist of the following format
$ cat whitelist.txt
tempest.api.network.admin.test_floating_ips_admin_actions.FloatingIPAdminTestJSON.test_create_list_show_floating_ip_with_tenant_id_by_admin
tempest.api.network.admin.test_floating_ips_admin_actions.FloatingIPAdminTestJSON.test_list_floating_ips_from_admin_and_nonadmin
tempest.api.network.admin.test_negative_quotas.QuotasNegativeTest.test_network_quota_exceeding
tempest.api.network.test_networks.NetworksIpV6Test.test_external_network_visibility
tempest.api.network.test_networks.NetworksTest.test_external_network_visibility
ostestr --whitelist-file whitelist.txt --list
will not match the list provided.
Perhaps it couldn't find any tests because it does not use regex when listing files from a whitelist and defaults to listing all tests if the result set is empty but I would like to have the output to include the test metadata too, like
tempest.scenario.test_volume_migrate_attached.TestVolumeMigrateRetypeAttached.test_volume_migrate_attached[compute,id-deadd2c2-beef-4dce-98be-f86765ff311b,slow,volume]
When running with a whitelist it does run the correct files (I don't have services running at the moment).
$ ostestr --whitelist-file whitelist.txt
2017-10-26 12:32:35.648 10441 INFO tempest [-] Using tempest config file /etc/tempest/tempest.conf
2017-10-26 12:32:35.676 10443 INFO tempest [-] Using tempest config file /etc/tempest/tempest.conf
2017-10-26 12:32:35.696 10445 INFO tempest [-] Using tempest config file /etc/tempest/tempest.conf
2017-10-26 12:32:35.699 10447 INFO tempest [-] Using tempest config file /etc/tempest/tempest.conf
{0} setUpClass (tempest.api.network.test_networks.NetworksIpV6Test) ... SKIPPED: Neutron support is required
{1} setUpClass (tempest.api.network.admin.test_floating_ips_admin_actions.FloatingIPAdminTestJSON) ... SKIPPED: Neutron support is required
{3} setUpClass (tempest.api.network.admin.test_negative_quotas.QuotasNegativeTest) ... SKIPPED: Neutron support is required
{2} setUpClass (tempest.api.network.test_networks.NetworksTest) ... SKIPPED: Neutron support is required
as advertised, the list option states:
--list, -l List all the tests which will be run.
but we can clearly see the --list option print all the tests, I did not include them for brevity but you can try it out.
stestr works fine though so we can skip this if we plan on replacing ostestr with stestr :)
$ stestr list --whitelist-file whitelist.txt
tempest.api.network.admin.test_floating_ips_admin_actions.FloatingIPAdminTestJSON.test_list_floating_ips_from_admin_and_nonadmin[id-64f2100b-5471-4ded-b46c-ddeeeb4f231b]
tempest.api.network.test_networks.NetworksTest.test_external_network_visibility[id-af774677-42a9-4e4b-bb58-16fe6a5bc1ec,smoke]
tempest.api.network.test_networks.NetworksIpV6Test.test_external_network_visibility[id-af774677-42a9-4e4b-bb58-16fe6a5bc1ec,smoke]
tempest.api.network.admin.test_negative_quotas.QuotasNegativeTest.test_network_quota_exceeding[id-644f4e1b-1bf9-4af0-9fd8-eb56ac0f51cf,negative]
tempest.api.network.admin.test_floating_ips_admin_actions.FloatingIPAdminTestJSON.test_create_list_show_floating_ip_with_tenant_id_by_admin[id-32727cc3-abe2-4485-a16e-48f2d54c14f2] |
|
