ostestr can't run with both --blacklist_file and --regex at the same time

[tuannguyen]

When I ran:
ostestr --blacklist_file blankfile.txt --regex smoke

Where blankfile.txt is emptied, the suite was able to execute with all the tempest smoke test.

However, when I added a line to exclude the following test to the blacklist file: tempest.scenario.test_volume_boot_pattern.TestVolumeBootPatternV2.test_volume_boot_pattern

Then I am unable to run a non-emptied blacklist file as shown below:
(tempest) root@uat-dal09-compute-313:/var/lib/tempest/default# ostestr --blacklist_file 1-test.txt --regex smoke --list
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-500} \
${PYTHON:-python} -m subunit.run discover -t /opt/bbc/openstack-11.0-master/tempest/local/lib/python2.7/site-packages/tempest /opt/bbc/openstack-11.0-master/tempest/local/lib/python2.7/site-packages/tempest/test_discover --list
Option "verbose" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future.
(tempest) root@uat-dal09-compute-313:/var/lib/tempest/default#

It seems that using the combination of both --blacklist_file and --regex at the same time doesn't work when the blacklist file has some tests to be excluded. When the file is emptied, then the above command works.

[Masayuki Igawa (igawa)]

In my environment, I can get a test list and run like below.

So, could you please add more information? such as ostestr version, tempest version, etc.
------------
$ ostestr --version
0.8.0
$ cat ~/tmp/blankfile.txt
tempest.scenario.test_volume_boot_pattern.TestVolumeBootPatternV2.test_volume_boot_pattern
$ ostestr --blacklist_file ~/tmp/blankfile.txt --regex smoke --list | wc -l
2016-11-16 17:10:30.052 16441 INFO tempest [-] Using tempest config file /etc/tempest/tempest.conf
     118
$ ostestr --blacklist_file ~/tmp/blankfile.txt --regex smoke
2016-11-16 17:12:33.054 16638 INFO tempest [-] Using tempest config file /etc/tempest/tempest.conf
 <...>
{0} setUpClass (tempest.api.compute.security_groups.test_security_group_rules.SecurityGroupRulesTestJSON) [0.000000s] ... FAILED
 <...>
------------

[Launchpad Janitor (janitor)]

[Expired for tempest because there has been no activity for 60 days.]

[Szymon Datko (sdatko)]

I proposed a patchset that fixes the issue in os-testr library: https://review.openstack.org/#/c/522768/

The problem is related to incorrect regex contructor present in os-testr and used when executing `tempest run` command.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/os-testr 1.1.0 release.