nose TestResult object is missing the addSkip method and addError method is invoked

[Nathan Biles]

The tempest/tests/volume/admin/test_multi_backend.py test is not being skipped for me (it's failing
in the setup method) even though I have multi_backend_enabled = false in tempest.conf

[Giulio Fidente (gfidente)]

are you sure you have set the option under the [volume] section of the tempest.conf file?

[Matt Riedemann (mriedem)]

Yeah, the test result actually logs this too:

"Cinder multi-backend feature is not available"

We're just cloning tempest from github on the master branch for each run in a new VM and copying tempest/etc/tempest.conf.sample to tempest/etc/tempest.conf and then setting some values (like the credentials, images to use, flavors, etc).

Fails like this:

  File "/usr/lib/python2.6/site-packages/nose/suite.py", line 208, in run
    self.setUp()
  File "/usr/lib/python2.6/site-packages/nose/suite.py", line 291, in setUp
    self.setupContext(ancestor)
  File "/usr/lib/python2.6/site-packages/nose/suite.py", line 314, in setupContext
    try_run(context, names)
  File "/usr/lib/python2.6/site-packages/nose/util.py", line 469, in try_run
    return func()
  File "/usr/lib/python2.6/site-packages/testtools/testcase.py", line 780, in skip_wrapper
    raise TestCase.skipException(reason)
Cinder multi-backend feature is not available

It should be noted this is running on python 2.6. We upgrade testtools to the latest version for each run of Tempest.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/30760

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/30760
Committed: http://github.com/openstack/tempest/commit/f4fa894fba31cbfd6234bd8b37df2b0c126be748
Submitter: Jenkins
Branch: master

commit f4fa894fba31cbfd6234bd8b37df2b0c126be748
Author: Giulio Fidente <email address hidden>
Date: Tue May 28 18:48:03 2013 +0200

    Fixes the multi-backend skip bug and the cleanup steps

    The multi-backend test class and the tests within should now be
    skipped correctly also when running on python 2.6. Also fixes
    a cleanup problem occurring in case of error in setUpClass and
    speed things up by not creating the 2nd volume/type if backends
    have the same name (in which case only part of the tests can be
    executed).

    Change-Id: Ibeb961ea000bb1d1034f857a64552c7aa75ed4d5
    Fixes: bug #1177608

[Nathan Biles (npbiles)]

Now I'm seeing this:

 File "/usr/lib/python2.6/site-packages/nose/suite.py", line 208, in run
    self.setUp()
  File "/usr/lib/python2.6/site-packages/nose/suite.py", line 291, in setUp
    self.setupContext(ancestor)
  File "/usr/lib/python2.6/site-packages/nose/suite.py", line 314, in setupContext
    try_run(context, names)
  File "/usr/lib/python2.6/site-packages/nose/util.py", line 469, in try_run
    return func()
  File "/tmp/tempest/tempest/tempest/api/volume/admin/test_multi_backend.py", line 32, in setUpClass
    raise cls.skipException("Cinder multi-backend feature disabled")

Should that result in an error when running tempest?

[Giulio Fidente (gfidente)]

I managed to reproduce the error , thanks for pointing this out.

[Giulio Fidente (gfidente)]

I'm going to update the title, the bug seems related to how the skipException is catched in py26 and affects all the tests skips which use it (not just the multi_backend thing)

[Robert Collins (lifeless)]

Where are you importing SkipTest from?

[Giulio Fidente (gfidente)]

we skip using skipException from within the setUpClass

[Robert Collins (lifeless)]

sorry, I misunderstood the question. The issue is that nose is using the python standard library result object and not fixing it up to include addSkip.

[Giulio Fidente (gfidente)]

<lifeless> python 2.6 has no setUpClass.
<lifeless> testtools has no setUpClass
<lifeless> so the call stack is
<lifeless> nose
<lifeless> SomeClass.setUpClass
<lifeless> ... raise skipException
<lifeless> and unless skipException is an exception type nose expects to catch, nose will perceive this as an error.
<lifeless> so. I think this is the issue
<lifeless> https://github.com/nose-devs/nose/blob/master/nose/plugins/skip.py
<lifeless> see how it tries to import from unittest
<lifeless> in 2.7 nose has imported it from there
<lifeless> and so has testtools
<lifeless> so testtools' skipException type is the same type.
<lifeless> In 2.6, we have to make our own, and so does nose, so they are different, and what you're raising it doesn't know about.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/33056

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/33769

[Giulio Fidente (gfidente)]

I think this has finally been fixed with the following: https://review.openstack.org/#/c/38284/

I'm moving the bug in FIXED state.

[Matt Riedemann (mriedem)]

Yup, confirmed this is fixed when running tempest on python 2.6 with the environment variable set to enable the monkey patch:

http://paste.openstack.org/show/44322/