| 2013-11-15 03:55:08 |
IAN DELANEY |
bug |
|
|
added bug |
| 2013-11-15 03:58:31 |
IAN DELANEY |
description |
tox in gentoo is simply toxic when it comes to running tests. It woud be so much more sane to assure that the testsuite will pass on the system installed dependencies. While tox has its place, I avoid it like the plague in testsuites in python based packages as a gentoo dev package maintainer. Gentoo also has a rule of no connecting to the internet in a test phase. Sheeeeeeesh.
Sure nosetests is just another test runner, but it's my first port of call.
testuser@archtester /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0 $ ./run_tests.sh
GLOB sdist-make: /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/setup.py
py27 create: /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/.tox/py27
py27 installdeps: -r/mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/requirements.txt, -r/mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/test-requirements.txt
py27 inst: /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/.tox/dist/python-swiftclient-1.6.0.zip
py27 runtests: commands[0]
running testr
running=${PYTHON:-python} -m subunit.run discover -t ./ ./tests --list
running=${PYTHON:-python} -m subunit.run discover -t ./ ./tests --load-list /tmp/tmpbYu4tl
running=${PYTHON:-python} -m subunit.run discover -t ./ ./tests --load-list /tmp/tmpn6K8gN
running=${PYTHON:-python} -m subunit.run discover -t ./ ./tests --load-list /tmp/tmp30rhT8
running=${PYTHON:-python} -m subunit.run discover -t ./ ./tests --load-list /tmp/tmpQ_1yJU
Ran 75 tests in 0.796s
PASSED (id=0)
___________________________________ summary ____________________________________
py27: commands succeeded
congratulations :)
GLOB sdist-make: /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/setup.py
pep8 create: /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/.tox/pep8
pep8 installdeps: -r/mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/requirements.txt, -r/mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/test-requirements.txt
pep8 inst: /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/.tox/dist/python-swiftclient-1.6.0.zip
pep8 runtests: commands[0]
pep8 runtests: commands[1]
____________________________________________________________ summary _____________________________________________________________
pep8: commands succeeded
congratulations :)
says tox worked and the testsuite passed. Well, nice, but for breaking gentoo policy in running tests at the very first step.
testuser@archtester /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0 $ PYTHONPATH=. nosetests tests
F..........................................................................
======================================================================
FAIL: tests.test_multithreading.TestMultiThreadingManager.test_instantiation
----------------------------------------------------------------------
_StringException: Traceback (most recent call last):
File "/usr/lib64/python2.7/site-packages/mock.py", line 1224, in patched
return func(*args, **keywargs)
File "/mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/tests/test_multithreading.py", line 248, in test_instantiation
self.assertEqual(sys.stdout, thread_manager.print_stream)
File "/usr/lib64/python2.7/site-packages/testtools/testcase.py", line 322, in assertEqual
self.assertThat(observed, matcher, message)
File "/usr/lib64/python2.7/site-packages/testtools/testcase.py", line 417, in assertThat
raise MismatchError(matchee, matcher, mismatch, verbose)
MismatchError: !=:
reference = <StringIO.StringIO instance at 0x1305f38>
actual = <StringIO.StringIO instance at 0xdcd680>
----------------------------------------------------------------------
Ran 75 tests in 1.699s
FAILED (failures=1)
says we have a problem possibly with system installed mock or testtools and it did in fact run 75 tests.
testuser@archtester /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0 $ PYTHONPATH=. py.test tests
====================================================== test session starts =======================================================
platform linux2 -- Python 2.7.5 -- pytest-2.3.5
collected 75 items
tests/test_multithreading.py sssssssssss
tests/test_swiftclient.py ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss
says let's take py.test out of the equation.
Now you could take the easy way out here and dismiss this whole entry since your virtual environment making to sees the testsuite pass. That leaves something amiss here though.
Now here is a classic conflict.
./.tox/pep8/lib/python2.7/site-packages/python_subunit-0.0.15-py2.7.egg-info/PKG-INFO:
says the virtual env has installed python-subunit. Gentoo's python-subunit is in fact called subunit. We have
/usr/lib64/python2.7/site-packages/subunit/.......
oooops.
I am guessing the use of a virtual env. assumes the instances of the virtual installed deps will perfectly match the system installed packages. Sorry to shatter that one
So, could you consider making a testsuite that runs perfectly under nose or pytest or
$ python tests/test_all.py?
It's really quite a sane request. 'We' can't use tox as a test runner for all the varioous reasons above. At this point I have the tox run testsuite declaring the tests fine and who knows what will happen on using system installed deps once installed. Only a few days ago I got a different result again where the issue was the version of testtools. |
tox in gentoo is simply toxic when it comes to running tests. It woud be so much more sane to assure that the testsuite will pass on the system installed dependencies. While tox has its place, I avoid it like the plague in testsuites in python based packages as a gentoo dev package maintainer. Gentoo also has a rule of no connecting to the internet in a test phase. Sheeeeeeesh.
Sure nosetests is just another test runner, but it's my first port of call.
testuser@archtester /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0 $ ./run_tests.sh
GLOB sdist-make: /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/setup.py
py27 create: /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/.tox/py27
py27 installdeps: -r/mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/requirements.txt, -r/mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/test-requirements.txt
py27 inst: /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/.tox/dist/python-swiftclient-1.6.0.zip
py27 runtests: commands[0]
running testr
running=${PYTHON:-python} -m subunit.run discover -t ./ ./tests --list
running=${PYTHON:-python} -m subunit.run discover -t ./ ./tests --load-list /tmp/tmpbYu4tl
running=${PYTHON:-python} -m subunit.run discover -t ./ ./tests --load-list /tmp/tmpn6K8gN
running=${PYTHON:-python} -m subunit.run discover -t ./ ./tests --load-list /tmp/tmp30rhT8
running=${PYTHON:-python} -m subunit.run discover -t ./ ./tests --load-list /tmp/tmpQ_1yJU
Ran 75 tests in 0.796s
PASSED (id=0)
___________________________________ summary ____________________________________
py27: commands succeeded
congratulations :)
GLOB sdist-make: /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/setup.py
pep8 create: /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/.tox/pep8
pep8 installdeps: -r/mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/requirements.txt, -r/mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/test-requirements.txt
pep8 inst: /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/.tox/dist/python-swiftclient-1.6.0.zip
pep8 runtests: commands[0]
pep8 runtests: commands[1]
____________________________________________________________ summary _____________________________________________________________
pep8: commands succeeded
congratulations :)
says tox worked and the testsuite passed. Well, nice, but for breaking gentoo policy in running tests at the very first step.
testuser@archtester /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0 $ PYTHONPATH=. nosetests tests
F..........................................................................
======================================================================
FAIL: tests.test_multithreading.TestMultiThreadingManager.test_instantiation
----------------------------------------------------------------------
_StringException: Traceback (most recent call last):
File "/usr/lib64/python2.7/site-packages/mock.py", line 1224, in patched
return func(*args, **keywargs)
File "/mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/tests/test_multithreading.py", line 248, in test_instantiation
self.assertEqual(sys.stdout, thread_manager.print_stream)
File "/usr/lib64/python2.7/site-packages/testtools/testcase.py", line 322, in assertEqual
self.assertThat(observed, matcher, message)
File "/usr/lib64/python2.7/site-packages/testtools/testcase.py", line 417, in assertThat
raise MismatchError(matchee, matcher, mismatch, verbose)
MismatchError: !=:
reference = <StringIO.StringIO instance at 0x1305f38>
actual = <StringIO.StringIO instance at 0xdcd680>
----------------------------------------------------------------------
Ran 75 tests in 1.699s
FAILED (failures=1)
says we have a problem possibly with system installed mock or testtools and it did in fact run 75 tests.
testuser@archtester /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0 $ PYTHONPATH=. py.test tests
====================================================== test session starts =======================================================
platform linux2 -- Python 2.7.5 -- pytest-2.3.5
collected 75 items
tests/test_multithreading.py sssssssssss
tests/test_swiftclient.py ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss
says let's take py.test out of the equation.
Now you could take the easy way out here and dismiss this whole entry since your 'virtual environment making' sees the testsuite pass. That leaves something amiss here though.
Now here is a classic conflict.
./.tox/pep8/lib/python2.7/site-packages/python_subunit-0.0.15-py2.7.egg-info/PKG-INFO:
says the virtual env has installed python-subunit. Gentoo's python-subunit is in fact called subunit. We have
/usr/lib64/python2.7/site-packages/subunit/.......
oooops.
I am guessing the use of a virtual env. assumes the instances of the virtual installed deps will perfectly match the system installed packages. Sorry to shatter that one
So, could you consider making a testsuite that runs perfectly under nose or pytest or
$ python tests/test_all.py?
It's really quite a sane request. 'We' can't use tox as a test runner for all the varioous reasons above. At this point I have the tox run testsuite declaring the tests fine and who knows what will happen on using system installed deps once installed. Only a few days ago I got a different result again where the issue was the version of testtools. |
|
| 2013-11-15 03:59:23 |
IAN DELANEY |
description |
tox in gentoo is simply toxic when it comes to running tests. It woud be so much more sane to assure that the testsuite will pass on the system installed dependencies. While tox has its place, I avoid it like the plague in testsuites in python based packages as a gentoo dev package maintainer. Gentoo also has a rule of no connecting to the internet in a test phase. Sheeeeeeesh.
Sure nosetests is just another test runner, but it's my first port of call.
testuser@archtester /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0 $ ./run_tests.sh
GLOB sdist-make: /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/setup.py
py27 create: /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/.tox/py27
py27 installdeps: -r/mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/requirements.txt, -r/mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/test-requirements.txt
py27 inst: /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/.tox/dist/python-swiftclient-1.6.0.zip
py27 runtests: commands[0]
running testr
running=${PYTHON:-python} -m subunit.run discover -t ./ ./tests --list
running=${PYTHON:-python} -m subunit.run discover -t ./ ./tests --load-list /tmp/tmpbYu4tl
running=${PYTHON:-python} -m subunit.run discover -t ./ ./tests --load-list /tmp/tmpn6K8gN
running=${PYTHON:-python} -m subunit.run discover -t ./ ./tests --load-list /tmp/tmp30rhT8
running=${PYTHON:-python} -m subunit.run discover -t ./ ./tests --load-list /tmp/tmpQ_1yJU
Ran 75 tests in 0.796s
PASSED (id=0)
___________________________________ summary ____________________________________
py27: commands succeeded
congratulations :)
GLOB sdist-make: /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/setup.py
pep8 create: /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/.tox/pep8
pep8 installdeps: -r/mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/requirements.txt, -r/mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/test-requirements.txt
pep8 inst: /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/.tox/dist/python-swiftclient-1.6.0.zip
pep8 runtests: commands[0]
pep8 runtests: commands[1]
____________________________________________________________ summary _____________________________________________________________
pep8: commands succeeded
congratulations :)
says tox worked and the testsuite passed. Well, nice, but for breaking gentoo policy in running tests at the very first step.
testuser@archtester /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0 $ PYTHONPATH=. nosetests tests
F..........................................................................
======================================================================
FAIL: tests.test_multithreading.TestMultiThreadingManager.test_instantiation
----------------------------------------------------------------------
_StringException: Traceback (most recent call last):
File "/usr/lib64/python2.7/site-packages/mock.py", line 1224, in patched
return func(*args, **keywargs)
File "/mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/tests/test_multithreading.py", line 248, in test_instantiation
self.assertEqual(sys.stdout, thread_manager.print_stream)
File "/usr/lib64/python2.7/site-packages/testtools/testcase.py", line 322, in assertEqual
self.assertThat(observed, matcher, message)
File "/usr/lib64/python2.7/site-packages/testtools/testcase.py", line 417, in assertThat
raise MismatchError(matchee, matcher, mismatch, verbose)
MismatchError: !=:
reference = <StringIO.StringIO instance at 0x1305f38>
actual = <StringIO.StringIO instance at 0xdcd680>
----------------------------------------------------------------------
Ran 75 tests in 1.699s
FAILED (failures=1)
says we have a problem possibly with system installed mock or testtools and it did in fact run 75 tests.
testuser@archtester /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0 $ PYTHONPATH=. py.test tests
====================================================== test session starts =======================================================
platform linux2 -- Python 2.7.5 -- pytest-2.3.5
collected 75 items
tests/test_multithreading.py sssssssssss
tests/test_swiftclient.py ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss
says let's take py.test out of the equation.
Now you could take the easy way out here and dismiss this whole entry since your 'virtual environment making' sees the testsuite pass. That leaves something amiss here though.
Now here is a classic conflict.
./.tox/pep8/lib/python2.7/site-packages/python_subunit-0.0.15-py2.7.egg-info/PKG-INFO:
says the virtual env has installed python-subunit. Gentoo's python-subunit is in fact called subunit. We have
/usr/lib64/python2.7/site-packages/subunit/.......
oooops.
I am guessing the use of a virtual env. assumes the instances of the virtual installed deps will perfectly match the system installed packages. Sorry to shatter that one
So, could you consider making a testsuite that runs perfectly under nose or pytest or
$ python tests/test_all.py?
It's really quite a sane request. 'We' can't use tox as a test runner for all the varioous reasons above. At this point I have the tox run testsuite declaring the tests fine and who knows what will happen on using system installed deps once installed. Only a few days ago I got a different result again where the issue was the version of testtools. |
tox in gentoo is simply toxic when it comes to running tests. It woud be so much more sane to assure that the testsuite will pass on the system installed dependencies. While tox has its place, I avoid it like the plague in testsuites in python based packages as a gentoo dev package maintainer. Gentoo also has a rule of no connecting to the internet in a test phase. Sheeeeeeesh.
Sure nosetests is just another test runner, but it's my first port of call.
testuser@archtester /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0 $ ./run_tests.sh
GLOB sdist-make: /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/setup.py
py27 create: /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/.tox/py27
py27 installdeps: -r/mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/requirements.txt, -r/mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/test-requirements.txt
py27 inst: /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/.tox/dist/python-swiftclient-1.6.0.zip
py27 runtests: commands[0]
running testr
running=${PYTHON:-python} -m subunit.run discover -t ./ ./tests --list
running=${PYTHON:-python} -m subunit.run discover -t ./ ./tests --load-list /tmp/tmpbYu4tl
running=${PYTHON:-python} -m subunit.run discover -t ./ ./tests --load-list /tmp/tmpn6K8gN
running=${PYTHON:-python} -m subunit.run discover -t ./ ./tests --load-list /tmp/tmp30rhT8
running=${PYTHON:-python} -m subunit.run discover -t ./ ./tests --load-list /tmp/tmpQ_1yJU
Ran 75 tests in 0.796s
PASSED (id=0)
___________________________________ summary ____________________________________
py27: commands succeeded
congratulations :)
GLOB sdist-make: /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/setup.py
pep8 create: /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/.tox/pep8
pep8 installdeps: -r/mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/requirements.txt, -r/mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/test-requirements.txt
pep8 inst: /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/.tox/dist/python-swiftclient-1.6.0.zip
pep8 runtests: commands[0]
pep8 runtests: commands[1]
____________________________________________________________ summary _____________________________________________________________
pep8: commands succeeded
congratulations :)
says tox worked and the testsuite passed. Well, nice, but for breaking gentoo policy in running tests at the very first step.
testuser@archtester /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0 $ PYTHONPATH=. nosetests tests
F..........................................................................
======================================================================
FAIL: tests.test_multithreading.TestMultiThreadingManager.test_instantiation
----------------------------------------------------------------------
_StringException: Traceback (most recent call last):
File "/usr/lib64/python2.7/site-packages/mock.py", line 1224, in patched
return func(*args, **keywargs)
File "/mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/tests/test_multithreading.py", line 248, in test_instantiation
self.assertEqual(sys.stdout, thread_manager.print_stream)
File "/usr/lib64/python2.7/site-packages/testtools/testcase.py", line 322, in assertEqual
self.assertThat(observed, matcher, message)
File "/usr/lib64/python2.7/site-packages/testtools/testcase.py", line 417, in assertThat
raise MismatchError(matchee, matcher, mismatch, verbose)
MismatchError: !=:
reference = <StringIO.StringIO instance at 0x1305f38>
actual = <StringIO.StringIO instance at 0xdcd680>
----------------------------------------------------------------------
Ran 75 tests in 1.699s
FAILED (failures=1)
says we have a problem possibly with system installed mock or testtools and it did in fact run 75 tests.
testuser@archtester /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0 $ PYTHONPATH=. py.test tests
====================================================== test session starts =======================================================
platform linux2 -- Python 2.7.5 -- pytest-2.3.5
collected 75 items
tests/test_multithreading.py sssssssssss
tests/test_swiftclient.py ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss
says let's take py.test out of the equation.
Now you could take the easy way out here and dismiss this whole entry since your 'virtual environment making' sees the testsuite pass. That leaves something amiss here though.
Now here is a classic conflict.
./.tox/pep8/lib/python2.7/site-packages/python_subunit-0.0.15-py2.7.egg-info/PKG-INFO:
says the virtual env has installed python-subunit. Gentoo's python-subunit is in fact called subunit. We have
/usr/lib64/python2.7/site-packages/subunit/.......
oooops.
I am guessing the use of a virtual env. assumes the instances of the virtual installed deps will perfectly match the system installed packages. Sorry to shatter that one for you.
So, could you consider making a testsuite that runs perfectly under nose or pytest or
$ python tests/test_all.py?
It's really quite a sane request. 'We' can't use tox as a test runner for all the varioous reasons above. At this point I have the tox run testsuite declaring the tests fine and who knows what will happen on using system installed deps once installed. Only a few days ago I got a different result again where the issue was the version of testtools. |
|
| 2013-11-15 12:44:55 |
IAN DELANEY |
description |
tox in gentoo is simply toxic when it comes to running tests. It woud be so much more sane to assure that the testsuite will pass on the system installed dependencies. While tox has its place, I avoid it like the plague in testsuites in python based packages as a gentoo dev package maintainer. Gentoo also has a rule of no connecting to the internet in a test phase. Sheeeeeeesh.
Sure nosetests is just another test runner, but it's my first port of call.
testuser@archtester /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0 $ ./run_tests.sh
GLOB sdist-make: /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/setup.py
py27 create: /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/.tox/py27
py27 installdeps: -r/mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/requirements.txt, -r/mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/test-requirements.txt
py27 inst: /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/.tox/dist/python-swiftclient-1.6.0.zip
py27 runtests: commands[0]
running testr
running=${PYTHON:-python} -m subunit.run discover -t ./ ./tests --list
running=${PYTHON:-python} -m subunit.run discover -t ./ ./tests --load-list /tmp/tmpbYu4tl
running=${PYTHON:-python} -m subunit.run discover -t ./ ./tests --load-list /tmp/tmpn6K8gN
running=${PYTHON:-python} -m subunit.run discover -t ./ ./tests --load-list /tmp/tmp30rhT8
running=${PYTHON:-python} -m subunit.run discover -t ./ ./tests --load-list /tmp/tmpQ_1yJU
Ran 75 tests in 0.796s
PASSED (id=0)
___________________________________ summary ____________________________________
py27: commands succeeded
congratulations :)
GLOB sdist-make: /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/setup.py
pep8 create: /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/.tox/pep8
pep8 installdeps: -r/mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/requirements.txt, -r/mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/test-requirements.txt
pep8 inst: /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/.tox/dist/python-swiftclient-1.6.0.zip
pep8 runtests: commands[0]
pep8 runtests: commands[1]
____________________________________________________________ summary _____________________________________________________________
pep8: commands succeeded
congratulations :)
says tox worked and the testsuite passed. Well, nice, but for breaking gentoo policy in running tests at the very first step.
testuser@archtester /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0 $ PYTHONPATH=. nosetests tests
F..........................................................................
======================================================================
FAIL: tests.test_multithreading.TestMultiThreadingManager.test_instantiation
----------------------------------------------------------------------
_StringException: Traceback (most recent call last):
File "/usr/lib64/python2.7/site-packages/mock.py", line 1224, in patched
return func(*args, **keywargs)
File "/mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/tests/test_multithreading.py", line 248, in test_instantiation
self.assertEqual(sys.stdout, thread_manager.print_stream)
File "/usr/lib64/python2.7/site-packages/testtools/testcase.py", line 322, in assertEqual
self.assertThat(observed, matcher, message)
File "/usr/lib64/python2.7/site-packages/testtools/testcase.py", line 417, in assertThat
raise MismatchError(matchee, matcher, mismatch, verbose)
MismatchError: !=:
reference = <StringIO.StringIO instance at 0x1305f38>
actual = <StringIO.StringIO instance at 0xdcd680>
----------------------------------------------------------------------
Ran 75 tests in 1.699s
FAILED (failures=1)
says we have a problem possibly with system installed mock or testtools and it did in fact run 75 tests.
testuser@archtester /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0 $ PYTHONPATH=. py.test tests
====================================================== test session starts =======================================================
platform linux2 -- Python 2.7.5 -- pytest-2.3.5
collected 75 items
tests/test_multithreading.py sssssssssss
tests/test_swiftclient.py ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss
says let's take py.test out of the equation.
Now you could take the easy way out here and dismiss this whole entry since your 'virtual environment making' sees the testsuite pass. That leaves something amiss here though.
Now here is a classic conflict.
./.tox/pep8/lib/python2.7/site-packages/python_subunit-0.0.15-py2.7.egg-info/PKG-INFO:
says the virtual env has installed python-subunit. Gentoo's python-subunit is in fact called subunit. We have
/usr/lib64/python2.7/site-packages/subunit/.......
oooops.
I am guessing the use of a virtual env. assumes the instances of the virtual installed deps will perfectly match the system installed packages. Sorry to shatter that one for you.
So, could you consider making a testsuite that runs perfectly under nose or pytest or
$ python tests/test_all.py?
It's really quite a sane request. 'We' can't use tox as a test runner for all the varioous reasons above. At this point I have the tox run testsuite declaring the tests fine and who knows what will happen on using system installed deps once installed. Only a few days ago I got a different result again where the issue was the version of testtools. |
tox in gentoo is simply toxic when it comes to running tests. It woud be so much more sane to assure that the testsuite will pass on the system installed dependencies. While tox has its place, I avoid it like the plague in testsuites in python based packages as a gentoo dev package maintainer. Gentoo also has a rule of no connecting to the internet in a test phase. Sheeeeeeesh.
Sure nosetests is just another test runner, but it's my first port of call.
testuser@archtester /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0 $ ./run_tests.sh
GLOB sdist-make: /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/setup.py
py27 create: /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/.tox/py27
py27 installdeps: -r/mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/requirements.txt, -r/mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/test-requirements.txt
py27 inst: /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/.tox/dist/python-swiftclient-1.6.0.zip
py27 runtests: commands[0]
running testr
running=${PYTHON:-python} -m subunit.run discover -t ./ ./tests --list
running=${PYTHON:-python} -m subunit.run discover -t ./ ./tests --load-list /tmp/tmpbYu4tl
running=${PYTHON:-python} -m subunit.run discover -t ./ ./tests --load-list /tmp/tmpn6K8gN
running=${PYTHON:-python} -m subunit.run discover -t ./ ./tests --load-list /tmp/tmp30rhT8
running=${PYTHON:-python} -m subunit.run discover -t ./ ./tests --load-list /tmp/tmpQ_1yJU
Ran 75 tests in 0.796s
PASSED (id=0)
___________________________________ summary ____________________________________
py27: commands succeeded
congratulations :)
GLOB sdist-make: /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/setup.py
pep8 create: /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/.tox/pep8
pep8 installdeps: -r/mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/requirements.txt, -r/mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/test-requirements.txt
pep8 inst: /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/.tox/dist/python-swiftclient-1.6.0.zip
pep8 runtests: commands[0]
pep8 runtests: commands[1]
____________________________________________________________ summary _____________________________________________________________
pep8: commands succeeded
congratulations :)
says tox worked and the testsuite passed. Well, nice, but for breaking gentoo policy in running tests at the very first step.
testuser@archtester /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0 $ PYTHONPATH=. nosetests tests
F..........................................................................
======================================================================
FAIL: tests.test_multithreading.TestMultiThreadingManager.test_instantiation
----------------------------------------------------------------------
_StringException: Traceback (most recent call last):
File "/usr/lib64/python2.7/site-packages/mock.py", line 1224, in patched
return func(*args, **keywargs)
File "/mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0/tests/test_multithreading.py", line 248, in test_instantiation
self.assertEqual(sys.stdout, thread_manager.print_stream)
File "/usr/lib64/python2.7/site-packages/testtools/testcase.py", line 322, in assertEqual
self.assertThat(observed, matcher, message)
File "/usr/lib64/python2.7/site-packages/testtools/testcase.py", line 417, in assertThat
raise MismatchError(matchee, matcher, mismatch, verbose)
MismatchError: !=:
reference = <StringIO.StringIO instance at 0x1305f38>
actual = <StringIO.StringIO instance at 0xdcd680>
----------------------------------------------------------------------
Ran 75 tests in 1.699s
FAILED (failures=1)
says we have a problem possibly with system installed mock or testtools and it did in fact run 75 tests.
testuser@archtester /mnt/gen2/TmpDir/portage/dev-python/python-swiftclient-1.6.0/work/python-swiftclient-1.6.0 $ PYTHONPATH=. py.test tests
====================================================== test session starts =======================================================
platform linux2 -- Python 2.7.5 -- pytest-2.3.5
collected 75 items
tests/test_multithreading.py sssssssssss
tests/test_swiftclient.py ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss
says let's take py.test out of the equation.
Now you could take the easy way out here and dismiss this whole entry since your 'virtual environment making' sees the testsuite pass. That leaves something amiss here though.
Now here is a classic conflict.
./.tox/pep8/lib/python2.7/site-packages/python_subunit-0.0.15-py2.7.egg-info/PKG-INFO:
says the virtual env has installed python-subunit. Gentoo's python-subunit is in fact called subunit. We have
/usr/lib64/python2.7/site-packages/subunit/.......
oooops.
I am guessing the use of a virtual env. assumes the instances of the virtual installed deps will perfectly match the system installed packages. Sorry to shatter that one for you.
So, could you consider making a testsuite that runs perfectly under nose or pytest or
$ python tests/test_all.py?
It's really quite a sane request. 'We' can't use tox as a test runner for all the various reasons above. At this point I have the tox run testsuite declaring the tests fine and who knows what will happen on using system installed deps once installed. Only a few days ago I got a different result again where the issue was the version of testtools. |
|
| 2013-12-31 15:46:55 |
Monty Taylor |
python-swiftclient: status |
New |
Incomplete |
|
