Jenkins job builder py27 tests fails when ran locally

[Philippe Godin]

Jenkins-job-builder py27 tests fails from a fresh clone of master when ran locally. Pep8, pyflakes and py26 tests are OK.
The error is when the output xml is compared to the expected version. During the test run, the xml tags and values are outputted on separate lines which make the string comparison fail.

Example:

- <protocol>HTTP</protocol>
+ <protocol>
+ HTML
+ </protocol>

[Khai Do (zaro0508)]

I cannot reproduce the test failures. Could you please provide the exact steps you used to run the test? When I run I do the following..
1. clone jenkins-job-builder
2. run 'tox -epy27'
Tests all pass for me.

Here is a list of dependent libs in my .tox/py27 ENV. Could you please check to see if you have the same?
Jinja2==2.7.2
MarkupSafe==0.18
PyYAML==3.10
Pygments==1.6
Sphinx==1.1.3
discover==0.4.0
docutils==0.11
extras==0.0.3
fixtures==0.3.14
jenkins-job-builder==0.6.0.64.ga32288a
python-jenkins==0.2.1
python-mimeparse==0.1.4
python-subunit==0.0.18
setuptools-git==1.0
sphinxcontrib-programoutput==0.8
testrepository==0.0.18
testscenarios==0.4
testtools==0.9.35
wsgiref==0.1.2

[Philippe Godin (godp1301)]

Khai,

The steps you have listed are correct. Simply clone and run tox or tox -e py27.
About the environment, i'm using python 2.7.2 on mac osx maverics. Activating the created tox py27 environment and running pip freeze gives me the exact list of libraries and versions you have specified in your comments.

TL;DR
~$ git clone https://github.com/openstack-infra/jenkins-job-builder.git
Cloning into 'jenkins-job-builder'...
remote: Reusing existing pack: 3627, done.
remote: Counting objects: 49, done.
remote: Compressing objects: 100% (48/48), done.
remote: Total 3676 (delta 24), reused 11 (delta 0)
Receiving objects: 100% (3676/3676), 1.13 MiB | 1.13 MiB/s, done.
Resolving deltas: 100% (2106/2106), done.
~$ cd jenkins-job-builder/
toRemote information is outdated. Running git fetch.
Fetching origin
∓ (master) ~/jenkins-job-builder$ tox -e py27
GLOB sdist-make: /Users/pgodin/jenkins-job-builder/setup.py
py27 create: /Users/pgodin/jenkins-job-builder/.tox/py27
py27 installdeps: -r/Users/pgodin/jenkins-job-builder/tools/pip-requires, -r/Users/pgodin/jenkins-job-builder/tools/test-requires
py27 inst: /Users/pgodin/jenkins-job-builder/.tox/dist/jenkins-job-builder-0.6.0.64.ga32288a.zip
py27 runtests: commands[0] | python setup.py testr --slowest --testr-args=
running testr
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} OS_LOG_CAPTURE=${OS_LOG_CAPTURE:-1} ${PYTHON:-python} -m subunit.run discover -t ./ tests --list
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} OS_LOG_CAPTURE=${OS_LOG_CAPTURE:-1} ${PYTHON:-python} -m subunit.run discover -t ./ tests --load-list /var/folders/48/95vvh2352nz1_g27cpjsf5mw3_zf4q/T/tmpmf4JWe
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} OS_LOG_CAPTURE=${OS_LOG_CAPTURE:-1} ${PYTHON:-python} -m subunit.run discover -t ./ tests --load-list /var/folders/48/95vvh2352nz1_g27cpjsf5mw3_zf4q/T/tmpDUR17q
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} OS_LOG_CAPTURE=${OS_LOG_CAPTURE:-1} ${PYTHON:-python} -m subunit.run discover -t ./ tests --load-list /var/folders/48/95vvh2352nz1_g27cpjsf5mw3_zf4q/T/tmpf6WwKG
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} OS_LOG_CAPTURE=${OS_LOG_CAPTURE:-1} ${PYTHON:-python} -m subunit.run discover -t ./ tests --load-list /var/folders/48/95vvh2352nz1_g27cpjsf5mw3_zf4q/T/tmpalfHvk
======================================================================
FAIL: tests.notifications.test_notifications.TestCaseModuleNotifications.test_yaml_snippet(http-endpoint001.yaml)
tags: worker-0
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/pgodin/jenkins-job-builder/tests/base.py", line 112, in test_yaml_snippet
    doctest.REPORT_NDIFF)
  File "/Users/pgodin/jenkins-job-builder/.tox/py27/lib/python2.7/site-packages/testtools/testcase.py", line 406, in assertThat
    raise mismatch_error
MismatchError: Differences (ndiff with -expected +actual):
      <?xml version="1.0" encoding="utf-8"?>
      <project>
        <prop...

[Khai Do (zaro0508)]

I am also on osx mavericks using python 2.7.5. I tried running on ubuntu precise with python 2.7.3. I could not reproduce the error on either system. I'm wondering if it could be a compatibility issue with a python lib outside of the tox environment. I've attached a file containing my global python libs, maybe you can take a look to see if there are any obvious differences that might cause the failure. I would at least try getting tox==1.6.1 to see if that fixes it for you.

[Clark Boylan (cboylan)]

Could be a difference in the underlying python xml lib. We ran into that with python26 as well. Perhaps the tests should compare the xml object created by JJB and the deserialized XML from the test fixtures instead of directly comparing strings? That should give us better results regardless of string formatting.

[Sergey Lukjanov (slukjanov)]

It's a python issue: http://bugs.python.org/issue4147 it was fixed in Python >= 2.7.3

Patch: http://hg.python.org/cpython/rev/cb6614e3438b/
Detailed blog post about it and different options about how to avoid this problem: http://ronrothman.com/public/leftbraned/xml-dom-minidom-toprettyxml-and-silly-whitespace/

In Savanna we're patching toprettyxml: https://github.com/openstack/savanna/blob/master/savanna/utils/patches.py#L17

[Philippe Godin (godp1301)]

I have verified Sergey's comments. I was not able to reproduce this bug using python 2.7.3 on the same machine.