plainbox tests fail in local CI (vagrant)

[Zygmunt Krynicki]

Using vagrant 1.7.2+dfsg-2 built from Debian git, with some local fixes (sent to the packager and reported as bugs in Debian) I can now run tests in Vagrant again.

On clean trunk plainbox has one failing test. The test fails because of pkg_resources warning that show up on precise (only on precise). I remember we patched some things for precise so that stuff like that doesn't show up but I wonder why this doesn't fail in our main CI and if anyone else has seen this.

======================================================================
FAIL: test_help (impl.commands.test_run.TestRun)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/vagrant/plainbox/plainbox/impl/commands/test_run.py", line 114, in test_help
    self.assertEqual(io.combined, cleandoc(expected) + "\n")
AssertionError: "/usr/lib/python3/dist-packages/pkg_resources.py:1756: ResourceWarning: unclosed [truncated]... != "usage: plainbox run [-h] [--non-interactive] [-n] [--dont-suppress-output]\n [truncated]...
- /usr/lib/python3/dist-packages/pkg_resources.py:1756: ResourceWarning: unclosed file <_io.TextIOWrapper name='/usr/local/lib/python3.2/dist-packages/plainbox.egg-link' mode='r' encoding='UTF-8'>
- break
- /usr/lib/python3/dist-packages/pkg_resources.py:1756: ResourceWarning: unclosed file <_io.TextIOWrapper name='/usr/local/lib/python3.2/dist-packages/checkbox-ng.egg-link' mode='r' encoding='UTF-8'>
- break
- /usr/lib/python3/dist-packages/pkg_resources.py:1756: ResourceWarning: unclosed file <_io.TextIOWrapper name='/usr/local/lib/python3.2/dist-packages/checkbox-support.egg-link' mode='r' encoding='UTF-8'>
- break
  usage: plainbox run [-h] [--non-interactive] [-n] [--dont-suppress-output]
                      [-f FORMAT] [-p OPTIONS] [-o FILE] [-t TRANSPORT]
                      [--transport-where WHERE] [--transport-options OPTIONS]
                      [-T TEST-PLAN-ID] [-i PATTERN] [-x PATTERN] [-w WHITELIST]

  optional arguments:
    -h, --help show this help message and exit

  user interface options:
    --non-interactive skip tests that require interactivity
    -n, --dry-run don't really run most jobs
    --dont-suppress-output
                          don't suppress the output of certain job plugin types

  output options:
    -f FORMAT, --output-format FORMAT
                          save test results in the specified FORMAT (pass ? for
                          a list of choices)
    -p OPTIONS, --output-options OPTIONS
                          comma-separated list of options for the export
                          mechanism (pass ? for a list of choices)
    -o FILE, --output-file FILE
                          save test results to the specified FILE (or to stdout
                          if FILE is -)
    -t TRANSPORT, --transport TRANSPORT
                          use TRANSPORT to send results somewhere (pass ? for a
                          list of choices)
    --transport-where WHERE
                          where to send data using the selected transport
    --transport-options OPTIONS
                          comma-separated list of key-value options (k=v) to be
                          passed to the transport

  test selection options:
    -T TEST-PLAN-ID, --test-plan TEST-PLAN-ID
                          load the specified test plan
    -i PATTERN, --include-pattern PATTERN
                          include jobs matching the given regular expression
    -x PATTERN, --exclude-pattern PATTERN
                          exclude jobs matching the given regular expression
    -w WHITELIST, --whitelist WHITELIST
                          load whitelist containing run patterns

----------------------------------------------------------------------
Ran 1844 tests in 26.251s

FAILED (failures=1, skipped=2, expected failures=2)

[Zygmunt Krynicki (zyga)]

I'm marking this as confirmed as it happens for me and nobody else uses vagrant to reproduce this.

[Zygmunt Krynicki (zyga)]

Ah, with some digging I see we work around this very issue in support/provision-testing-environment but the fix is only applied to LXC.