Running tests via nosestests fails due to insufficient test isolation

Bug #1658200 reported by Lars Kellogg-Stedman on 2017-01-20
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
cloud-init
Medium
Unassigned

Bug Description

I don't know if we care about this or not, since it involves a test harness other than tox.

Attempting to run the unittests using nosetests (nosetets tests/unittests) will fail because the _set_mock_metadata method appears to only run once...so tests that expect non-default metadata (such as test_instance_level_keys_replace_project_level_keys) will fail if any prior test calls _set_mock_metadata().

This can be avoided by calling httppretty.reset() before calling httppretty.register_uri(...).

Related branches

Lars Kellogg-Stedman (larsks) wrote :
Download full text (6.3 KiB)

A failure looks like this:

  ......................SSSSS...............................................................................................................................................................................F
  ======================================================================
  FAIL: test_instance_level_keys_replace_project_level_keys (tests.unittests.test_datasource.test_gce.TestDataSourceGCE)
  ----------------------------------------------------------------------
  Traceback (most recent call last):
    File "/home/lars/src/cloud-init/tests/unittests/test_datasource/test_gce.py", line 150, in test_instance_level_keys_replace_project_level_keys
      self.assertEqual([key_content], self.ds.get_public_ssh_keys())
  AssertionError: Lists differ: ['ssh-rsa JustAUser root@server'] != ['ssh-rsa AA2..+aRD0fyVw== root@server']

  First differing element 0:
  ssh-rsa JustAUser root@server
  ssh-rsa AA2..+aRD0fyVw== root@server

  - ['ssh-rsa JustAUser root@server']
  + ['ssh-rsa AA2..+aRD0fyVw== root@server']
  -------------------- >> begin captured logging << --------------------
  cloudinit.url_helper: DEBUG: [0/1] open 'http://metadata.google.internal/computeMetadata/v1/instance/id' with {'url': 'http://metadata.google.internal/computeMetadata/v1/instance/id', 'allow_redirects': True, 'method': 'GET', 'headers': {'X-Google-Metadata-Request': 'True', 'User-Agent': 'Cloud-Init/0.7.9'}} configuration
  requests.packages.urllib3.connectionpool: INFO: Starting new HTTP connection (1): metadata.google.internal
  requests.packages.urllib3.connectionpool: DEBUG: "GET /computeMetadata/v1/instance/id HTTP/1.1" 200 3
  cloudinit.url_helper: DEBUG: Read from http://metadata.google.internal/computeMetadata/v1/instance/id (200, 3b) after 1 attempts
  cloudinit.url_helper: DEBUG: [0/1] open 'http://metadata.google.internal/computeMetadata/v1/instance/zone' with {'url': 'http://metadata.google.internal/computeMetadata/v1/instance/zone', 'allow_redirects': True, 'method': 'GET', 'headers': {'X-Google-Metadata-Request': 'True', 'User-Agent': 'Cloud-Init/0.7.9'}} configuration
  requests.packages.urllib3.connectionpool: INFO: Starting new HTTP connection (1): metadata.google.internal
  requests.packages.urllib3.connectionpool: DEBUG: "GET /computeMetadata/v1/instance/zone HTTP/1.1" 200 7
  cloudinit.url_helper: DEBUG: Read from http://metadata.google.internal/computeMetadata/v1/instance/zone (200, 7b) after 1 attempts
  cloudinit.url_helper: DEBUG: [0/1] open 'http://metadata.google.internal/computeMetadata/v1/instance/hostname' with {'url': 'http://metadata.google.internal/computeMetadata/v1/instance/hostname', 'allow_redirects': True, 'method': 'GET', 'headers': {'X-Google-Metadata-Request': 'True', 'User-Agent': 'Cloud-Init/0.7.9'}} configuration
  requests.packages.urllib3.connectionpool: INFO: Starting new HTTP connection (1): metadata.google.internal
  requests.packages.urllib3.connectionpool: DEBUG: "GET /computeMetadata/v1/instance/hostname HTTP/1.1" 200 24
  cloudinit.url_helper: DEBUG: Read from http://metadata.google.internal/computeMetadata/v1/instance/hostname (200, 24b) after 1 attempts
  cloudinit.url_helper: DEBUG:...

Read more...

Scott Moser (smoser) wrote :

I've opened an upstream issue
https://github.com/gabrielfalcao/HTTPretty/issues/316

I'm pretty sure that is the root of why you see it in your case and we do not when running in tox.

Scott Moser (smoser) on 2017-01-24
Changed in cloud-init:
status: New → Fix Committed
importance: Undecided → Medium

This bug is believed to be fixed in cloud-init in 17.1. If this is still a problem for you, please make a comment and set the state back to New

Thank you.

Changed in cloud-init:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.