Unit tests fail with ConfigNotFound: Could not find config at $state_path/etc/nova/api-paste.ini

[Scott Devoid]

Steps to replicate:
- Ubuntu 10.04 running devstack
- cd to /opt/stack/nova
- Check out nova at master (7093b847f624300dcc9452063187c1da735e0c85)
- tox -e py27

Many tests fail with this error message. For example:
======================================================================
FAIL: nova.tests.integrated.test_api_samples.AdminActionsSamplesJsonTest.test_post_reset_state
tags: worker-2
----------------------------------------------------------------------
Empty attachments:
  stdout

pythonlogging:'': {{{
AUDIT [nova.service] Starting conductor node (version 2014.1)
INFO [nova.virt.driver] Loading compute driver 'nova.virt.fake.FakeDriver'
AUDIT [nova.service] Starting compute node (version 2014.1)
AUDIT [nova.compute.resource_tracker] Auditing locally available compute resources
AUDIT [nova.compute.resource_tracker] Free ram (MB): 7680
AUDIT [nova.compute.resource_tracker] Free disk (GB): 1028
AUDIT [nova.compute.resource_tracker] Free VCPUS: 1
INFO [nova.compute.resource_tracker] Compute_service record created for 5fdeebe84e494959ad0e057ec33d5079:fake-mini
AUDIT [nova.service] Starting cert node (version 2014.1)
AUDIT [nova.service] Starting consoleauth node (version 2014.1)
INFO [nova.network.driver] Loading network driver 'nova.network.linux_net'
AUDIT [nova.service] Starting network node (version 2014.1)
AUDIT [nova.service] Starting scheduler node (version 2014.1)
WARNING [nova.cells.manager] The cells feature of Nova is considered experimental by the OpenStack project because it receives much less testing than the rest of Nova. This may change in the future, but current deployers should be aware that the use of it in production right now may be risky.
AUDIT [nova.service] Starting cells node (version 2014.1)
}}}

stderr: {{{
nova/servicegroup/api.py:78: DeprecationWarning: object.__init__() takes no parameters
  super(API, self).__init__(*args, **kwargs)
nova/servicegroup/api.py:78: DeprecationWarning: object.__init__() takes no parameters
  super(API, self).__init__(*args, **kwargs)
nova/servicegroup/api.py:78: DeprecationWarning: object.__init__() takes no parameters
  super(API, self).__init__(*args, **kwargs)
nova/servicegroup/api.py:78: DeprecationWarning: object.__init__() takes no parameters
  super(API, self).__init__(*args, **kwargs)
nova/servicegroup/api.py:78: DeprecationWarning: object.__init__() takes no parameters
  super(API, self).__init__(*args, **kwargs)
nova/servicegroup/api.py:78: DeprecationWarning: object.__init__() takes no parameters
  super(API, self).__init__(*args, **kwargs)
nova/servicegroup/api.py:78: DeprecationWarning: object.__init__() takes no parameters
  super(API, self).__init__(*args, **kwargs)
}}}

Traceback (most recent call last):
  File "nova/tests/integrated/test_api_samples.py", line 1832, in setUp
    super(AdminActionsSamplesJsonTest, self).setUp()
  File "nova/tests/integrated/test_api_samples.py", line 96, in setUp
    super(ApiSampleTestBaseV2, self).setUp()
  File "nova/tests/integrated/integrated_helpers.py", line 80, in setUp
    self._start_api_service()
  File "nova/tests/integrated/integrated_helpers.py", line 105, in _start_api_service
    self.osapi = service.WSGIService("osapi_compute")
  File "nova/service.py", line 349, in __init__
    self.loader = loader or wsgi.Loader()
  File "nova/wsgi.py", line 489, in __init__
    raise exception.ConfigNotFound(path=config_path)
ConfigNotFound: Could not find config at $state_path/etc/nova/api-paste.ini
======================================================================

[Juan Manuel Ollé (juan-m-olle)]

Changing in the tox.ini file solve the problem for me

[testenv]
sitepackages = False

I am pleased to make the change if this doesn't impact jenkins

[Sabari Murugesan (smurugesan)]

@Juan: Can you post a patch ? It will be nice to run the tests in virtual env without inheriting site packages.

[Juan Manuel Ollé (juan-m-olle)]

https://review.openstack.org/#/c/75162/

[Mike Spreitzer (mike-spreitzer)]

I get this problem too, in Ubuntu 13.10 and 12.04

[Mike Spreitzer (mike-spreitzer)]

This is among the problems I reported in http://lists.openstack.org/pipermail/openstack-dev/2014-February/027454.html

[Ben Nemec (bnemec)]

Seeing this on Fedora 20 too. Need to figure this out as it's blocking me getting anything done on the oslo sync. :-(

[Ben Nemec (bnemec)]

Okay, the reason this works with site-packages disabled is that then it pulls down oslo.config from pypi, which is older than the one installed by devstack. I git bisected the problem down to this commit: https://github.com/openstack/oslo.config/commit/2422d4118c97734067ea0b37ae159bc2e3c492c5

It appears that a legitimate bug crept in with that change, but I'm flummoxed as to how this passed the integration tests in the gate. The Nova unit tests should be blowing up _any_ time they are running with an oslo.config newer than that commit. :-/

[Ben Nemec (bnemec)]

Apparently the unit tests in the gate just install whatever is specified in (test-)requirements.txt, so that explains why this isn't failing in the gate. The unit tests there are getting the equivalent behavior of site-packages=False.

[Ben Nemec (bnemec)]

Also, since the Gerrit bot doesn't seem to have noticed, https://review.openstack.org/#/c/77256/ is the fix for this in oslo.config. If you pull that down locally it should allow you to run with site-packages again.

[Ben Nemec (bnemec)]

This wasn't a bug in Nova.