nova-compute fails to start if quantum is down

[Aaron Rosen]

If starting nova-compute and quantum is down nova-compute fails with:
2013-05-31 10:37:21.015 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/service.py", line 155, in start
2013-05-31 10:37:21.015 TRACE nova.openstack.common.threadgroup self.manager.init_host()
2013-05-31 10:37:21.015 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/compute/manager.py", line 632, in init_host
2013-05-31 10:37:21.015 TRACE nova.openstack.common.threadgroup self._init_instance(context, instance)
2013-05-31 10:37:21.015 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/compute/manager.py", line 519, in _init_instance
2013-05-31 10:37:21.015 TRACE nova.openstack.common.threadgroup net_info = self._get_instance_nw_info(context, instance)
2013-05-31 10:37:21.015 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/compute/manager.py", line 725, in _get_instance_nw_info
2013-05-31 10:37:21.015 TRACE nova.openstack.common.threadgroup instance, conductor_api=self.conductor_api)
2013-05-31 10:37:21.015 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/network/quantumv2/api.py", line 369, in get_instance_nw_info
2013-05-31 10:37:21.015 TRACE nova.openstack.common.threadgroup result = self._get_instance_nw_info(context, instance, networks)
2013-05-31 10:37:21.015 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/network/quantumv2/api.py", line 377, in _get_instance_nw_info
2013-05-31 10:37:21.015 TRACE nova.openstack.common.threadgroup nw_info = self._build_network_info_model(context, instance, networks)
2013-05-31 10:37:21.015 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/network/quantumv2/api.py", line 839, in _build_network_info_model
2013-05-31 10:37:21.015 TRACE nova.openstack.common.threadgroup data = client.list_ports(**search_opts)
2013-05-31 10:37:21.015 TRACE nova.openstack.common.threadgroup File "/opt/stack/python-quantumclient/quantumclient/v2_0/client.py", line 108, in with_params
2013-05-31 10:37:21.015 TRACE nova.openstack.common.threadgroup ret = self.function(instance, *args, **kwargs)
2013-05-31 10:37:21.015 TRACE nova.openstack.common.threadgroup File "/opt/stack/python-quantumclient/quantumclient/v2_0/client.py", line 262, in list_ports
2013-05-31 10:37:21.015 TRACE nova.openstack.common.threadgroup **_params)
2013-05-31 10:37:21.015 TRACE nova.openstack.common.threadgroup File "/opt/stack/python-quantumclient/quantumclient/v2_0/client.py", line 1021, in list
2013-05-31 10:37:21.015 TRACE nova.openstack.common.threadgroup for r in self._pagination(collection, path, **params):
2013-05-31 10:37:21.015 TRACE nova.openstack.common.threadgroup File "/opt/stack/python-quantumclient/quantumclient/v2_0/client.py", line 1034, in _pagination
2013-05-31 10:37:21.015 TRACE nova.openstack.common.threadgroup res = self.get(path, params=params)
2013-05-31 10:37:21.015 TRACE nova.openstack.common.threadgroup File "/opt/stack/python-quantumclient/quantumclient/v2_0/client.py", line 1007, in get
2013-05-31 10:37:21.015 TRACE nova.openstack.common.threadgroup headers=headers, params=params)
2013-05-31 10:37:21.015 TRACE nova.openstack.common.threadgroup File "/opt/stack/python-quantumclient/quantumclient/v2_0/client.py", line 992, in retry_request
2013-05-31 10:37:21.015 TRACE nova.openstack.common.threadgroup headers=headers, params=params)
2013-05-31 10:37:21.015 TRACE nova.openstack.common.threadgroup File "/opt/stack/python-quantumclient/quantumclient/v2_0/client.py", line 934, in do_request
2013-05-31 10:37:21.015 TRACE nova.openstack.common.threadgroup self._handle_fault_response(status_code, replybody)
2013-05-31 10:37:21.015 TRACE nova.openstack.common.threadgroup File "/opt/stack/python-quantumclient/quantumclient/v2_0/client.py", line 908, in _handle_fault_response
2013-05-31 10:37:21.015 TRACE nova.openstack.common.threadgroup exception_handler_v20(status_code, des_error_body)
2013-05-31 10:37:21.015 TRACE nova.openstack.common.threadgroup File "/opt/stack/python-quantumclient/quantumclient/v2_0/client.py", line 88, in exception_handler_v20
2013-05-31 10:37:21.015 TRACE nova.openstack.common.threadgroup message=message)
2013-05-31 10:37:21.015 TRACE nova.openstack.common.threadgroup QuantumClientException: [Errno 111] ECONNREFUSED

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/31250

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/grizzly
Review: https://review.openstack.org/32777

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/31250
Committed: http://github.com/openstack/nova/commit/6fda12bb14ab91592910f9ebbde9ef9f8d588b08
Submitter: Jenkins
Branch: master

commit 6fda12bb14ab91592910f9ebbde9ef9f8d588b08
Author: Aaron Rosen <email address hidden>
Date: Fri May 31 16:17:58 2013 -0700

    Fix nova-compute fails to start if quantum is down

    When nova-compute starts it needs to check the state of instances that
    were previously running and needs to query quantum. Previously, if
    it queried quantum and quantum raised an error or was unreachable
    nova-compute would terminate. This patch adds a retry loop to prevent
    this from happening.

    Fixes bug 1186357

    Change-Id: I0d865e464f7bf60a0b6dc4ca8a42436d16e29790

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/32777
Committed: http://github.com/openstack/nova/commit/60774bd58b7ab9e95e0ca71acb2f473dea9e1a67
Submitter: Jenkins
Branch: stable/grizzly

commit 60774bd58b7ab9e95e0ca71acb2f473dea9e1a67
Author: Aaron Rosen <email address hidden>
Date: Fri May 31 16:17:58 2013 -0700

    Fix nova-compute fails to start if quantum is down

    When nova-compute starts it needs to check the state of instances that
    were previously running and needs to query quantum. Previously, if
    it queried quantum and quantum raised an error or was unreachable
    nova-compute would terminate. This patch adds a retry loop to prevent
    this from happening.

    Fixes bug 1186357

    Change-Id: I0d865e464f7bf60a0b6dc4ca8a42436d16e29790
    (cherry picked from commit 6fda12bb14ab91592910f9ebbde9ef9f8d588b08)