check_exit_code TypeError: argument of type 'bool' is not iterable

[Dan Prince]

Using nova trunk revision ed1b801 with Libvirt.

I'm now getting the following exception when booting an instance:

==> /var/log/nova/nova-api.log <==

2011-12-15 02:27:21,591 ERROR nova.api.openstack.v2 [23d65bde-8e94-41ef-8011-a44abbe19209 1 1] Caught error: Instance 654f3c5a-5782-4764-8e10-4f57167ce883 in vm_state building. Cannot wrapped_f while the instance is in this state.
(nova.api.openstack.v2): TRACE: Traceback (most recent call last):
(nova.api.openstack.v2): TRACE: File "/usr/lib/python2.7/dist-packages/nova/api/openstack/v2/__init__.py", line 61, in __call__
(nova.api.openstack.v2): TRACE: return req.get_response(self.application)
(nova.api.openstack.v2): TRACE: File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1053, in get_response
(nova.api.openstack.v2): TRACE: application, catch_exc_info=False)
(nova.api.openstack.v2): TRACE: File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1022, in call_application
(nova.api.openstack.v2): TRACE: app_iter = application(self.environ, start_response)
(nova.api.openstack.v2): TRACE: File "/usr/lib/python2.7/dist-packages/keystone/middleware/auth_token.py", line 316, in __call__
(nova.api.openstack.v2): TRACE: return self._forward_request(env, start_response, proxy_headers)
(nova.api.openstack.v2): TRACE: File "/usr/lib/python2.7/dist-packages/keystone/middleware/auth_token.py", line 499, in _forward_request
(nova.api.openstack.v2): TRACE: return self.app(env, start_response)
(nova.api.openstack.v2): TRACE: File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 159, in __call__
(nova.api.openstack.v2): TRACE: return resp(environ, start_response)
(nova.api.openstack.v2): TRACE: File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 159, in __call__
(nova.api.openstack.v2): TRACE: return resp(environ, start_response)
(nova.api.openstack.v2): TRACE: File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 147, in __call__
(nova.api.openstack.v2): TRACE: resp = self.call_func(req, *args, **self.kwargs)
(nova.api.openstack.v2): TRACE: File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 208, in call_func
(nova.api.openstack.v2): TRACE: return self.func(req, *args, **kwargs)
(nova.api.openstack.v2): TRACE: File "/usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py", line 499, in __call__
(nova.api.openstack.v2): TRACE: response = req.get_response(self.application)
(nova.api.openstack.v2): TRACE: File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1053, in get_response
(nova.api.openstack.v2): TRACE: application, catch_exc_info=False)
(nova.api.openstack.v2): TRACE: File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1022, in call_application
(nova.api.openstack.v2): TRACE: app_iter = application(self.environ, start_response)
(nova.api.openstack.v2): TRACE: File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 159, in __call__
(nova.api.openstack.v2): TRACE: return resp(environ, start_response)
(nova.api.openstack.v2): TRACE: File "/usr/lib/pymodules/python2.7/routes/middleware.py", line 131, in __call__
(nova.api.openstack.v2): TRACE: response = self.app(environ, start_response)
(nova.api.openstack.v2): TRACE: File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 159, in __call__
(nova.api.openstack.v2): TRACE: return resp(environ, start_response)
(nova.api.openstack.v2): TRACE: File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 159, in __call__
(nova.api.openstack.v2): TRACE: return resp(environ, start_response)
(nova.api.openstack.v2): TRACE: File "/usr/lib/pymodules/python2.7/routes/middleware.py", line 131, in __call__
(nova.api.openstack.v2): TRACE: response = self.app(environ, start_response)
(nova.api.openstack.v2): TRACE: File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 159, in __call__
(nova.api.openstack.v2): TRACE: return resp(environ, start_response)
(nova.api.openstack.v2): TRACE: File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 147, in __call__
(nova.api.openstack.v2): TRACE: resp = self.call_func(req, *args, **self.kwargs)
(nova.api.openstack.v2): TRACE: File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 208, in call_func
(nova.api.openstack.v2): TRACE: return self.func(req, *args, **kwargs)
(nova.api.openstack.v2): TRACE: File "/usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py", line 567, in __call__
(nova.api.openstack.v2): TRACE: action_result = self.dispatch(request, action, args)
(nova.api.openstack.v2): TRACE: File "/usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py", line 599, in dispatch
(nova.api.openstack.v2): TRACE: return controller_method(req=request, **action_args)
(nova.api.openstack.v2): TRACE: File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 43, in new_f
(nova.api.openstack.v2): TRACE: ret = f(*args, **kwargs)
(nova.api.openstack.v2): TRACE: File "/usr/lib/python2.7/dist-packages/nova/scheduler/api.py", line 400, in new_f
(nova.api.openstack.v2): TRACE: return f(*args, **kwargs)
(nova.api.openstack.v2): TRACE: File "/usr/lib/python2.7/dist-packages/nova/api/openstack/v2/servers.py", line 693, in delete
(nova.api.openstack.v2): TRACE: self._delete(req.environ['nova.context'], id)
(nova.api.openstack.v2): TRACE: File "/usr/lib/python2.7/dist-packages/nova/api/openstack/v2/servers.py", line 465, in _delete
(nova.api.openstack.v2): TRACE: self.compute_api.delete(context, instance)
(nova.api.openstack.v2): TRACE: File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 75, in inner
(nova.api.openstack.v2): TRACE: method=f.__name__)
(nova.api.openstack.v2): TRACE: InstanceInvalidState: Instance 654f3c5a-5782-4764-8e10-4f57167ce883 in vm_state building. Cannot wrapped_f while the instance is in this state.
(nova.api.openstack.v2): TRACE:

[Dan Prince (dan-prince)]

More information in nova-network.log:

(nova): TRACE: Traceback (most recent call last):
(nova): TRACE: File "/usr/bin/nova-network", line 49, in <module>
(nova): TRACE: service.wait()
(nova): TRACE: File "/usr/lib/python2.7/dist-packages/nova/service.py", line 393, in wait
(nova): TRACE: _launcher.wait()
(nova): TRACE: File "/usr/lib/python2.7/dist-packages/nova/service.py", line 113, in wait
(nova): TRACE: service.wait()
(nova): TRACE: File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 166, in wait
(nova): TRACE: return self._exit_event.wait()
(nova): TRACE: File "/usr/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait
(nova): TRACE: return hubs.get_hub().switch()
(nova): TRACE: File "/usr/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 177, in switch
(nova): TRACE: return self.greenlet.switch()
(nova): TRACE: File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 192, in main
(nova): TRACE: result = function(*args, **kwargs)
(nova): TRACE: File "/usr/lib/python2.7/dist-packages/nova/service.py", line 83, in run_server
(nova): TRACE: server.start()
(nova): TRACE: File "/usr/lib/python2.7/dist-packages/nova/service.py", line 143, in start
(nova): TRACE: self.manager.init_host()
(nova): TRACE: File "/usr/lib/python2.7/dist-packages/nova/network/manager.py", line 1183, in init_host
(nova): TRACE: self.driver.ensure_metadata_ip()
(nova): TRACE: File "/usr/lib/python2.7/dist-packages/nova/network/linux_net.py", line 445, in ensure_metadata_ip
(nova): TRACE: run_as_root=True, check_exit_code=False)
(nova): TRACE: File "/usr/lib/python2.7/dist-packages/nova/network/linux_net.py", line 783, in _execute
(nova): TRACE: return utils.execute(*cmd, **kwargs)
(nova): TRACE: File "/usr/lib/python2.7/dist-packages/nova/utils.py", line 211, in execute
(nova): TRACE: if _returncode not in check_exit_code:
(nova): TRACE: TypeError: argument of type 'bool' is not iterable
(nova): TRACE:

----

Looks like commit ed1b801a2cc3691cdab6ca2ae95f7299a964d00f is the cause here. (utils.execute was changed there)

[Devin Carlen (devcamcar)]

This is a critical issue that needs to be backported to essex2.

[Thierry Carrez (ttx)]

This actually doesn't currently affect essex-2, but will if we backport the fix for bug 887402.

[Openstack Gerrit (openstack-gerrit)]

Reviewed: https://review.openstack.org/2351
Committed: http://github.com/openstack/nova/commit/de815b7f4426134c93ce5910b3401bf036df211d
Submitter: Jenkins
Branch: master

 status fixcommitted
 done

commit de815b7f4426134c93ce5910b3401bf036df211d
Author: Dan Prince <email address hidden>
Date: Wed Dec 14 22:16:25 2011 -0500

    Update utils.execute so that check_exit_code handles booleans.
    Fixes LP bug #904560.

    Change-Id: I5d09b65dade166763981dc6b248034d86ba217f1

[Openstack Gerrit (openstack-gerrit)]

Reviewed: https://review.openstack.org/2365
Committed: http://github.com/openstack/nova/commit/44955a1937ae25514bfe4ee5ccb561642681ab1c
Submitter: Jenkins
Branch: milestone-proposed

 tag in-milestone-proposed
 done

commit 44955a1937ae25514bfe4ee5ccb561642681ab1c
Author: Dean Troyer <email address hidden>
Date: Tue Dec 13 19:05:21 2011 -0600

    Fixes bug 887402

    Change utils.execute to accept a list or a single integer in
    check_exit_code.

    In libvirt.disconnect_volume() return codes 0 and 255 are both valid
    for logout/delete, where 255 is returned if the volume is already
    disconnected.

    Also includes a fix by Dan Prince for a regression introduced by
    the above fix (bug 904560)

    Change-Id: Icffeb0fe8269a02d95ac6ed180ba0bb9f458a6ed