Fix the APIError typo

Bug #788550 reported by Édouard Thuleau
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Medium
vivek.ys

Bug Description

I use Nova rev1113 with VMware hypervisor.
I try to use pause/unpause command with OpenStack API, but VMware hypervisor doesn't support it, so it sends an APIError exception :

2011-05-26 12:03:49,137 INFO nova.compute.manager [TH1YE1-Y5KER904YBL1Y user1 simple] check_instance_lock: decorating: |<function pause_instance at 0x26da050>|
2011-05-26 12:03:49,138 INFO nova.compute.manager [TH1YE1-Y5KER904YBL1Y user1 simple] check_instance_lock: arguments: |<nova.compute.manager.ComputeManager object at 0x25ced50>| |<nova.context.RequestContext object at 0xbf71090>| |29|
2011-05-26 12:03:49,202 INFO nova.compute.manager [TH1YE1-Y5KER904YBL1Y user1 simple] check_instance_lock: locked: |False|
2011-05-26 12:03:49,202 INFO nova.compute.manager [TH1YE1-Y5KER904YBL1Y user1 simple] check_instance_lock: admin: |True|
2011-05-26 12:03:49,202 INFO nova.compute.manager [TH1YE1-Y5KER904YBL1Y user1 simple] check_instance_lock: executing: |<function pause_instance at 0x26da050>|
2011-05-26 12:03:49,261 AUDIT nova.compute.manager [TH1YE1-Y5KER904YBL1Y user1 simple] instance 29: pausing
2011-05-26 12:03:49,809 ERROR nova.exception [-] Uncaught exception
(nova.exception): TRACE: Traceback (most recent call last):
(nova.exception): TRACE: File "/usr/lib/pymodules/python2.6/nova/exception.py", line 87, in _wrap
(nova.exception): TRACE: return f(*args, **kw)
(nova.exception): TRACE: File "/usr/lib/pymodules/python2.6/nova/compute/manager.py", line 105, in decorated_function
(nova.exception): TRACE: function(self, context, instance_id, *args, **kwargs)
(nova.exception): TRACE: File "/usr/lib/pymodules/python2.6/nova/compute/manager.py", line 669, in pause_instance
(nova.exception): TRACE: lambda result: self._update_state_callback(self,
(nova.exception): TRACE: File "/usr/lib/pymodules/python2.6/nova/virt/vmwareapi_conn.py", line 145, in pause
(nova.exception): TRACE: self._vmops.pause(instance, callback)
(nova.exception): TRACE: File "/usr/lib/pymodules/python2.6/nova/virt/vmwareapi/vmops.py", line 593, in pause
(nova.exception): TRACE: raise exception.APIError("pause not supported for vmwareapi")
(nova.exception): TRACE: AttributeError: 'module' object has no attribute 'APIError'
(nova.exception): TRACE:
2011-05-26 12:03:49,810 ERROR nova [-] Exception during message handling
(nova): TRACE: Traceback (most recent call last):
(nova): TRACE: File "/usr/lib/pymodules/python2.6/nova/rpc.py", line 198, in _receive
(nova): TRACE: rval = node_func(context=ctxt, **node_args)
(nova): TRACE: File "/usr/lib/pymodules/python2.6/nova/exception.py", line 93, in _wrap
(nova): TRACE: raise Error(str(e))
(nova): TRACE: Error: 'module' object has no attribute 'APIError'
(nova): TRACE:

The instance still running (ping or ssh OK) but the instance status is pausing/BUILD for Nova:

$ nova list
+----+-----------+--------+-----------+------------+
| ID | Name | Status | Public IP | Private IP |
+----+-----------+--------+-----------+------------+
| 29 | Server 29 | BUILD | | 172.16.1.4 |
+----+-----------+--------+-----------+------------+

$ euca-describe-instances
RESERVATION r-srnudn54 simple default
INSTANCE i-0000001d ami-00000001 172.16.1.4 172.16.1.4 pausing key (simple, p-novavmware) 0 m1.small 2011-05-26T08:59:16Z nova

In the same tenant, if an instance is running and if we try to resume it, nova-compute raise a InstanceResumeFailure exception :

2011-05-26 12:17:17,837 INFO nova.compute.manager [2WRZ55G6B8H60I3C33QL user1 simple] check_instance_lock: decorating: |<function resume_instance at 0x26da578>|
2011-05-26 12:17:17,838 INFO nova.compute.manager [2WRZ55G6B8H60I3C33QL user1 simple] check_instance_lock: arguments: |<nova.compute.manager.ComputeManager object at 0x25ced50>| |<nova.context.RequestContext object at 0x141fa990>| |29|
2011-05-26 12:17:17,901 INFO nova.compute.manager [2WRZ55G6B8H60I3C33QL user1 simple] check_instance_lock: locked: |False|
2011-05-26 12:17:17,901 INFO nova.compute.manager [2WRZ55G6B8H60I3C33QL user1 simple] check_instance_lock: admin: |True|
2011-05-26 12:17:17,902 INFO nova.compute.manager [2WRZ55G6B8H60I3C33QL user1 simple] check_instance_lock: executing: |<function resume_instance at 0x26da578>|
2011-05-26 12:17:17,962 AUDIT nova.compute.manager [2WRZ55G6B8H60I3C33QL user1 simple] instance 29: resuming
2011-05-26 12:17:18,361 ERROR nova.exception [-] Uncaught exception
(nova.exception): TRACE: Traceback (most recent call last):
(nova.exception): TRACE: File "/usr/lib/pymodules/python2.6/nova/exception.py", line 87, in _wrap
(nova.exception): TRACE: return f(*args, **kw)
(nova.exception): TRACE: File "/usr/lib/pymodules/python2.6/nova/compute/manager.py", line 105, in decorated_function
(nova.exception): TRACE: function(self, context, instance_id, *args, **kwargs)
(nova.exception): TRACE: File "/usr/lib/pymodules/python2.6/nova/compute/manager.py", line 728, in resume_instance
(nova.exception): TRACE: lambda result: self._update_state_callback(self,
(nova.exception): TRACE: File "/usr/lib/pymodules/python2.6/nova/virt/vmwareapi_conn.py", line 157, in resume
(nova.exception): TRACE: self._vmops.resume(instance, callback)
(nova.exception): TRACE: File "/usr/lib/pymodules/python2.6/nova/virt/vmwareapi/vmops.py", line 641, in resume
(nova.exception): TRACE: raise exception.InstanceResumeFailure(reason=reason)
(nova.exception): TRACE: InstanceResumeFailure: Failed to resume server: instance is not in a suspended state.
(nova.exception): TRACE:
2011-05-26 12:17:18,362 ERROR nova [-] Exception during message handling
(nova): TRACE: Traceback (most recent call last):
(nova): TRACE: File "/usr/lib/pymodules/python2.6/nova/rpc.py", line 198, in _receive
(nova): TRACE: rval = node_func(context=ctxt, **node_args)
(nova): TRACE: File "/usr/lib/pymodules/python2.6/nova/exception.py", line 93, in _wrap
(nova): TRACE: raise Error(str(e))
(nova): TRACE: Error: Failed to resume server: instance is not in a suspended state.

The instance still running (ping or ssh OK) but it is in state resuming/BUILD for Nova.

Related branches

Revision history for this message
Brian Waldon (bcwaldon) wrote :

The first exception referring to the APIError is actually just a typo. It should be "ApiError". Even when that is fixed, it appears we still dont support 'pause' on that hypervisor.

As for the second issue, an instance must have the status 'suspended' to be able to resume it.

Revision history for this message
Brian Waldon (bcwaldon) wrote :

So I probably just reiterated things you already knew in my last comment :) The real bug here is that the instance statuses shouldn't be changing if nothing is happening, correct?

Revision history for this message
Édouard Thuleau (ethuleau) wrote :

yes, it's what I tried to explain but my poor English limits me :)

I had just tested it with VMware hypervisor but perhaps, the problem is the same for the others.

Revision history for this message
Édouard Thuleau (ethuleau) wrote :

I try with libvirt/KVM hypervisor.
I tested a unrescue a running instance, compute log said :

2011-05-26 15:42:48,568 AUDIT nova.compute.manager [UJ9SK-JSU4BZ-FZFC5DY user1 simple] instance 30: unrescuing
2011-05-26 15:42:48,628 ERROR nova.exception [-] Uncaught exception
(nova.exception): TRACE: Traceback (most recent call last):
(nova.exception): TRACE: File "/usr/lib/pymodules/python2.6/nova/exception.py", line 87, in _wrap
(nova.exception): TRACE: return f(*args, **kw)
(nova.exception): TRACE: TypeError: unrescue() takes exactly 2 arguments (3 given)
(nova.exception): TRACE:
2011-05-26 15:42:48,628 ERROR nova [-] Exception during message handling
(nova): TRACE: Traceback (most recent call last):
(nova): TRACE: File "/usr/lib/pymodules/python2.6/nova/rpc.py", line 198, in _receive
(nova): TRACE: rval = node_func(context=ctxt, **node_args)
(nova): TRACE: File "/usr/lib/pymodules/python2.6/nova/exception.py", line 87, in _wrap
(nova): TRACE: return f(*args, **kw)
(nova): TRACE: File "/usr/lib/pymodules/python2.6/nova/compute/manager.py", line 105, in decorated_function
(nova): TRACE: function(self, context, instance_id, *args, **kwargs)
(nova): TRACE: File "/usr/lib/pymodules/python2.6/nova/compute/manager.py", line 502, in unrescue_instance
(nova): TRACE: self.driver.unrescue(instance_ref, _update_state)
(nova): TRACE: File "/usr/lib/pymodules/python2.6/nova/exception.py", line 93, in _wrap
(nova): TRACE: raise Error(str(e))
(nova): TRACE: Error: unrescue() takes exactly 2 arguments (3 given)
(nova): TRACE:

The instance passed to status 'unrescue' for few seconds and after logs :

2011-05-26 15:43:24,014 INFO nova.compute.manager [-] Updating host status
2011-05-26 15:43:24,047 INFO nova.compute.manager [-] DB/VM state mismatch. Changing state from '0' to '1'

the instance came back to status 'running'.

I think there is typo error for the first exception, like you said. But nothing synchronize the db and instance status with the VMware hypervisor.

Revision history for this message
Brian Waldon (bcwaldon) wrote :

So it seems that the pause/unpause and suspend/resume functionality is not implemented in our libvirt driver either, it isn't just the vmware driver. The bigger problem here is that the compute manager isn't handling error cases when its driver fails to complete an action (such as pause, unpause, etc).

This bug might be more useful as "Fix the APIError typo" and maybe file another bug to address "Poor driver exception handling in compute manager" with your examples from above.

Revision history for this message
Édouard Thuleau (ethuleau) wrote :

No, it's correctly handling by libvirt driver but not by VMware driver.

Do you want I create a new bug ?
Can I change the title of this bug ?

Revision history for this message
Brian Waldon (bcwaldon) wrote :

I think that'll work. Nobody else seems to have anything to say, so I am going to assume that is the right way to handle it.

Revision history for this message
Thierry Carrez (ttx) wrote :

@Edouard:
I will rename this bug to "Fix the APIError typo" and triage it.
Could you file a separate bug for "Poor driver exception handling in compute manager" with your examples from above ?
Thanks !

summary: - Instance status and raise compute exceptions
+ Fix the APIError typo
Changed in nova:
importance: Undecided → Medium
status: New → Triaged
tags: added: low-hanging-fruit
removed: instances status
Revision history for this message
Édouard Thuleau (ethuleau) wrote :
vivek.ys (vivekys)
Changed in nova:
assignee: nobody → vivek.ys (vivekys)
Thierry Carrez (ttx)
Changed in nova:
status: Triaged → In Progress
Changed in nova:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in nova:
milestone: none → diablo-1
Thierry Carrez (ttx)
Changed in nova:
milestone: diablo-1 → 2011.3
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.