OpenStack Compute (nova)

backup fails to reset instance task_state to None

Bug #1051069 reported by Nikhil Komawar on 2012-09-14

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	OpenStack Compute (nova)	Fix Released	High	Eddie Sheffield	OpenStack Compute (nova) 2012.2 "folsom"

Bug Description

when we try to create a backup for an instance, the instance task_state is stuck at image_backup and never resets to None thus failing the call

the compute manager logs look like this:

2012-09-14 20:36:22 TRACE nova.openstack.common.rpc.amqp Traceback (most recent call last):
2012-09-14 20:36:22 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/openstack/common/rpc/amqp.py", line 275, in _process_data
2012-09-14 20:36:22 TRACE nova.openstack.common.rpc.amqp rval = self.proxy.dispatch(ctxt, version, method, **args)
2012-09-14 20:36:22 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/openstack/common/rpc/dispatcher.py", line 145, in dispatch
2012-09-14 20:36:22 TRACE nova.openstack.common.rpc.amqp return getattr(proxyobj, method)(ctxt, **kwargs)
2012-09-14 20:36:22 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/exception.py", line 117, in wrapped
2012-09-14 20:36:22 TRACE nova.openstack.common.rpc.amqp temp_level, payload)
2012-09-14 20:36:22 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
2012-09-14 20:36:22 TRACE nova.openstack.common.rpc.amqp self.gen.next()
2012-09-14 20:36:22 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/exception.py", line 92, in wrapped
2012-09-14 20:36:22 TRACE nova.openstack.common.rpc.amqp return f(*args, **kw)
2012-09-14 20:36:22 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/compute/manager.py", line 167, in decorated_function
2012-09-14 20:36:22 TRACE nova.openstack.common.rpc.amqp return function(self, context, *args, **kwargs)
2012-09-14 20:36:22 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/compute/manager.py", line 202, in decorated_function
2012-09-14 20:36:22 TRACE nova.openstack.common.rpc.amqp kwargs['instance']['uuid'], e, sys.exc_info())
2012-09-14 20:36:22 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
2012-09-14 20:36:22 TRACE nova.openstack.common.rpc.amqp self.gen.next()
2012-09-14 20:36:22 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/compute/manager.py", line 196, in decorated_function
2012-09-14 20:36:22 TRACE nova.openstack.common.rpc.amqp return function(self, context, *args, **kwargs)
2012-09-14 20:36:22 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/compute/manager.py", line 1143, in snapshot_instance
2012-09-14 20:36:22 TRACE nova.openstack.common.rpc.amqp expected_task_state=task_states.IMAGE_SNAPSHOT)
2012-09-14 20:36:22 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/compute/manager.py", line 254, in _instance_update
2012-09-14 20:36:22 TRACE nova.openstack.common.rpc.amqp context, instance_uuid, kwargs)
2012-09-14 20:36:22 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/db/api.py", line 675, in instance_update_and_get_original
2012-09-14 20:36:22 TRACE nova.openstack.common.rpc.amqp values)
2012-09-14 20:36:22 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 130, in wrapper
2012-09-14 20:36:22 TRACE nova.openstack.common.rpc.amqp return f(*args, **kwargs)
2012-09-14 20:36:22 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 1768, in instance_update_and_get_original
2012-09-14 20:36:22 TRACE nova.openstack.common.rpc.amqp copy_old_instance=True)
2012-09-14 20:36:22 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 1788, in _instance_update
2012-09-14 20:36:22 TRACE nova.openstack.common.rpc.amqp expected=expected)
2012-09-14 20:36:22 TRACE nova.openstack.common.rpc.amqp UnexpectedTaskStateError: unexpected task state: expecting ('image_snapshot',) but the actual state is image_backup
2012-09-14 20:36:22 TRACE nova.openstack.common.rpc.amqp

See original description

Nikhil Komawar (nikhil-komawar) on 2012-09-14

Changed in nova:
status:	New → Confirmed
description:	updated

Eddie Sheffield (eddie-sheffield) on 2012-09-14

Changed in nova:
assignee:	nobody → Eddie Sheffield (eddie-sheffield)

Dan Prince (dan-prince) on 2012-09-15

Changed in nova:
importance:	Undecided → High

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2012-09-17: Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/13152

Changed in nova:
status:	Confirmed → In Progress

Vish Ishaya (vishvananda) on 2012-09-17

Changed in nova:
milestone:	none → folsom-rc1

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2012-09-17: Fix merged to nova (master)

Reviewed: https://review.openstack.org/13152
Committed: http://github.com/openstack/nova/commit/73e8d5a7b668e3a9700101b2fcd742ed349e0b26
Submitter: Jenkins
Branch: master

commit 73e8d5a7b668e3a9700101b2fcd742ed349e0b26
Author: eddie-sheffield <email address hidden>
Date: Mon Sep 17 11:10:06 2012 -0400

Reset the task state after backup done.

    The compute manager needs to reset the task state after a backup
    is complete. However it was only correctly reseting after a
    snapshot due to the current expected state being hardcoded.
    Now looks for the correct expected state depending on whether a
    backup or snapshot was being executed.

Fixes bug 1051069

Change-Id: Ie91f7745b98232370be6c99d318a8033a93cc50c