automatic confirmation after resize fails

Bug #1152792 reported by Phil Nelson
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
High
Dan Smith

Bug Description

To enable automatic confirmation after resize/migrate, we set the value of confirm_resize_window to 1. This did in fact enable the processing for the automatic confirmation, but an error was returned because of a database access issue:

2013-03-08 12:13:42.072 93054 INFO nova.compute.manager [-] Found 1 unconfirmed migrations older than 1 seconds
2013-03-08 12:13:42.074 93054 INFO nova.compute.manager [-] Automatically confirming migration 18 for instance da9eba9d-0175-4b1c-b85f-246d7bb20c05
2013-03-08 12:13:42.074 93054 DEBUG nova.openstack.common.rpc.amqp [-] Making synchronous call on conductor ... multicall /usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py:541
2013-03-08 12:13:42.074 93054 DEBUG nova.openstack.common.rpc.amqp [-] MSG_ID is 14ef28def01b4e55aa8db832b236dfbe multicall /usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py:544
2013-03-08 12:13:42.231 93054 DEBUG nova.utils [-] Reloading cached file /etc/nova/policy.json read_cached_file /usr/lib/python2.6/site-packages/nova/utils.py:1123
2013-03-08 12:13:42.242 93054 ERROR nova.compute [-] No db access allowed in nova-compute: File "/usr/lib/python2.6/site-packages/eventlet/greenthread.py", line 192, in main
    result = function(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/nova/utils.py", line 636, in _inner
    idle = self.f(*self.args, **self.kw)
  File "/usr/lib/python2.6/site-packages/nova/service.py", line 571, in periodic_tasks
    return self.manager.periodic_tasks(ctxt, raise_on_error=raise_on_error)
  File "/usr/lib/python2.6/site-packages/nova/manager.py", line 241, in periodic_tasks
    task(self, context)
  File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 3248, in _poll_unconfirmed_resizes
    self.compute_api.confirm_resize(context, instance)
  File "/usr/lib/python2.6/site-packages/nova/compute/api.py", line 170, in wrapped
    return func(self, context, target, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/nova/compute/api.py", line 160, in inner
    return function(self, context, instance, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/nova/compute/api.py", line 141, in inner
    return f(self, context, instance, *args, **kw)
  File "/usr/lib/python2.6/site-packages/nova/compute/api.py", line 1852, in confirm_resize
    instance['uuid'], 'finished')
  File "/usr/lib/python2.6/site-packages/nova/db/api.py", line 397, in migration_get_by_instance_and_status
    status)
  File "/usr/bin/nova-compute", line 66, in __call__
    stacktrace = "".join(traceback.format_stack())

After searching the community and coming across the blueprint at https://blueprints.launchpad.net/nova/+spec/no-db-compute and the bug at https://bugs.launchpad.net/nova/+bug/823000, I started to suspect the issue could have been caused by this change. To test this theory, in /usr/bin/nova-compute we commented out lines 78 and 79 and set the parm referenced in line 83 to db_allowed=True (https://github.com/openstack/nova/blob/master/bin/nova-compute#L78). After restarting nova and trying a new resize operation, the automatic confirmation was working.

Dan Smith (danms)
Changed in nova:
assignee: nobody → Dan Smith (danms)
importance: Undecided → High
milestone: none → grizzly-rc1
status: New → Confirmed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/24090

Changed in nova:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/24090
Committed: http://github.com/openstack/nova/commit/d8d7d148fe2c185f2efda0c4cfc2ca125d4008d7
Submitter: Jenkins
Branch: master

commit d8d7d148fe2c185f2efda0c4cfc2ca125d4008d7
Author: Dan Smith <email address hidden>
Date: Mon Mar 11 13:13:11 2013 -0400

    Pass migration_ref when when auto-confirming

    The _poll_unconfirmed_resizes() task in compute/manager calls back to
    compute_api to confirm resizes. This triggers a db lookup for the
    migration, which is not allowed by no-db-compute. Since we already have
    the migration, pass it through to avoid the lookup.

    Since this doesn't change rpcapi (and thus, RPC clients should not
    know about this optional paramter), we don't need to bump the RPC
    version here.

    Fixes bug 1152792

    Change-Id: I4bd3b0c88968ed7f64e7df0afa12e83009ee8ccf

Changed in nova:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in nova:
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in nova:
milestone: grizzly-rc1 → 2013.1
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.