Poll rescued instances fails with key error

Bug #1277054 reported by John Garbutt
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Medium
John Garbutt

Bug Description

After an instance has been in the rescue state for some time, a periodic task triggers to unrescue the instances:
_poll_rescued_instances

File nova/notifications.py info_from_instance
  instance_type = flavors.extract_flavor(instance_ref)
File "nova/compute/flavors.py" in extract_flavor
  instance_type[key] = type_fn(sys_meta[type_key])
KeyError: 'instance_type_memory_mb'

This then continues to happen on every run of the periodic task, and starts to fill up the DB with instance faults.

Tags: compute
Changed in nova:
importance: Undecided → Medium
status: New → Triaged
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/71504

Changed in nova:
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/71504
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=ea7242a293abe59dd91add5e44e58a2eb6cd99fc
Submitter: Jenkins
Branch: master

commit ea7242a293abe59dd91add5e44e58a2eb6cd99fc
Author: John Garbutt <email address hidden>
Date: Thu Feb 6 11:09:33 2014 +0000

    Fix auto instance unrescue after poll period

    We were seeing massive amounts on instance faults from failed automatic
    unrescue. It was coming from the instance not having system metadata
    populated.

    While the route cause is a combination of using non-object instances
    that were not populated with system metadata. With an object, the system
    metadata would lazyload as required, but clearly not with a pure dict.

    This fix ensures the non-object instance has system_metadata, so the
    extraction of the flavor from the instance, during notifications sent
    during unrescue, now succeed.

    Closes-Bug: #1277054
    Change-Id: I63bdd6c49505557829241869fd3efd2b2754530b

Changed in nova:
status: In Progress → Fix Committed
Changed in nova:
milestone: none → icehouse-3
Thierry Carrez (ttx)
Changed in nova:
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in nova:
milestone: icehouse-3 → 2014.1
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.