notification failure in _sync_power_states

[Andrew Laski]

The _sync_power_states periodic task pull instances without system_metadata in order to reduce network bandwidth being unnecessarily consumed. Most of the time this is fine, but "if vm_power_state != db_power_state" then the instance is updated and saved. As part of saving the instance a notification is sent. In order to send the notification it extracts flavor information from the system_metadata on the instance. But system_metadata isn't loaded, and won't be lazy loaded. So an exception is raised and the notification isn't sent.

2013-10-23 03:30:35.714 21492 ERROR nova.notifications [-] [instance: fa0cee4b-6825-47af-bf6f-64491326feab] Failed to send state update notification
2013-10-23 03:30:35.714 21492 TRACE nova.notifications [instance: fa0cee4b-6825-47af-bf6f-64491326feab] Traceback (most recent call last):
2013-10-23 03:30:35.714 21492 TRACE nova.notifications [instance: fa0cee4b-6825-47af-bf6f-64491326feab] File "/opt/rackstack/472.23/nova/lib/python2.6/site-packages/nova/notifications.py", line 146, in send_update
2013-10-23 03:30:35.714 21492 TRACE nova.notifications [instance: fa0cee4b-6825-47af-bf6f-64491326feab] old_display_name=old_display_name)
2013-10-23 03:30:35.714 21492 TRACE nova.notifications [instance: fa0cee4b-6825-47af-bf6f-64491326feab] File "/opt/rackstack/472.23/nova/lib/python2.6/site-packages/nova/notifications.py", line 199, in _send_instance_update_notification
2013-10-23 03:30:35.714 21492 TRACE nova.notifications [instance: fa0cee4b-6825-47af-bf6f-64491326feab] payload = info_from_instance(context, instance, None, None)
2013-10-23 03:30:35.714 21492 TRACE nova.notifications [instance: fa0cee4b-6825-47af-bf6f-64491326feab] File "/opt/rackstack/472.23/nova/lib/python2.6/site-packages/nova/notifications.py", line 343, in info_from_instance
2013-10-23 03:30:35.714 21492 TRACE nova.notifications [instance: fa0cee4b-6825-47af-bf6f-64491326feab] instance_type = flavors.extract_flavor(instance_ref)
2013-10-23 03:30:35.714 21492 TRACE nova.notifications [instance: fa0cee4b-6825-47af-bf6f-64491326feab] File "/opt/rackstack/472.23/nova/lib/python2.6/site-packages/nova/compute/flavors.py", line 282, in extract_flavor
2013-10-23 03:30:35.714 21492 TRACE nova.notifications [instance: fa0cee4b-6825-47af-bf6f-64491326feab] instance_type[key] = type_fn(sys_meta[type_key])
2013-10-23 03:30:35.714 21492 TRACE nova.notifications [instance: fa0cee4b-6825-47af-bf6f-64491326feab] KeyError: 'instance_type_memory_mb'
2013-10-23 03:30:35.714 21492 TRACE nova.notifications [instance: fa0cee4b-6825-47af-bf6f-64491326feab]
2013-10-23 03:30:35.718 21492 WARNING nova.compute.manager [-] [instance: fa0cee4b-6825-47af-bf6f-64491326feab] Instance shutdown by itself. Calling the stop API.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/53708
Committed: http://github.com/openstack/nova/commit/e03963b1e42ca2900b108201e77edf2386c5c46a
Submitter: Jenkins
Branch: master

commit e03963b1e42ca2900b108201e77edf2386c5c46a
Author: Andrew Laski <email address hidden>
Date: Thu Oct 24 13:48:24 2013 -0400

    Pull system_metadata for notifications on instance.save()

    When saving changes on an instance object a notification is sent out.
    An attempt is made to pull flavor information from the instance but if
    system_metadata is not joined then an exception is raised and the
    notification fails. This ensures that system_metadata is available for
    those notifications.

    Change-Id: Ifd317f847c3839cd6019038bb3dc856b9f107000
    Closes-bug: 1244311

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/havana
Review: https://review.openstack.org/54863

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/54863
Committed: http://github.com/openstack/nova/commit/e360dc2adcf18cfdc2b39cbea708bb27bf741e2d
Submitter: Jenkins
Branch: stable/havana

commit e360dc2adcf18cfdc2b39cbea708bb27bf741e2d
Author: Andrew Laski <email address hidden>
Date: Thu Oct 24 13:48:24 2013 -0400

    Pull system_metadata for notifications on instance.save()

    When saving changes on an instance object a notification is sent out.
    An attempt is made to pull flavor information from the instance but if
    system_metadata is not joined then an exception is raised and the
    notification fails. This ensures that system_metadata is available for
    those notifications.

    Conflicts:

     nova/objects/base.py

    The conflict is from change Icd2944f17b6100d51007dbc48da90e4e992bbd48

    Change-Id: Ifd317f847c3839cd6019038bb3dc856b9f107000
    Closes-bug: 1244311
    (cherry picked from commit e03963b1e42ca2900b108201e77edf2386c5c46a)