Instance failure: Ignore notifications for certain events raised from libivrt

[Tushar Patil]

Masakari-monitors send notification for following libvirt events to masakari.

<pre>
event:LIFECYCLE
virt_domain_event:SUSPENDED_IOERROR|SUSPENDED_WATCHDOG|SUSPENDED_API_ERROR|STOPPED_SHUTDOWN|STOPPED_DESTROYED|STOPPED_FAILED|SHUTDOWN_FINISHED

event: REBOOT
virt_domain_event:UNKNOWN

event: WATCHDOG
virt_domain_event: WATCHDOG_NONE|WATCHDOG_PAUSE|WATCHDOG_RESET|WATCHDOG_POWEROFF|WATCHDOG_SHUTDOWN|WATCHDOG_DEBUG

event: IO_ERROR
virt_domain_event: IO_ERROR_NONE|IO_ERROR_PAUSE|IO_ERROR_REPORT

event: IO_ERROR_REASON
virt_domain_event: UNKNOWN

event: CONTROL_ERROR
virt_domain_event: UNKNOWN
</pre>

Out of all above events, masakari-engine should run instance recovery workflow only for the below events and for all the other remaining events it should simply log message with info log level.

<pre>

event:LIFECYCLE
virt_domain_event:STOPPED_FAILED

event: IO_ERROR
virt_domain_event: IO_ERROR_REPORT

</pre>

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/430121
Committed: https://git.openstack.org/cgit/openstack/masakari/commit/?id=74dff4b446bf280f6c3dd1dd656e6c56594d38ed
Submitter: Jenkins
Branch: master

commit 74dff4b446bf280f6c3dd1dd656e6c56594d38ed
Author: Dinesh Bhor <email address hidden>
Date: Thu Jan 19 15:22:33 2017 +0530

    InstanceFailure: Ignore notifications for certain events

    For instance failure recovery Masakari doesn't check for the event from
    payload received in notification, as a result it executes workflow for
    all the events received.

    When a instance is stopped (normal operation), masakari-instancemonitor
    sends notifications related to following two events one after another to
    masakari-api service.
        A) event: LIFECYCLE detail: STOPPED_SHUTDOWN
        B) event: LIFECYCLE detail: SHUTDOWN_FINISHED

    When masakari-engine processes the first notification, it checks for vmstate
    of the instance, if it’s 'stopped', it won't stop the vm. But if it's not
    stopped it will attempt to stop it. Then again, libvirt will send the above
    two events to masakari-instancemonitor and masakari-instancemonitor will send
    related notifications to masakari-api and this way it will go into the recursion.

    To solve this issue we can only process the below mentioned events at masakari
    side which actually requires actions to be taken:
        A) 'event': 'LIFECYCLE', 'vir_domain_event': 'STOPPED_FAILED'
        B) 'event': 'IO_ERROR', 'vir_domain_event': 'IO_ERROR_REPORT'

    So removed handling of all other events. For all other events masakari will
    simply log the message at INFO level and mark the notification status as
    'ignored'.

    Closes-Bug: #1662994
    Change-Id: If1113a599894cc7bab32b9656b6c42de5e5d7a54