test_resize_server_error_and_reschedule_was_failed failing due to missing notification

Bug #1717917 reported by Balazs Gibizer on 2017-09-18
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Undecided
Balazs Gibizer

Bug Description

The test_resize_server_error_and_reschedule_was_failed case was failed in jenkins couple of times[1]. It seems that the test only wait for the instance to go to ERROR state but compute.exception notification emitted after that which make the test racy.

[1] http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22testtools.matchers._impl.MismatchError%3A%202%20!%3D%201%3A%20Unexpected%20number%20of%20notifications%3A%5C%22

description: updated
tags: added: notifications testing
Changed in nova:
assignee: nobody → Balazs Gibizer (balazs-gibizer)

Fix proposed to branch: master
Review: https://review.openstack.org/504930

Changed in nova:
status: New → In Progress

Reviewed: https://review.openstack.org/504930
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=0262e4146d63a0d7b3fbb1942a57a7a136820aec
Submitter: Jenkins
Branch: master

commit 0262e4146d63a0d7b3fbb1942a57a7a136820aec
Author: Balazs Gibizer <email address hidden>
Date: Mon Sep 18 14:33:13 2017 +0200

    stabilize test_resize_server_error_and_reschedule_was_failed

    The test_resize_server_error_and_reschedule_was_failed notifiation sample
    test only waits for the instance to go to ERROR state and then asserts that
    two notifications are emitted. However the second notification,
    compute.exception, only emited after the instance is put into ERROR state.
    This makes the test unstable.

    This patch makes the test stable by waiting for the compute.exception
    notification to arrive before asserting the received notifications.

    Change-Id: Ia5311ffc12784987c138b127e43cfc52019cb3ea
    Closes-Bug: #1717917

Changed in nova:
status: In Progress → Fix Released
Changed in nova:
status: Fix Released → In Progress

Reviewed: https://review.openstack.org/509750
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=265ed23bb20c8d860ec49daeff2e9c8eea1b57f0
Submitter: Jenkins
Branch: master

commit 265ed23bb20c8d860ec49daeff2e9c8eea1b57f0
Author: Balazs Gibizer <email address hidden>
Date: Thu Oct 5 11:56:56 2017 +0200

    stabilize test_resize_server_error_and_reschedule_was_failed

    The Ia5311ffc12784987c138b127e43cfc52019cb3ea patch tried to fix a fluctuation
    in the test_resize_server_error_and_reschedule_was_failed functional test.
    But the test still fluctuates even after that patch is merged[1]. Looking back
    to the origin patch the problem is obvious and embarrassing. The extra
    _wait_for_notification call was introduced _after_ the assert that checks
    the number of received notifications. So the original patch never worked.

    This patch moves the extra wait call _before_ the assert.

    [1] http://logs.openstack.org/55/483955/9/check/gate-nova-tox-functional-ubuntu-xenial/209d046/console.html#_2017-10-04_21_52_53_807624

    Change-Id: I4c104842cbde587dc9c67f5fde84140252cdcdb5
    Closes-Bug: #1717917

Changed in nova:
status: In Progress → Fix Released

This issue was fixed in the openstack/nova 17.0.0.0b1 development milestone.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers