test_driver_spawn_fail_when_unshelving_instance fluctuates

[Balazs Gibizer]

The functional test test_driver_spawn_fail_when_unshelving_instance fluctuates at the following place:
Captured traceback:
2017-10-04 17:29:33.464455 | ~~~~~~~~~~~~~~~~~~~
2017-10-04 17:29:33.464483 | b'Traceback (most recent call last):'
2017-10-04 17:29:33.464577 | b' File "/home/jenkins/workspace/gate-nova-tox-functional-py35-ubuntu-xenial/nova/tests/functional/test_servers.py", line 2664, in test_driver_spawn_fail_when_unshelving_instance'
2017-10-04 17:29:33.464618 | b" {'vcpus': 0, 'ram': 0, 'disk': 0}, usages)"
2017-10-04 17:29:33.464695 | b' File "/home/jenkins/workspace/gate-nova-tox-functional-py35-ubuntu-xenial/nova/tests/functional/test_servers.py", line 1117, in assertFlavorMatchesAllocation'
2017-10-04 17:29:33.464733 | b" self.assertEqual(flavor['vcpus'], allocation['VCPU'])"
2017-10-04 17:29:33.464816 | b' File "/home/jenkins/workspace/gate-nova-tox-functional-py35-ubuntu-xenial/.tox/functional-py35/lib/python3.5/site-packages/testtools/testcase.py", line 411, in assertEqual'
2017-10-04 17:29:33.464849 | b' self.assertThat(observed, matcher, message)'
2017-10-04 17:29:33.464931 | b' File "/home/jenkins/workspace/gate-nova-tox-functional-py35-ubuntu-xenial/.tox/functional-py35/lib/python3.5/site-packages/testtools/testcase.py", line 498, in assertThat'
2017-10-04 17:29:33.464953 | b' raise mismatch_error'
2017-10-04 17:29:33.464985 | b'testtools.matchers._impl.MismatchError: 0 != 1'
2017-10-04 17:29:33.464998 | b''
2017-10-04 17:29:33.465008 |

It is because the test waits for the instance state to be set to SHELVED_OFFLOADED and then asserts that the allocation of the instance is deleted in Placement. But the compute/manager set the instance state _before_ it deletes that allocation so the test is racy.

[1] http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22in%20test_driver_spawn_fail_when_unshelving_instance%5C%22

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/509759

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/509818

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/509759
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=b5dca17f74b7a789e16b78178ae5874a30f3e0da
Submitter: Jenkins
Branch: master

commit b5dca17f74b7a789e16b78178ae5874a30f3e0da
Author: Balazs Gibizer <email address hidden>
Date: Thu Oct 5 14:01:35 2017 +0200

    fix unstable shelve offload functional tests

    The functional tests that are shelved offloaded instances and
    asserted that the resource allocation of the instance are
    freed were unstable. These tests only waited for the instance
    state to become SHELVED_OFFLOADED before checked the allocations.
    However the compute manager sets the instance state to
    SHELVED_OFFLOADED before deleting the allocations[1]. Therefore these
    tests were racy.

    With this patch the test will wait not only for the instance status to
    change but also for the instance host to be nulled as that happens
    after the resources are freed.

    [1] https://github.com/openstack/nova/blob/e4f89ed5dd4259188d020749fa8fb1c77be2c03a/nova/compute/manager.py#L4502-L4521

    Change-Id: Ibb90571907cafcb649284e4ea30810a307f1737e
    Closes-Bug: #1721514

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/509818
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=888482c9b4fb8a1bc23d4758301e319b7c5bc997
Submitter: Zuul
Branch: stable/pike

commit 888482c9b4fb8a1bc23d4758301e319b7c5bc997
Author: Balazs Gibizer <email address hidden>
Date: Thu Oct 5 14:01:35 2017 +0200

    fix unstable shelve offload functional tests

    The functional tests that are shelved offloaded instances and
    asserted that the resource allocation of the instance are
    freed were unstable. These tests only waited for the instance
    state to become SHELVED_OFFLOADED before checked the allocations.
    However the compute manager sets the instance state to
    SHELVED_OFFLOADED before deleting the allocations[1]. Therefore these
    tests were racy.

    With this patch the test will wait not only for the instance status to
    change but also for the instance host to be nulled as that happens
    after the resources are freed.

    [1] https://github.com/openstack/nova/blob/e4f89ed5dd4259188d020749fa8fb1c77be2c03a/nova/compute/manager.py#L4502-L4521

    Change-Id: Ibb90571907cafcb649284e4ea30810a307f1737e
    Closes-Bug: #1721514
    (cherry picked from commit b5dca17f74b7a789e16b78178ae5874a30f3e0da)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 17.0.0.0b1 development milestone.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 16.0.2 release.