[integration tests] Timeout during many instaces deleting on live-migration tests

[Georgy Dyuldin]

Test result:

https://mirantis.testrail.com/index.php?/tests/view/6048961

Trace:

self = <mos_tests.nova.live_migration_test.TestLiveMigration object at 0x7f0050621e10>

    @pytest.yield_fixture
    def cleanup_instances(self):
        yield
> self.delete_instances(force=True)

mos_tests/nova/live_migration_test.py:241:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <mos_tests.nova.live_migration_test.TestLiveMigration object at 0x7f0050621e10>
force = True

    def delete_instances(self, force=False):
        for instance in self.instances:
            if force:
                delete = instance.force_delete
            else:
                delete = instance.delete
            try:
                delete()
            except nova_exceptions.NotFound:
                pass
        common.wait(
            lambda: all(self.os_conn.is_server_deleted(x.id)
                        for x in self.instances),
            timeout_seconds=2 * 60,
> waiting_for='instances to be deleted')
E TimeoutExpired: Timeout of 120 seconds expired waiting for instances to be deleted

mos_tests/nova/live_migration_test.py:235: TimeoutExpired

[Anna Babich (ababich)]

It's not clear from test result's log, what could happen with instances that they couldn't be deleted, even force-deleted.

timeout_seconds=2 * 60 - is this value for single instance or for all booted during the test?

[Dina Belova (dbelova)]

More than a month for the incomplete state -> moving to invalid. Please mark back to confirmed if this will be reproduced again