[integration tests][nova] TimeoutExpired: Timeout of 120 seconds expired waiting for instances to became to ACTIVE status

[Georgy Dyuldin]

Test result:

https://mirantis.testrail.com/index.php?/tests/view/6075272

Trace:

os_conn = <mos_tests.environment.os_actions.OpenStackActions object at 0x7fc9fc255c10>
env = <mos_tests.environment.fuel_client.Environment object at 0x7fc9fc1d68d0>
network = {u'network': {u'tenant_id': u'4befc4f57ef04ba894f9f3381f308aed', u'created_at'...id': u'29629560-5d4f-4230-acf3-6c930fd98de4', u'name': u'net01', u'mtu': 1500}}
ubuntu_image_id = '1eba70b8-cc5e-44e8-b087-483ee1916295'
keypair = <Keypair: instancekey>
security_group = <SecurityGroup description=descr, id=3b28955b-359c-4bd6-8bb5-175b318cf6fb, name=test-sg1674959307, rules=[], tenant_id=4befc4f57ef04ba894f9f3381f308aed>
instances_cleanup = None

    @pytest.mark.testrail_id('843871')
    @pytest.mark.check_env_("is_any_compute_suitable_for_max_flavor")
    def test_metadata_reach_all_booted_vm(os_conn, env, network, ubuntu_image_id,
                                          keypair, security_group,
                                          instances_cleanup):
        """[Bug #1545043] Check that metadata reach all booted VMs

        Scenario:
            1. Create a Glance image based on Ubuntu image
            2. Boot an instance based on previously created image
            3. Check that this instance is reachable via ssh connection
            4. Delete instance
            5. Repeat pp 2-4 100 times
        """
        flavor = os_conn.nova.flavors.find(name='m1.small')
        instances_count = 0
        # Determine available m1.small instances count
        for hypervisor in os_conn.nova.hypervisors.list():
            instances_count += os_conn.get_hypervisor_capacity(hypervisor, flavor)
        iterations_count = 100 / instances_count
        for i in range(100 / instances_count):
            logger.info('Check metadata iteration {i} from {iterations_count}. '
                        'Boot {instances_count} instances'.format(
                            i=i,
                            iterations_count=iterations_count,
                            instances_count=instances_count))
            instances = []
            for j in range(instances_count):
                instance = os_conn.create_server(
                    name='ubuntu_server_{:02d}'.format(j),
                    availability_zone='nova',
                    key_name=keypair.name,
                    image_id=ubuntu_image_id,
                    flavor=flavor,
                    nics=[{'net-id': network['network']['id']}],
                    security_groups=[security_group.id],
                    wait_for_active=False,
                    wait_for_avaliable=False)
                instances.append(instance)

            common.wait(
                lambda: all(os_conn.is_server_active(x) for x in instances),
                timeout_seconds=2 * 60,
> waiting_for='instances to became to ACTIVE status')
E TimeoutExpired: Timeout of 120 seconds expired waiting for instances to became to ACTIVE status

mos_tests/nova/metadata_test.py:97: TimeoutExpired

[Anna Babich (ababich)]

The time interval for instance to get ACTIVE status depend on environment and an image where an instance is booted from. For example, on an env with virtual computes the instance booted from Ubuntu image under combination of circumstances may take longer that 120 sec. Moreover, if it goes to ACTIVE status, it doesn't mean that its operating system is loaded completely and ssh-ready

[Dina Belova (dbelova)]

More than a month in the incomplete state -> marking as invalid. Please move back to confirmed if this will be reproduced again.