[integration tests][nova] TimeoutExpired: Timeout of 60 seconds expired waiting for volume to be deleted

Bug #1584776 reported by Georgy Dyuldin
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Mirantis OpenStack
Invalid
High
Georgy Dyuldin

Bug Description

Test result:

https://mirantis.testrail.com/index.php?/tests/view/6075224

Trace:

cls = <mos_tests.nova.nova_test.TestNovaDeferredDelete object at 0x7fc9fc255950>
os_conn = <mos_tests.environment.os_actions.OpenStackActions object at 0x7fc9fd3313d0>

    @classmethod
    @pytest.yield_fixture
    def volumes(cls, os_conn):
        """Volumes cleanUp"""
        volumes = []
        yield volumes
        for volume in volumes:
> common_functions.delete_volume(os_conn.cinder, volume)

mos_tests/nova/nova_test.py:625:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

cinder_client = <cinderclient.v2.client.Client object at 0x7fc9fcebeb50>
volume = <Volume: dbe45541-bd83-4653-b842-0665c11106a4>

    def delete_volume(cinder_client, volume):
        """Delete volume and check that it is absent in the list
            :param cinder_client: Cinder API client connection point
            :param volume: volume
        """
        if volume in cinder_client.volumes.list():
            volume.get()
            if len(volume.attachments) > 0:
                volume.detach()
            cinder_client.volumes.delete(volume)
            volume_id = volume.id
            wait(lambda: not is_volume_exists(cinder_client, volume_id),
                 timeout_seconds=60,
> waiting_for='volume to be deleted')
E TimeoutExpired: Timeout of 60 seconds expired waiting for volume to be deleted

mos_tests/functions/common.py:461: TimeoutExpired

Tags: area-qa
Changed in mos:
milestone: 10.0 → 9.0
Revision history for this message
Anna Babich (ababich) wrote :

As we can see from http://cz7776.bud.mirantis.net:8080/jenkins/job/9.0_Neutron_NovaCeph/2/testReport/mos_tests.nova.nova_test/TestNovaDeferredDelete/test_restore_deleted_instance_86400___842493__/ , it's a cleanup issue, not related to the test scenario.
Anyway, I've tried to reproduce it on env (Cinder Ceph + Nova Ceph):
[root@nailgun ~]# shotgun2 short-report
cat /etc/fuel_build_id:
 398
cat /etc/fuel_build_number:
 398
cat /etc/fuel_release:
 9.0
cat /etc/fuel_openstack_version:
 mitaka-9.0
rpm -qa | egrep 'fuel|astute|network-checker|nailgun|packetary|shotgun':
 fuel-release-9.0.0-1.mos6346.noarch
 fuel-bootstrap-cli-9.0.0-1.mos282.noarch
 fuel-migrate-9.0.0-1.mos8386.noarch
 rubygem-astute-9.0.0-1.mos746.noarch
 fuel-misc-9.0.0-1.mos8386.noarch
 network-checker-9.0.0-1.mos72.x86_64
 fuel-mirror-9.0.0-1.mos136.noarch
 fuel-openstack-metadata-9.0.0-1.mos8693.noarch
 fuel-notify-9.0.0-1.mos8386.noarch
 nailgun-mcagents-9.0.0-1.mos746.noarch
 fuel-provisioning-scripts-9.0.0-1.mos8693.noarch
 python-fuelclient-9.0.0-1.mos315.noarch
 fuelmenu-9.0.0-1.mos270.noarch
 fuel-9.0.0-1.mos6346.noarch
 fuel-utils-9.0.0-1.mos8386.noarch
 fuel-setup-9.0.0-1.mos6346.noarch
 fuel-library9.0-9.0.0-1.mos8386.noarch
 shotgun-9.0.0-1.mos89.noarch
 fuel-agent-9.0.0-1.mos282.noarch
 fuel-ui-9.0.0-1.mos2699.noarch
 fuel-ostf-9.0.0-1.mos934.noarch
 python-packetary-9.0.0-1.mos136.noarch
 fuel-nailgun-9.0.0-1.mos8693.noarch

and the results are: http://paste.openstack.org/show/506315/ , the volume has been detached and deleted successfully.

Also, I have a question concerning:
def delete_volume(cinder_client, volume):
        """Delete volume and check that it is absent in the list
            :param cinder_client: Cinder API client connection point
            :param volume: volume
        """
        if volume in cinder_client.volumes.list():
            volume.get()
            if len(volume.attachments) > 0:
                volume.detach()
            cinder_client.volumes.delete(volume)
            volume_id = volume.id
            wait(lambda: not is_volume_exists(cinder_client, volume_id),
                 timeout_seconds=60,
> waiting_for='volume to be deleted')

we can see that detach() method is called within cinder-client, but in fact it's Nova's action (nova-attach, nova-detach), and I'm not sure now that it's cinder-client's one as well. Is the nova-client enabled earlier for these operations?

Changed in mos:
assignee: MOS QA Team (mos-qa) → Georgy Dyuldin (g-dyuldin)
status: Confirmed → Incomplete
Revision history for this message
Dina Belova (dbelova) wrote :

More than a month in the incomplete state -> moving to invalid. Please mark back to confirmed if this will be reproduced again.

Changed in mos:
status: Incomplete → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.