live migration (non-block-migration) with shared instance storage and config drive fails

Bug #1351002 reported by Solly Ross
40
This bug affects 7 people
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
High
Dmitry Borodaenko

Bug Description

Note: the reproduction case below has been fixed by not blocking migration on config drives. However, the underlying issue of NFS not being marked
as shared storage still stands, since the 'is_shared_block_storage' data
is used elsewhere as well.

---------------------------------------------------------------------------

To reproduce:

1. Set up shared instance storage via NFS and use one of the file-based image backends
2. Boot an instance with a config drive
3. Attempt to live migrate said instance w/o doing a block migration

The issue is caused by the following lines in nova/virt/libvirt/driver.py:
        if not (is_shared_instance_path and is_shared_block_storage):
            # NOTE(mikal): live migration of instances using config drive is
            # not supported because of a bug in libvirt (read only devices
            # are not copied by libvirt). See bug/1246201
            if configdrive.required_by(instance):
                raise exception.NoLiveMigrationForConfigDriveInLibVirt()

The issue, I believe, was caused by commit bc45c56f1, which separated checks for shared instance directories and shared block storage backends like Ceph. The issue is that if a deployer is not using Ceph, the call to self.image_backend.backend().is_shared_block_storage() returns False. However, is_shared_block_storage should not even be considered if the image backend is a file-based one.

Tags: libvirt
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/111082

Changed in nova:
assignee: nobody → Dmitry Borodaenko (dborodaenko)
status: New → In Progress
Changed in nova:
assignee: Dmitry Borodaenko (dborodaenko) → Solly Ross (sross-7)
Changed in nova:
assignee: Solly Ross (sross-7) → Dmitry Borodaenko (dborodaenko)
Changed in nova:
assignee: Dmitry Borodaenko (dborodaenko) → Solly Ross (sross-7)
Changed in nova:
assignee: Solly Ross (sross-7) → Dmitry Borodaenko (dborodaenko)
assignee: Dmitry Borodaenko (dborodaenko) → Solly Ross (sross-7)
Changed in nova:
assignee: Solly Ross (sross-7) → Dmitry Borodaenko (dborodaenko)
Changed in nova:
assignee: Dmitry Borodaenko (dborodaenko) → Solly Ross (sross-7)
Revision history for this message
Tushar Patil (tpatil) wrote :

Solly:
I couldn't reproduce this problem with the latest master code.
It seems like this issue got fixed in LP Bug [1], Patch [2].

[1] : LP Bug: https://bugs.launchpad.net/nova/+bug/1246201
[2] : Merged Commit: https://review.openstack.org/#/c/112014

Revision history for this message
Solly Ross (sross-7) wrote :

@tpatil: The symptom was fixed by the bug and patch you referenced. However, the underlying issue, which is that NFS is not marked as shared storage in certain places, is fixed by the proposed patch above.

I'll update the main description.

description: updated
Changed in nova:
assignee: Solly Ross (sross-7) → Dmitry Borodaenko (dborodaenko)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/111082
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=d96fbffcab5fafe48f4a91ea6f0a24310d63d9c7
Submitter: Jenkins
Branch: master

commit d96fbffcab5fafe48f4a91ea6f0a24310d63d9c7
Author: Dmitry Borodaenko <email address hidden>
Date: Thu Jul 31 13:50:00 2014 -0700

    libvirt: support live migration with shared instances dir

    If instance path is shared between compute nodes (e.g. over NFS) and image
    backend uses files placed in the instance directory (e.g. Raw, Qcow2), live
    migration should consider block storage of the instances shared.

    Change-Id: I913c57152974562765502ab57be63ccb6dfe5208
    Closes-Bug: #1351002

Changed in nova:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in nova:
milestone: none → juno-rc1
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in nova:
milestone: juno-rc1 → 2014.2
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.