Libvirt pre-Grizzly instances cannot be block migrated

Bug #1185588 reported by Rafi Khardalian
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Medium
Rafi Khardalian
Grizzly
Fix Released
Medium
Rafi Khardalian

Bug Description

This impacts Grizzly and master.

Any instances created prior to Grizzly will fail to block migrate. Prior to Grizzly, the default image structure was to convert all base images to RAW, then resize said image to match that of the selected flavor. Thus, the _base directory would contain <image_hash> and <image_hash>_<size_gb>. From there, QCOW2 files would be created, based on the resized backing file. The entire notion of resized base files was eliminated with Grizzly (rightfully so) and with it the code which would handle creating these resized backing files.

The problem is that many environments still have instances created with this backing structure. When utilizing live block migrations (nova live-migration --block-migrate), we rely on the destination hypervisor to re-populate _base images from Glance. Therein lies the problem, we do not have the code to detect or generate <image_hash>_<size_gb> base images. Thus, the block live migration will fail. The same situation applies if a _base image happens to go missing. We added code in Grizzly to re-download/generate missing _base images upon instance reboot, which cannot currently succeed due to the same issue.

Revision history for this message
Rafi Khardalian (rkhardalian) wrote :

I'll be submitting a patch for this shortly.

Changed in nova:
assignee: nobody → Rafi Khardalian (rkhardalian)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/30951

Changed in nova:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/30951
Committed: http://github.com/openstack/nova/commit/6e768363450774cfee90e41aa5c40af780d3e04a
Submitter: Jenkins
Branch: master

commit 6e768363450774cfee90e41aa5c40af780d3e04a
Author: Rafi Khardalian <email address hidden>
Date: Wed May 29 21:26:19 2013 +0000

    Regenerate missing resized backing files

    Fixes bug 1185588

    The only way to reliably determine whether we are using a legacy
    (pre-Grizzly) backing structure is to look at the backing file
    reference for the instance/disk itself. Thus, if we determine the
    backing file contains an underscore, we use the value following it
    as our legacy_backing_size, which is then used to create the
    necessary backing file.

    Change-Id: I145f71068c972a604e5361ae79d6315286136786

Changed in nova:
status: In Progress → Fix Committed
tags: added: grizzly-backport-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/grizzly)

Fix proposed to branch: stable/grizzly
Review: https://review.openstack.org/32767

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/33007

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/grizzly)

Reviewed: https://review.openstack.org/32767
Committed: http://github.com/openstack/nova/commit/69dd7c50a95518788018dd1cb8aad60029fdac42
Submitter: Jenkins
Branch: stable/grizzly

commit 69dd7c50a95518788018dd1cb8aad60029fdac42
Author: Rafi Khardalian <email address hidden>
Date: Wed May 29 21:26:19 2013 +0000

    Regenerate missing resized backing files

    Fixes bug 1185588 (for stable/grizzly)

    The only way to reliably determine whether we are using a legacy
    (pre-Grizzly) backing structure is to look at the backing file
    reference for the instance/disk itself. Thus, if we determine the
    backing file contains an underscore, we use the value following it
    as our legacy_backing_size, which is then used to create the
    necessary backing file.

    Change-Id: I145f71068c972a604e5361ae79d6315286136786
    Cherry-Pick: 6e768363450774cfee90e41aa5c40af780d3e04a

tags: added: in-stable-grizzly
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/33007
Committed: http://github.com/openstack/nova/commit/cf3cf3c8e5137e775eec9939e88aa1df339ba1bf
Submitter: Jenkins
Branch: master

commit cf3cf3c8e5137e775eec9939e88aa1df339ba1bf
Author: Rafi Khardalian <email address hidden>
Date: Fri Jun 14 06:47:57 2013 +0000

    libvirt fix resize/migrates with swap or ephemeral

    Fixes bug 1191608

    The code was not properly guarding against backing files not
    existing with QCOW2 images. As such, get_disk_backing_file() would
    return None, then the subsequent call to basename() on None would
    cause an exception.

    This bug was introduced with the fix for bug 1185588 and commit
    6e768363.

    Change-Id: I4b1237ddf01b9398bfdc141180260136b36cae5a

Thierry Carrez (ttx)
Changed in nova:
milestone: none → havana-2
status: Fix Committed → Fix Released
Alan Pevec (apevec)
tags: removed: grizzly-backport-potential in-stable-grizzly
Changed in nova:
importance: Undecided → Medium
Thierry Carrez (ttx)
Changed in nova:
milestone: havana-2 → 2013.2
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.