Exception when live block migration multiple ephemerals

Bug #1628449 reported by Sahid Orentino
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
High
Sahid Orentino
Newton
Fix Committed
High
Sahid Orentino

Bug Description

When block live migrate an instance with multiple ephemeral an exception FlavorDiskSmallerThanImage is raised.

Steps to Reproduce:
1) Created flavor with two ephemeral support.

~~~
[root@allinone9 ~(keystone_admin)]# nova flavor-create 2ephemeral-disks 6 512 1 1 --ephemeral 2
~~~

2) Spawned instance using created flavor.

~~~
[root@allinone9 ~(keystone_admin)]# nova boot --flavor 2ephemeral-disks --image cirros --ephemeral size=1 --ephemeral size=1 internal1
~~~

3) Instance spawned successfully.

~~~
[root@allinone9 ~(keystone_admin)]# nova list --field name,status,host | grep -i internal1
| 08619d2d-e3a2-4f67-a959-33cfbc08d153 | internal1 | ACTIVE | allinone9 |
~~~

4) Verifying that two extra ephemeral disks are connected with instance.

~~~
[root@allinone9 ~(keystone_admin)]# virsh domblklist 4
Target Source
------------------------------------------------
vda /var/lib/nova/instances/08619d2d-e3a2-4f67-a959-33cfbc08d153/disk
vdb /var/lib/nova/instances/08619d2d-e3a2-4f67-a959-33cfbc08d153/disk.eph0
vdc /var/lib/nova/instances/08619d2d-e3a2-4f67-a959-33cfbc08d153/disk.eph1
~~~

5) Tried to perform the block migration but it end with same error which you have seen.

[root@allinone9 ~(keystone_admin)]# nova live-migration 08619d2d-e3a2-4f67-a959-33cfbc08d153 compute1-9 --block-migrate

~~~
From : /var/log/nova/nova-compute.log

2016-09-26 08:53:12.033 3958 ERROR nova.compute.manager [req-f24d49f7-4d8e-4683-bcc0-952254764fca b09d7a1af46d42398c79a1dc0da02954 ca23990ed6c846b0b8d588fb5e304aeb - - -] [instance: 08619d2d-e3a2-4f67-a959-33cfbc08d153] Pre live migration failed at compute1-9

2016-09-26 08:53:12.033 3958 ERROR nova.compute.manager [instance: 08619d2d-e3a2-4f67-a959-33cfbc08d153]
2016-09-26 08:53:12.033 3958 ERROR nova.compute.manager [instance: 08619d2d-e3a2-4f67-a959-33cfbc08d153] FlavorDiskSmallerThanImage: Flavor's disk is too small for requested image. Flavor disk is 1073741824 bytes, image is 2147483648 bytes.
~~~

That error is because two mistake:

... LINE ~ 6588 in libvirt.py (method libvirt._create_images_and_backing)

  image = self.image_backend.image(instance,
                                          instance_disk,
                                          CONF.libvirt.images_type)
                if cache_name.startswith('ephemeral'):
                    image.cache(fetch_func=self._create_ephemeral,
    fs_label=cache_name,
    os_type=instance.os_type,
                         filename=cache_name,
                                size=info['virt_disk_size'], (a)
                                ephemeral_size=instance.flavor.ephemeral_gb) (b)
         elif cache_name.startswith('swap'):
             inst_type = instance.get_flavor()
                    swap_mb = inst_type.swap

...

(a) That argument 'size' does not exist in _create_ephemeral
(b) We should report here the actual size of the ephemeral disk (which is what has been asked by user during boot insteado of the total size allowed by the flavor for ephemeral disks)

Changed in nova:
assignee: nobody → sahid (sahid-ferdjaoui)
tags: added: newton-backport-potential
Revision history for this message
Sahid Orentino (sahid-ferdjaoui) wrote :

Actually it's little bit more complicated. Because we cache ephemeral disk which seems to be a bad idea. An ephemeral disk can have been create with 2G the first time and an other flavor can request for only 1G... That case is going to fail

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/378560

Changed in nova:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/newton)

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/382365

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/378560
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=407e659eb9c228eb1ec06ec49864279aeab0a1a1
Submitter: Jenkins
Branch: master

commit 407e659eb9c228eb1ec06ec49864279aeab0a1a1
Author: Sahid Orentino Ferdjaoui <email address hidden>
Date: Wed Sep 28 07:49:45 2016 -0400

    libvirt: fix DiskSmallerThanImage when block migrate ephemerals

    When block live migrate an instance with ephemerals an exception
    FlavorDiskSmallerThanImage can be raised because the size used to
    create the base ephemeral disk is the total size allowed by flavor
    which can be greater than the size effectively requested by the
    instance when spwaned.

    Closes-Bug: #1628449
    Change-Id: I264f5beb73d9b8ba441aec8f8a317b553a7e22c0

Changed in nova:
status: In Progress → Fix Released
Matt Riedemann (mriedem)
Changed in nova:
importance: Undecided → Medium
tags: added: libvirt live-migration
removed: newton-backport-potential
Changed in nova:
importance: Medium → High
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/newton)

Reviewed: https://review.openstack.org/382365
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=da6cb543df6692dcc0ea66f9fde1c3ff3f7999a1
Submitter: Jenkins
Branch: stable/newton

commit da6cb543df6692dcc0ea66f9fde1c3ff3f7999a1
Author: Sahid Orentino Ferdjaoui <email address hidden>
Date: Wed Sep 28 07:49:45 2016 -0400

    libvirt: fix DiskSmallerThanImage when block migrate ephemerals

    When block live migrate an instance with ephemerals an exception
    FlavorDiskSmallerThanImage can be raised because the size used to
    create the base ephemeral disk is the total size allowed by flavor
    which can be greater than the size effectively requested by the
    instance when spwaned.

    Closes-Bug: #1628449
    Change-Id: I264f5beb73d9b8ba441aec8f8a317b553a7e22c0
    (cherry picked from commit 407e659eb9c228eb1ec06ec49864279aeab0a1a1)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 14.0.1

This issue was fixed in the openstack/nova 14.0.1 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 15.0.0.0b1

This issue was fixed in the openstack/nova 15.0.0.0b1 development milestone.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.