nova live snapshot of rbd instance fails on xen hypervisor

Bug #1656242 reported by on 2017-01-13
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)

Bug Description

We use a Mitaka environment with one control and three compute nodes (all running on openSUSE Leap 42.1), the compute nodes are xen hypervisors, our storage backend is ceph (for nova, glance and cinder).

When we try to snapshot a running instance, it's always a cold snapshot, nova-compute reports:

2017-01-12 12:55:51.919 [instance: 14b75237-7619-481f-9636-792b64d1be17] Beginning cold snapshot process
2017-01-12 12:59:27.085 [instance: 14b75237-7619-481f-9636-792b64d1be17] Snapshot image upload complete

On rbd level the live snapshot process works as expected, without any downtime of the instance, we use it for our backup strategy for example.

With some additional log statements in /usr/lib/python2.7/site-packages/nova/virt/libvirt/ I found that nova always passes hard coded hypervisor-driver "qemu" into the function _host.has_min_version(), it always returns "false" so that "live_snapshot" is disabled. Replacing host.HV_DRIVER_QEMU with host.HV_DRIVER_XEN results in a working live snapshot:

---cut here---
compute1:~ # diff -u /usr/lib/python2.7/site-packages/nova/virt/libvirt/ /usr/lib/python2.7/site-packages/nova/virt/libvirt/
--- /usr/lib/python2.7/site-packages/nova/virt/libvirt/ 2017-01-13 09:33:23.257525708 +0100
+++ /usr/lib/python2.7/site-packages/nova/virt/libvirt/ 2017-01-13 09:33:46.349105366 +0100
@@ -1649,9 +1649,14 @@
         # redundant because LVM supports only cold snapshots.
         # It is necessary in case this situation changes in the
         # future.
+ if CONF.libvirt.virt_type == 'xen':
+ hv_driver = host.HV_DRIVER_XEN
+ else:
+ hv_driver = host.HV_DRIVER_QEMU
         if (self._host.has_min_version(MIN_LIBVIRT_LIVESNAPSHOT_VERSION,
+ hv_driver)
              and source_type not in ('lvm')
              and not CONF.ephemeral_storage_encryption.enabled
              and not CONF.workarounds.disable_libvirt_livesnapshot):
---cut here---

nova-compute reports:

2017-01-12 17:20:22.760 [instance: 14b75237-7619-481f-9636-792b64d1be17] instance snapshotting
2017-01-12 17:20:24.049 [instance: 14b75237-7619-481f-9636-792b64d1be17] Beginning live snapshot process
2017-01-12 17:24:38.997 [instance: 14b75237-7619-481f-9636-792b64d1be17] Snapshot image upload complete

The versions we use:

compute1:~ # nova --version

compute1:~ # ceph --version
ceph version 0.94.7-84-g8e6f430 (8e6f430683e4d8293e31fd4eb6cb09be96960cfa)

compute1:~ # libvirtd --version
libvirtd (libvirt) 2.5.0

compute1:~ # qemu-img --version
qemu-img version 2.7.0((SUSE Linux)), Copyright (c) 2003-2016 Fabrice Bellard and the QEMU Project developers

compute1:~ # rpm -qa | grep xen

Matt Riedemann (mriedem) wrote :

What is the value of the "virt_type" config option in nova.conf on the compute? That defaults to 'kvm' for the libvirt driver, but if you're using xen with the libvirt driver then virt_type=xen in nova.conf in the [libvirt] group.

Changed in nova:
status: New → Incomplete (eblock) wrote :

Yes, that's what we use:

compute1:~ # grep virt_type /etc/nova/nova.conf
virt_type = xen

I just use the virt_type value as a workaround, there's probably a smarter way to enable live snapshots with xen, but this seems to work for me at the moment.

Launchpad Janitor (janitor) wrote :

[Expired for OpenStack Compute (nova) because there has been no activity for 60 days.]

Changed in nova:
status: Incomplete → Expired (eblock) wrote :

There's still no improvement after upgrading to Leap 42.2. I still have to use my little patch to enable live snapshots on rbd based images.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers