AttributeError: 'virDomain' object has no attribute 'fsFreeze

[Ross Martyn]

Kilo Nova/Cinder Version 1:2015.1.2-0ubuntu2~cloud0
Ceph Version 0.94.5-1trusty

Apologies, this issue crosses multiple packages, Nova/Cinder/CEPH, I am fairly confident this issue stems from nova/libvirt/driver.py but I lack the necessary knowledge to dig much further.

ERROR oslo_messaging._drivers.common [req-2d6b8f15-fb08-4091-8456-e96f99410815 3f3836f15ed443e89fffe1b102649db2 36ce305340dd40e9a342b23aeb8f171b - - -] ['Traceback (most recent call last):\n', ' File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply\n executor_callback))\n', ' File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch\n executor_callback)\n', ' File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 130, in _do_dispatch\n result = func(ctxt, **new_args)\n', ' File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 7002, in quiesce_instance\n return self.manager.quiesce_instance(ctxt, instance)\n', ' File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 142, in inner\n return func(*args, **kwargs)\n', ' File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 88, in wrapped\n payload)\n', ' File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__\n six.reraise(self.type_, self.value, self.tb)\n', ' File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 71, in wrapped\n return f(self, context, *args, **kw)\n', ' File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 6658, in quiesce_instance\n self.driver.quiesce(context, instance, image_meta)\n', ' File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1522, in quiesce\n self._set_quiesced(context, instance, image_meta, True)\n', ' File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1506, in _set_quiesced\n domain.fsFreeze()\n', ' File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 176, in __getattr__\n f = getattr(self._obj, attr_name)\n', "AttributeError: 'virDomain' object has no attribute 'fsFreeze'\n"]

Steps to reproduce this error:

1) Create Image with metadata, as recommended by : http://docs.ceph.com/docs/master/rbd/rbd-openstack/

hw_scsi_model=virtio-scsi: add the virtio-scsi controller and get better performance and support for discard operation
hw_disk_bus=scsi: connect every cinder block devices to that controller
hw_qemu_guest_agent=yes: enable the QEMU guest agent
os_require_quiesce=yes: send fs-freeze/thaw calls through the QEMU guest agent

2) Upload image with Qemu Guest Agent installed.

3) Create an instance with a Cinder Volume.

4) Snapshot that instance whilst the instance is running.

Note: A forced snapshot of the volume through cinder will succeed, as will the snapshot if the require_quiesce is disabled. This may result in an inconsistent snapshot.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[sean redmond (sean-redmond1)]

Is this because the version of python-libvirt does not seem to come from the ubuntu cloud archive repo?

# dpkg -l | grep -i libvirt
ii libvirt-bin 1.2.16-2ubuntu11.15.10.4~cloud0 amd64 programs for the libvirt library
ii libvirt0 1.2.16-2ubuntu11.15.10.4~cloud0 amd64 library for interfacing with different virtualization systems
ii nova-compute-libvirt 2:12.0.2-0ubuntu1~cloud0 all OpenStack Compute - compute node libvirt support
ii python-libvirt 1.2.2-0ubuntu2 amd64 libvirt Python bindings

[sean redmond (sean-redmond1)]

^^ As this does not match version >=1.2.5 - this looks to be a requirement in the spec (https://github.com/openstack/nova-specs/blob/master/specs/kilo/implemented/quiesced-image-snapshots-with-qemu-guest-agent.rst)

I also could not find a reference to this API call (fsFreeze) in the libvirt python packages on trusty.

# grep -i fsfreeze /usr/lib/python2.7/dist-packages/libvirt.*

[James Page (james-page)]

This is a bit of a gap in the UCA pockets for kilo and liberty; mitaka ships a matched python-libvirt with the core libvirt package (1.3.1).

I'll look into fixing this gap, but we'll need todo a full regression so might take a while to turn the wheel on this one...

[James Page (james-page)]

Marking Nova (Ubuntu) bug task as invalid as this problem is specific to the UCA.

[sean redmond (sean-redmond1)]

If there is anything I can do to help/test just let me know.

[James Page (james-page)]

Sean

If you're running liberty, testing with proposed would be helpful:

  add-apt-repository cloud-archive:liberty-proposed

I'll bump the kilo update forward soon.

[James Page (james-page)]

Actually its already in proposed for kilo:

  add-apt-repository cloud-archive:kilo-proposed

[Matt Riedemann (mriedem)]

Where was this actually fixed for mitaka UCA because I don't see anything about this bug in the change log:

https://launchpadlibrarian.net/242454264/libvirt-python_1.3.1-1ubuntu1~cloud0_source.changes

[James Page (james-page)]

@Matt

You won't for Mitaka - the problem was more that libvirt-python was missing from the Kilo and Liberty UCA pockets, whereas we already had it backported in the mitaka UCA.