OpenStack Compute (nova)

Unable to snapshot an instance with ephemeral RBD

Bug #1261675 reported by Pavel Kirpichyov on 2013-12-17

This bug affects 5 people

Affects		Status	Importance	Assigned to	Milestone
	OpenStack Compute (nova)	Fix Released	Undecided	Pavel Kirpichyov	OpenStack Compute (nova) 2014.1 "icehouse"

Bug Description

When creating a snapshot from a VM instance, following error appears in Nova compute log:

2013-12-17 10:39:47.943 28210 ERROR nova.openstack.common.rpc.amqp [req-3a82cad9-3213-47e8-8a42-0f6e2a75008a a2eecc1caf5f40c5ab50405b68730c20 e255029e4a614ed1a5412192db588e74] Exception during message handling
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp Traceback (most recent call last):
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py", line 461, in _process_data
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp **args)
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/dispatcher.py", line 172, in dispatch
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp result = getattr(proxyobj, method)(ctxt, **kwargs)
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 353, in decorated_function
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp return function(self, context, *args, **kwargs)
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 90, in wrapped
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp payload)
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 73, in wrapped
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp return f(self, context, *args, **kw)
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 243, in decorated_function
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp pass
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 229, in decorated_function
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp return function(self, context, *args, **kwargs)
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 271, in decorated_function
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp e, sys.exc_info())
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 258, in decorated_function
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp return function(self, context, *args, **kwargs)
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 319, in decorated_function
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp % image_id, instance=instance)
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 309, in decorated_function
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp *args, **kwargs)
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2293, in snapshot_instance
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp task_states.IMAGE_SNAPSHOT)
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2324, in _snapshot_instance
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp update_task_state)
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1399, in snapshot
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp snapshot_backend.snapshot_extract(out_path, image_format)
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/imagebackend.py", line 531, in snapshot_extract
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp images.convert_image(snap, target, out_format)
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/virt/images.py", line 179, in convert_image
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp utils.execute(*cmd, run_as_root=run_as_root)
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/utils.py", line 177, in execute
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp return processutils.execute(*cmd, **kwargs)
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/openstack/common/processutils.py", line 178, in execute
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp cmd=' '.join(cmd))
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp ProcessExecutionError: Unexpected error while running command.
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp Command: qemu-img convert -O qcow2 rbd:instances/instance-0000000f_disk /var/lib/nova/instances/snapshots/tmp63tXq0/b916d3ca7fbe46cba4ca7ce5e0138ee9
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp Exit code: 1
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp Stdout: ''
2013-12-17 10:39:47.943 28210 TRACE nova.openstack.common.rpc.amqp Stderr: "qemu-img: error connecting\nqemu-img: Could not open 'rbd:instances/instance-0000000f_disk': Operation not supported\nqemu-img: Could not open 'rbd:instances/instance-0000000f_disk'\n"

Tags:

Revision history for this message

Pavel Kirpichyov (pavel-kirpichyov) wrote on 2013-12-17:

https://review.openstack.org/#/c/62592

Changed in nova:
assignee:	nobody → Pavel Kirpichyov (pavel-kirpichyov)

OpenStack Infra (hudson-openstack) on 2013-12-17

Changed in nova:
status:	New → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-01-17: Fix merged to nova (master)

Reviewed: https://review.openstack.org/62592
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=9e7e8daa66f0c05ac7ce3927562d246baf70067f
Submitter: Jenkins
Branch: master

commit 9e7e8daa66f0c05ac7ce3927562d246baf70067f
Author: Pavel Kirpichyov <email address hidden>
Date: Tue Dec 17 13:47:57 2013 +0200

Pass rbd_user id and conf path as part of RBD URI for qemu-img

Unlike rados.py, qemu-img does not use id from CEPH_ARGS and has to be
given a ceph user id and conf path explicitly in RBD URI

Closes-Bug: #1261675
Change-Id: I9226515f2a860ed45d12012fd5b6bc7e842f6d32

Changed in nova:
status:	In Progress → Fix Committed

Thierry Carrez (ttx) on 2014-01-22

Changed in nova:
milestone:	none → icehouse-2
status:	Fix Committed → Fix Released

Thierry Carrez (ttx) on 2014-04-17

Changed in nova:
milestone:	icehouse-2 → 2014.1

Report a bug

This report contains Public information

Everyone can see this information.

Duplicates of this bug

Bug #1300086

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.