GlusterFS: fail snapshot creation gracefully if user can't write to share

Bug #1236966 reported by Dafna Ron on 2013-10-08
24
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Cinder
Medium
Eric Harney
Havana
Undecided
Unassigned

Bug Description

I configured cinder to work with gluster as backend but when i try to create a snapshot I fail with the following error:

>>>>>>>>>>>>>>>>>
2013-10-08 19:53:26.918 7297 ERROR cinder.openstack.common.rpc.amqp [req-5beb2863-2be2-439f-8356-80f58b0ec440 c02995f25ba44cfab1a3cbd419f045a1 c77235c29fd0431a8e6628ef6d18e07f] Exception during message handling
2013-10-08 19:53:26.918 7297 TRACE cinder.openstack.common.rpc.amqp Traceback (most recent call last):
2013-10-08 19:53:26.918 7297 TRACE cinder.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/cinder/openstack/common/rpc/amqp.py", line 441, in _process_data
2013-10-08 19:53:26.918 7297 TRACE cinder.openstack.common.rpc.amqp **args)
2013-10-08 19:53:26.918 7297 TRACE cinder.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/cinder/openstack/common/rpc/dispatcher.py", line 148, in dispatch
2013-10-08 19:53:26.918 7297 TRACE cinder.openstack.common.rpc.amqp return getattr(proxyobj, method)(ctxt, **kwargs)
2013-10-08 19:53:26.918 7297 TRACE cinder.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/cinder/volume/manager.py", line 326, in create_snapshot
2013-10-08 19:53:26.918 7297 TRACE cinder.openstack.common.rpc.amqp {'status': 'error'})
2013-10-08 19:53:26.918 7297 TRACE cinder.openstack.common.rpc.amqp File "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__
2013-10-08 19:53:26.918 7297 TRACE cinder.openstack.common.rpc.amqp self.gen.next()
2013-10-08 19:53:26.918 7297 TRACE cinder.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/cinder/volume/manager.py", line 317, in create_snapshot
2013-10-08 19:53:26.918 7297 TRACE cinder.openstack.common.rpc.amqp model_update = self.driver.create_snapshot(snapshot_ref)
2013-10-08 19:53:26.918 7297 TRACE cinder.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/cinder/volume/drivers/glusterfs.py", line 349, in create_snapshot
2013-10-08 19:53:26.918 7297 TRACE cinder.openstack.common.rpc.amqp new_snap_path)
2013-10-08 19:53:26.918 7297 TRACE cinder.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/cinder/volume/drivers/glusterfs.py", line 438, in _create_qcow2_snap_file
2013-10-08 19:53:26.918 7297 TRACE cinder.openstack.common.rpc.amqp self._execute(*command, run_as_root=True)
2013-10-08 19:53:26.918 7297 TRACE cinder.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/cinder/utils.py", line 142, in execute
2013-10-08 19:53:26.918 7297 TRACE cinder.openstack.common.rpc.amqp return processutils.execute(*cmd, **kwargs)
2013-10-08 19:53:26.918 7297 TRACE cinder.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/cinder/openstack/common/processutils.py", line 173, in execute
2013-10-08 19:53:26.918 7297 TRACE cinder.openstack.common.rpc.amqp cmd=' '.join(cmd))
2013-10-08 19:53:26.918 7297 TRACE cinder.openstack.common.rpc.amqp ProcessExecutionError: Unexpected error while running command.
2013-10-08 19:53:26.918 7297 TRACE cinder.openstack.common.rpc.amqp Command: sudo cinder-rootwrap /etc/cinder/rootwrap.conf qemu-img create -f qcow2 -o backing_file=/var/lib/cinder/mnt/4a31bc6e5fb9244971075aa23d364364/volume-e78978af-0f46-4caf-948b-218afb5de6ef /var/lib/cinder/mnt/4a31bc6e5fb9244971075aa23d364364/volume-e78978af-0f46-4caf-948b-218afb5de6ef.28dd6a1e-9e96-4dc8-8c09-e38504b6ef6d
2013-10-08 19:53:26.918 7297 TRACE cinder.openstack.common.rpc.amqp Exit code: 1
2013-10-08 19:53:26.918 7297 TRACE cinder.openstack.common.rpc.amqp Stdout: "Formatting '/var/lib/cinder/mnt/4a31bc6e5fb9244971075aa23d364364/volume-e78978af-0f46-4caf-948b-218afb5de6ef.28dd6a1e-9e96-4dc8-8c09-e38504b6ef6d', fmt=qcow2 size=1073741824 backing_file='/var/lib/cinder/mnt/4a31bc6e5fb9244971075aa23d364364/volume-e78978af-0f46-4caf-948b-218afb5de6ef' encryption=off cluster_size=65536 \n"
2013-10-08 19:53:26.918 7297 TRACE cinder.openstack.common.rpc.amqp Stderr: '/var/lib/cinder/mnt/4a31bc6e5fb9244971075aa23d364364/volume-e78978af-0f46-4caf-948b-218afb5de6ef.28dd6a1e-9e96-4dc8-8c09-e38504b6ef6d: error while creating qcow2: No such file or directory\n'

tags: added: drivers glusterfs
Eric Harney (eharney) wrote :

The driver expects that the Cinder user can write to the GlusterFS share. This failure probably happened due to permissions settings in that regard.

Eric Harney (eharney) on 2013-11-14
summary: - fail to create a snapshot when cinder is configured to work with gluster
+ GlusterFS: snapshot creation fails if user can't write to share
summary: - GlusterFS: snapshot creation fails if user can't write to share
+ GlusterFS: fail snapshot creation gracefully if user can't write to
+ share

Fix proposed to branch: master
Review: https://review.openstack.org/56482

Changed in cinder:
assignee: nobody → Eric Harney (eharney)
status: New → In Progress

Fix proposed to branch: master
Review: https://review.openstack.org/57542

Reviewed: https://review.openstack.org/57542
Committed: http://github.com/openstack/cinder/commit/371fa540600b20b97eae389e1f976145866cadae
Submitter: Jenkins
Branch: master

commit 371fa540600b20b97eae389e1f976145866cadae
Author: Eric Harney <email address hidden>
Date: Tue Nov 19 18:01:55 2013 -0500

    GlusterFS: Ensure Cinder can write to shares

    Ensure the Cinder user can write to the GlusterFS share. This
    is required for snapshot functionality, and means the admin
    does not have to set this permission manually.

    Closes-Bug: #1236966
    Change-Id: I4a9ea40df9681ca6931ad6b390aa21b09d6cfec9

Changed in cinder:
status: In Progress → Fix Committed
Eric Harney (eharney) on 2013-12-03
tags: added: havana-backport-potential
Thierry Carrez (ttx) on 2013-12-04
Changed in cinder:
milestone: none → icehouse-1
status: Fix Committed → Fix Released
Eric Harney (eharney) on 2014-01-24
Changed in cinder:
importance: Undecided → Medium
Thierry Carrez (ttx) on 2014-04-17
Changed in cinder:
milestone: icehouse-1 → 2014.1
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers