Cannot restore a backup to a larger volume on ceph

Bug #1529257 reported by Yuriy Nesenenko
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Mirantis OpenStack
Fix Released
High
Ivan Berezovskiy

Bug Description

Steps to reproduce the bug:
1. Create two volumes with 2G and 3G size (Bootale=False).
2. Create a backup from volume of 2G size.

$ cinder backup-list
+--------------------------------------+--------------------------------------+-----------+----------+------+--------------+-----------+
| ID | Volume ID | Status | Name | Size | Object Count | Container |
+--------------------------------------+--------------------------------------+-----------+----------+------+--------------+-----------+
| b56220c1-6845-4c71-89a1-11932933c909 | ec650bb3-1f69-4ad8-80e7-5ed48221e042 | available | vol1-bak | 2 | 0 | backup |
+--------------------------------------+--------------------------------------+-----------+----------+------+--------------+-----------+

3. Try restoring a backup volume of 2G to larger volume.

$ cinder backup-restore b56220c1-6845-4c71-89a1-11932933c909
               --volume 561e706e-969c-4a8e-860b-bd593eefaa86

Expected results:

$ cinder list
+--------------------------------------+-----------------+------------------+-------------+------+-------------+----------+-------------+-------------+
| ID | Status | Migration Status | Name | Size | Volume Type | Bootable | Multiattach | Attached to |
+--------------------------------------+-----------------+------------------+-------------+------+-------------+----------+-------------+-------------+
| 561e706e-969c-4a8e-860b-bd593eefaa86 | available | - | vol-restore | 3 | ceph | false | False | |
| ec650bb3-1f69-4ad8-80e7-5ed48221e042 | available | - | vol1 | 2 | ceph | false | False | |
+--------------------------------------+-----------------+------------------+-------------+------+-------------+----------+-------------+-------------+

Real results:

Restoring a backup vol of 2G to a larger volume fails with status "error_restoring"

$ cinder list
+--------------------------------------+-----------------+------------------+-------------+------+-------------+----------+-------------+-------------+
| ID | Status | Migration Status | Name | Size | Volume Type | Bootable | Multiattach | Attached to |
+--------------------------------------+-----------------+------------------+-------------+------+-------------+----------+-------------+-------------+
| 561e706e-969c-4a8e-860b-bd593eefaa86 | error_restoring | - | vol-restore | 3 | ceph | false | False | |
| ec650bb3-1f69-4ad8-80e7-5ed48221e042 | available | - | vol1 | 2 | ceph | false | False | |
+--------------------------------------+-----------------+------------------+-------------+------+-------------+----------+-------------+-------------+

This problem is caused by the lack of read/write permissions to volumes pool for backup user. We need such permissions to change volume size during restoring backup when the backup is smaller than a volume.

Changed in mos:
status: New → Confirmed
assignee: nobody → MOS Puppet Team (mos-puppet)
importance: Undecided → High
Revision history for this message
Yuriy Nesenenko (ynesenenko) wrote :

This bug can be fixed in such a way https://review.openstack.org/#/c/261005/ for upstream.

tags: added: area-cinder
Changed in mos:
milestone: none → 8.0
Revision history for this message
Ivan Berezovskiy (iberezovskiy) wrote :

Does this really affects Fuel? As I can see this fix is already in fuel-library:
https://github.com/openstack/fuel-library/blob/master/deployment/puppet/osnailyfacter/modular/ceph/ceph_pools.pp#L32

Please, provide information about ISO and environment, if you reproduce this issue.

Changed in mos:
status: Confirmed → Incomplete
assignee: MOS Puppet Team (mos-puppet) → Yuriy Nesenenko (ynesenenko)
Revision history for this message
Yuriy Nesenenko (ynesenenko) wrote :

Please look at:
https://github.com/openstack/fuel-library/blob/master/deployment/puppet/osnailyfacter/modular/ceph/ceph_pools.pp#L40

As you've seen there is only 'allow rx pool=${cinder_pool}'
The backup_user don't have write permission to volumes pool. So he can't change volume size during restoring backup when the backup is smaller than a volume.

Changed in mos:
assignee: Yuriy Nesenenko (ynesenenko) → Ivan Berezovskiy (iberezovskiy)
Revision history for this message
Ivan Berezovskiy (iberezovskiy) wrote :

Need to change allow rx pool=${cinder_pool} to allow rwx pool=${cinder_pool}

Changed in mos:
status: Incomplete → Confirmed
Revision history for this message
Ivan Berezovskiy (iberezovskiy) wrote :
Changed in mos:
status: Confirmed → In Progress
Revision history for this message
Ivan Berezovskiy (iberezovskiy) wrote :

Patch for stable/8.0 branch: https://review.openstack.org/#/c/265748/

Changed in mos:
status: In Progress → Fix Committed
tags: added: cinder
tags: removed: cinder
Egor Kotko (ykotko)
tags: added: on-verification
Revision history for this message
Egor Kotko (ykotko) wrote :

I have tried to restore small volume (2GB) created from image to empty volume (not bootable) - it was ok,
but if restore in big bootable volume - got the error:
http://paste.openstack.org/show/487236/

Please add steps to reproduce to bug.

description: updated
description: updated
Revision history for this message
Egor Kotko (ykotko) wrote :

ISO #573

Changed in mos:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.