Snapshot Backup fails with remotefs based Drivers

Bug #1703405 reported by Silvan Kaiser on 2017-07-10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Silvan Kaiser

Bug Description

The new tempest test for snapshot backups [1] fails with:

InvalidSnapshot: Snapshot status must be "available" to clone. But is: backing-up

Example log for the Quobyte driver at [2].

As the volume is in the 'backing-up' state the snapshot operation fails as it requires the volume to be in the state 'available'.

This probably hits every Driver that uses the default implementation of _create_volume_from_snapshot from .


Silvan Kaiser (2-silvan) on 2017-07-10
tags: added: quobyte
Eric Harney (eharney) wrote :

The tempest test only checks for "volume_feature_enabled.backup", meaning it equates backup support with snapshot backup support. This is not correct, not all Cinder drivers support snapshot backup.

Change abandoned by Silvan Kaiser (<email address hidden>) on branch: master
Reason: This just overrides the default and does not fix #1703405

Changed in tempest:
assignee: nobody → Vishakha Agarwal (vishakha.agarwal)
Changed in tempest:
assignee: Vishakha Agarwal (vishakha.agarwal) → nobody

Hi Silvan,

Are you still facing the issue.
I can see mentioned test is currently passing.

Last time, test modified under:

Seems not reproducible. If still it's issue then could you please provide the reproducing steps?

Uemit Seren (timeu) wrote :

We are running into a similar issue both in above tempest tests as well as when manually creating a backup of a volume with an attached snapshot. This is happening with OSP12 with a Isilon backed NFS cinder backend using the generic NFS driver (The error does not occur when using a NetApp backend with its native driver)

The error in /etc/cinder/backup.log is:

2018-10-11 13:54:30,855.855 785498 ERROR oslo_messaging.rpc.server RemoteError: Remote error: InvalidSnapshot Snapshot status must be "available" to clone. But is: backing-up

The steps to reproduce are:

1.) Create a volume
2.) Create a snapshot of the volume
3.) Create a backup of the volume using the snapshot

openstack volume create test-vol --type nfs --size 1
openstack volume snapshot create --volume test-vol test-snapshot
openstack volume backup create test-vol --snapshot test-snapshot --name test-backup # this fails

If you don't specify the snapshot the backup works:

openstack volume backup create test-vol --name test-backup2 # this works

openstack volume backup list
| ID | Name | Description | Status | Size |
| fecd40b9-76da-4fc2-a25c-e38401622068 | test-backup2 | None | available | 1 |
| 169195d5-2939-4f71-9c98-3b5de2db86bb | test-backup | None | error | 1 |

Interestingly the snapshot has the state available:

openstack volume snapshot list
| ID | Name | Description | Status | Size |
| 6310c270-4ec1-4af0-ae9e-cae156d69c98 | test-snapshot | None | available | 1 |

Silvan Kaiser (2-silvan) wrote :

Hi Manik!
Yes, this issue is still current, i just manually verified to be sure.
Running current master branches with Devstack and using the Cinder Quobyte driver and the PosixBackupDriver to run the tempest test referenced above.

I think Eric pointed out the core issue, not all Cinder drivers support backup by snapshot, this might require another config flag?

Ghanshyam Mann (ghanshyammann) wrote :

yeah as per Eric, not all cinder driver support snapshot backup. is there any way to detect this via cinder API?

I am hesitate to introduce the feature flag for this feature. if this is commonly implemented/supported feature then, we can ask driver to implement it and till then they can skip this test to run maually via backlist.

Jay Bryant (jsbryant) wrote :

There are a few features that fail tempest testing on various backends. They blacklist those tests so that testing passes. I don't know off the top of my head how many drivers don't implement this but I think that just disabling the test for drivers that don't is the best way to go rather than trying to add more config options.

Ghanshyam Mann (ghanshyammann) wrote :

I agree with Jay. marking invalid for Tempest.

Changed in tempest:
status: New → Invalid

Fix proposed to branch: master

Changed in cinder:
assignee: nobody → Silvan Kaiser (2-silvan)
status: New → In Progress
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers