Snapshot of a bootable volume (created without using image) goes into error state

Bug #1413880 reported by Ankit Agrawal
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Cinder
Fix Released
High
Pranali Deore
Juno
Won't Fix
Undecided
Pranali Deore

Bug Description

If an empty volume is created without using image and updated to set as bootable, Then snapshot of this volume goes into error state. This issue affects both V1 and V2 APIs.

Steps to reproduce:

1. Create a non-bootable volume.
$ cinder create --name test_vol 1

2. Update volume to bootable.
$ cinder set-bootable <volume_id> True

3. Create volume snapshot.
$ cinder snapshot-create <volume_id> --name test_vol_snap

4. Show volume snapshot list.
$ cinder snapshot-list

And you’ll find volume snapshot is in “error” state.

cinder-volume logs:

2015-01-22 06:22:54.783 ERROR cinder.volume.manager [req-e0f05a27-458b-4fd6-88f4-e6e9b2e31041 232eef31698047f4afcc104f58b64998 2447420aa17d41e68e2fb91
42b35224f] Failed updating 44ecef25-52c4-4dfd-ab53-1ef6c66b4827 metadata using the provided volumes 64a7aee4-53c3-49ed-bfaf-05b77fdd045f metadata
2015-01-22 06:22:54.783 TRACE cinder.volume.manager Traceback (most recentcall last):
2015-01-22 06:22:54.783 TRACE cinder.volume.manager File "/opt/stack/cinder/cinder/volume/manager.py", line 566, in create_snapshot
2015-01-22 06:22:54.783 TRACE cinder.volume.manager context, snapshot_ref['id'], volume_id)
2015-01-22 06:22:54.783 TRACE cinder.volume.manager File "/opt/stack/cinder/cinder/db/api.py", line 629, in volume_glance_metadata_copy_to_snapshot
2015-01-22 06:22:54.783 TRACE cinder.volume.manager volume_id)
2015-01-22 06:22:54.783 TRACE cinder.volume.manager File "/opt/stack/cinder/cinder/db/sqlalchemy/api.py", line 174, in wrapper
2015-01-22 06:22:54.783 TRACE cinder.volume.manager return f(*args, **kwargs)
2015-01-22 06:22:54.783 TRACE cinder.volume.manager File "/opt/stack/cinder/cinder/db/sqlalchemy/api.py", line 201, in wrapper
2015-01-22 06:22:54.783 TRACE cinder.volume.manager return f(context, snapshot_id, *args, **kwargs)
2015-01-22 06:22:54.783 TRACE cinder.volume.manager File "/opt/stack/cinder/cinder/db/sqlalchemy/api.py", line 2796, in volume_glance_metadata_copy_to_snapshot
2015-01-22 06:22:54.783 TRACE cinder.volume.manager session=session)
2015-01-22 06:22:54.783 TRACE cinder.volume.manager File "/opt/stack/cinder/cinder/db/sqlalchemy/api.py", line 174, in wrapper
2015-01-22 06:22:54.783 TRACE cinder.volume.manager return f(*args, **kwargs)
2015-01-22 06:22:54.783 TRACE cinder.volume.manager File "/opt/stack/cinder/cinder/db/sqlalchemy/api.py", line 187, in wrapper
2015-01-22 06:22:54.783 TRACE cinder.volume.manager return f(context, volume_id, *args, **kwargs)
2015-01-22 06:22:54.783 TRACE cinder.volume.manager File "/opt/stack/cinder/cinder/db/sqlalchemy/api.py", line 2718, in _volume_glance_metadata_get
2015-01-22 06:22:54.783 TRACE cinder.volume.manager raise exception.GlanceMetadataNotFound(id=volume_id)
2015-01-22 06:22:54.783 TRACE cinder.volume.manager GlanceMetadataNotFound: Glance metadata for volume/snapshot 64a7aee4-53c3-49ed-bfaf-05b77fdd045f cannot be found.
2015-01-22 06:22:54.783 TRACE cinder.volume.manager
2015-01-22 06:22:54.839 ERROR oslo.messaging.rpc.dispatcher [req-e0f05a27-458b-4fd6-88f4-e6e9b2e31041 232eef31698047f4afcc104f58b64998 2447420aa17d41e68e2fb9142b35224f] Exception during message handling: Failed to copy metadata to volume: Glance metadata for volume/snapshot 64a7aee4-53c3-49ed-bfaf-05b77fdd045f cannot be found.
2015-01-22 06:22:54.839 TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last):
2015-01-22 06:22:54.839 TRACE oslo.messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 137, in _dispatch_and_reply
2015-01-22 06:22:54.839 TRACE oslo.messaging.rpc.dispatcher incoming.message))
2015-01-22 06:22:54.839 TRACE oslo.messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 180, in _dispatch
2015-01-22 06:22:54.839 TRACE oslo.messaging.rpc.dispatcher return self._do_dispatch(endpoint, method, ctxt, args)
2015-01-22 06:22:54.839 TRACE oslo.messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 126, in _do_dispatch
2015-01-22 06:22:54.839 TRACE oslo.messaging.rpc.dispatcher result = getattr(endpoint, method)(ctxt, **new_args)
2015-01-22 06:22:54.839 TRACE oslo.messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/osprofiler/profiler.py", line 105, in wrapper
2015-01-22 06:22:54.839 TRACE oslo.messaging.rpc.dispatcher return f(*args, **kwargs)
2015-01-22 06:22:54.839 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/cinder/cinder/volume/manager.py", line 576, in create_snapshot
2015-01-22 06:22:54.839 TRACE oslo.messaging.rpc.dispatcher raise exception.MetadataCopyFailure(reason=ex)
2015-01-22 06:22:54.839 TRACE oslo.messaging.rpc.dispatcher MetadataCopyFailure: Failed to copy metadata to volume: Glance metadata for volume/snapshot 64a7aee4-53c3-49ed-bfaf-05b77fdd045f cannot be found.

Revision history for this message
Mike Perez (thingee) wrote :

So if we were to catch this exception and continue, the snapshot would be created without updating it with glance metadata. Later on if an end user creates a volume from that snapshot, it will not automatically be set as bootable because we determine it's bootable automatically because it has glance metadata. Does it make sense when the volume is set to bootable via Cinder API to set some glance metadata?

Changed in cinder:
milestone: none → kilo-2
status: New → Confirmed
importance: Undecided → High
tags: removed: ntt
Changed in cinder:
assignee: nobody → Pranali Deore (pranali-deore)
Revision history for this message
Max Krasilnikov (pseudo) wrote :

Hello!

I have the same bug in cinder 2014.2.1-0ubuntu1 on Ubuntu 14.10 utopic. When ignoring all the errors with reset-state/snapshot-reset-state all the things doing well.

Mike Perez (thingee)
Changed in cinder:
milestone: kilo-2 → kilo-3
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to cinder (master)

Fix proposed to branch: master
Review: https://review.openstack.org/153170

Changed in cinder:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to cinder (master)

Reviewed: https://review.openstack.org/153170
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=8583aaf8542225da5542d4871763617c1378b1e3
Submitter: Jenkins
Branch: master

commit 8583aaf8542225da5542d4871763617c1378b1e3
Author: PranaliDeore <email address hidden>
Date: Tue Jan 27 04:46:58 2015 -0800

    Snapshot of bootable volume goes in error state

    If volume is not created from image, no glance
    metadata would be available for that volume/snapshot
    in volume glance metadata table. So simply catch
    GlanceMetadataNotFound and do nothing.

    Closes-Bug: #1413880
    Change-Id: Ic3267d422912f94e966412859e2b8bddeddfb75f

Changed in cinder:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to cinder (stable/juno)

Fix proposed to branch: stable/juno
Review: https://review.openstack.org/156936

Thierry Carrez (ttx)
Changed in cinder:
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in cinder:
milestone: kilo-3 → 2015.1.0
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on cinder (stable/juno)

Change abandoned by Mike Perez (<email address hidden>) on branch: stable/juno
Review: https://review.openstack.org/156936
Reason: Over a month with no update.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.