VNXCreateSnapError: failed to create snap when running test_volumes_clone

Bug #1796825 reported by Sam Wan on 2018-10-09
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Cinder
Undecided
Ryan Liang

Bug Description

tempest case 'tempest.api.volume.test_volumes_clone.VolumesCloneTest.test_create_from_bootable_volume' failed with below errors in cinder log
======
Oct 07 23:28:27 vnxvm1 cinder-volume[23320]: ERROR oslo_messaging.rpc.server keep_for=keep_for)
Oct 07 23:28:27 vnxvm1 cinder-volume[23320]: ERROR oslo_messaging.rpc.server File "/home/sam/storops/storops/vnx/resource/lun.py", line 185, in create_snap
Oct 07 23:28:27 vnxvm1 cinder-volume[23320]: ERROR oslo_messaging.rpc.server auto_delete, keep_for)
Oct 07 23:28:27 vnxvm1 cinder-volume[23320]: ERROR oslo_messaging.rpc.server File "/home/sam/storops/storops/vnx/resource/snap.py", line 51, in create
Oct 07 23:28:27 vnxvm1 cinder-volume[23320]: ERROR oslo_messaging.rpc.server ex.raise_if_err(out, msg, default=ex.VNXCreateSnapError)
Oct 07 23:28:27 vnxvm1 cinder-volume[23320]: ERROR oslo_messaging.rpc.server File "/home/sam/storops/storops/exception.py", line 227, in raise_if_err
Oct 07 23:28:27 vnxvm1 cinder-volume[23320]: ERROR oslo_messaging.rpc.server raise ex_clz(msg)
Oct 07 23:28:27 vnxvm1 cinder-volume[23320]: ERROR oslo_messaging.rpc.server VNXCreateSnapError: failed to create snap "tmp-snap-2f7749d0-d73a-4e17-8802-55f0eecfee82"
for 63774 detail:
Oct 07 23:28:27 vnxvm1 cinder-volume[23320]: ERROR oslo_messaging.rpc.server The snapshot's primary LUN is migrating. Stop the migration and retry the command, or spe
cify "-ignoreMigrationCheck" switch to proceed with the snapshot creation.
Oct 07 23:28:27 vnxvm1 cinder-volume[23320]: ERROR oslo_messaging.rpc.server Note: After migration, these snapshots will be automatically deleted. Reissue the command
with "-ignoreMigrationCheck" switch for the command to succeed.
Oct 07 23:28:27 vnxvm1 cinder-volume[23320]: ERROR oslo_messaging.rpc.server
Oct 07 23:28:27 vnxvm1 cinder-volume[23320]: DEBUG cinder.coordination [None req-6ca41ee3-497c-4b92-a54a-d64deeb5332d tempest-VolumesCloneTest-2145099117 None] Lock "
/opt/stack/data/cinder/cinder-5e2b523f-9aa0-48b1-9a03-a477815b24d5-delete_volume" acquired by "delete_volume" :: waited 4.009s {{(pid=23435) _synchronized /opt/stack/
======

Ryan Liang (ryan-liang) on 2018-10-09
Changed in cinder:
assignee: nobody → Ryan Liang (ryan-liang)
Ryan Liang (ryan-liang) on 2018-10-09
tags: added: dellemc
tags: added: vnx

Fix proposed to branch: master
Review: https://review.openstack.org/609340

Changed in cinder:
status: New → In Progress
Ryan Liang (ryan-liang) wrote :

Async migration was introduced to accelerate the cloning of volumes
(The cloned volumes can be accessed after the migrations start,
no need to wait them for finishing).

However, before migrations finish, snapshots cannot be created on the
source volumes any more, which could affect the follow-up clone from the
same source volumes. That is the constraint of async migration.

The fix is adding the constraint to the doc of VNX driver. And give
users some advises on how to make the follow-up volume clone work.

Ryan Liang (ryan-liang) wrote :

The change to the doc:

**Constraints**

- Before the migrations finish, snapshots cannot be created on the source
  volumes, which could affects the follow-up clone from the same source
  volumes. The typical affected use case is that creating volume-2 via cloning
  volume-1 immediately after creating volume-1 via cloning volume-0. To achieve
  so, users are advised to 1) wait for the first clone finishing, or 2) create
  volume-2 via cloning volume-0 instead of volume-1, or 3) create volume-1 with
  ``--metadata async_migrate=False``.

Reviewed: https://review.openstack.org/609340
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=3269da58d8fb6cdbb902487914e7a6dedee8b22f
Submitter: Zuul
Branch: master

commit 3269da58d8fb6cdbb902487914e7a6dedee8b22f
Author: Ryan Liang <email address hidden>
Date: Wed Oct 10 17:24:34 2018 +0800

    VNX: Add constraints for async migration

    Async migration was introduced to accelerate the cloning of volume (
    The cloned volume can be accessed after the migration starts, no need
    to wait it for finishing).

    However, before the migration finishes, snapshots cannot be created from
    the source volume any more, which could affect subsequent clones from
    the same source volume. That is the constraint of async migration.

    The fix is adding the constraint to the doc of VNX driver. And give
    users some advises on how to make the subsequent volume clone work.

    Change-Id: I6ad3c6b967d5a33966b40c3a54c0f110576c8d87
    Closes-bug: #1796825

Changed in cinder:
status: In Progress → Fix Released

This issue was fixed in the openstack/cinder 14.0.0.0rc1 release candidate.

Reviewed: https://review.openstack.org/639754
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=c5e2ed9daf2be132819755f7edaea5c29b1dca4c
Submitter: Zuul
Branch: stable/rocky

commit c5e2ed9daf2be132819755f7edaea5c29b1dca4c
Author: Ryan Liang <email address hidden>
Date: Wed Oct 10 17:24:34 2018 +0800

    VNX: Add constraints for async migration

    Async migration was introduced to accelerate the cloning of volume (
    The cloned volume can be accessed after the migration starts, no need
    to wait it for finishing).

    However, before the migration finishes, snapshots cannot be created from
    the source volume any more, which could affect subsequent clones from
    the same source volume. That is the constraint of async migration.

    The fix is adding the constraint to the doc of VNX driver. And give
    users some advises on how to make the subsequent volume clone work.

    Change-Id: I6ad3c6b967d5a33966b40c3a54c0f110576c8d87
    Closes-bug: #1796825
    (cherry picked from commit 3269da58d8fb6cdbb902487914e7a6dedee8b22f)

tags: added: in-stable-rocky

This issue was fixed in the openstack/cinder 13.0.5 release.

Reviewed: https://review.opendev.org/639277
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=3ecdc2bfc827c5ae8003f877bd95ca8c1385c596
Submitter: Zuul
Branch: master

commit 3ecdc2bfc827c5ae8003f877bd95ca8c1385c596
Author: Ryan Liang <email address hidden>
Date: Tue Feb 26 17:12:27 2019 +0800

    VNX: add option vnx_async_migrate

    Before this fix, customers can disable async migration only by passing
    `--metadata async_migrate=False` in volumes metadata. This fix adds an
    option named `vnx_async_migrate` which could help set the default value
    of async migration for whole backend. If customers don't pass in the
    `async_migrate` in metadata, the value of this option will be used.

    Change-Id: I8e59039da78ecc9fb5415a10ff2a7c26ee689171
    Closes-bug: #1796825

This issue was fixed in the openstack/cinder 15.0.0.0rc1 release candidate.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers