Live migration disconnects attached volumes on target

Bug #1361738 reported by Alessandro Pilotti
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Cinder
Fix Released
High
Alessandro Pilotti
Icehouse
Fix Released
High
Alessandro Pilotti

Bug Description

Live migration disconnects attached volumes on target. The action itself does not fail and the instance appears online on the target host, but the underlying iSCSi connection results closed on the destination host, resulting in failures in the guest when accessing the volume(s).

The issue can be traced to this call to the following self.volume_api.terminate_connection(...) in the manager:

https://github.com/openstack/nova/blob/c594340359e3d148b6c349c388f10327ac7b8bfa/nova/compute/manager.py#L4868

And in turn to this Cinder patch that introduced the regression:
https://review.openstack.org/#/c/76471/7/cinder/volume/manager.py

Verified on Hyper-V using Devstack with both LVM and Windows Cinder drivers.

Changed in cinder:
status: New → Confirmed
Revision history for this message
Vish Ishaya (vishvananda) wrote :

This is not Hyper-V only. This is due to a change that moved detach into terminate_connection which was incorrect:

https://review.openstack.org/76471

Changed in cinder:
importance: Undecided → High
summary: - Live migration on Hyper-V disconnects attached volumes on target
+ Live migration disconnects attached volumes on target
description: updated
description: updated
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to cinder (master)

Fix proposed to branch: master
Review: https://review.openstack.org/117033

Changed in cinder:
assignee: nobody → Alessandro Pilotti (alexpilotti)
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to cinder (stable/icehouse)

Fix proposed to branch: stable/icehouse
Review: https://review.openstack.org/117050

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to cinder (master)

Reviewed: https://review.openstack.org/117033
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=ddcad011db507cde66b6b1e655d5ffc91ab8880f
Submitter: Jenkins
Branch: master

commit ddcad011db507cde66b6b1e655d5ffc91ab8880f
Author: Alessandro Pilotti <email address hidden>
Date: Wed Aug 27 00:38:43 2014 +0300

    Fixes terminate_connection live migration issue

    Reverts the changes to cinder/volume/manager.py added in
    commit b868ae707f9ecbe254101e21d9d7ffa0b05b17d1 as calling
    remove_export in terminate_connection causes Nova live
    migration to fail when volumes are attached.

    Change-Id: I38f3db336db22bb096a8855bcbbfeba16e517b80
    Closes-Bug: #1361738

Changed in cinder:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in cinder:
milestone: none → juno-3
status: Fix Committed → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to cinder (stable/icehouse)

Reviewed: https://review.openstack.org/117050
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=8f7823d300d8e85ac8cc5d80437eec736f83aad6
Submitter: Jenkins
Branch: stable/icehouse

commit 8f7823d300d8e85ac8cc5d80437eec736f83aad6
Author: Alessandro Pilotti <email address hidden>
Date: Wed Aug 27 00:38:43 2014 +0300

    Fixes terminate_connection live migration issue

    Reverts the changes to cinder/volume/manager.py added in
    commit b868ae707f9ecbe254101e21d9d7ffa0b05b17d1 as calling
    remove_export in terminate_connection causes Nova live
    migration to fail when volumes are attached.

    Conflicts:
            cinder/volume/manager.py

    Change-Id: I38f3db336db22bb096a8855bcbbfeba16e517b80
    Closes-Bug: #1361738
    (cherry picked from ddcad011db507cde66b6b1e655d5ffc91ab8880f)

tags: added: in-stable-icehouse
Thierry Carrez (ttx)
Changed in cinder:
milestone: juno-3 → 2014.2
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.