live-migration fails when FC multipath is used

Bug #1327497 reported by Jeegn Chen
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Undecided
Jeegn Chen
Icehouse
Fix Released
Undecided
ChangBo Guo(gcb)

Bug Description

I tried live-migration against VM with multipath access to FC bootable volume and FC data volume.
After checking the code, I found the reason is that
1. /dev/dm-<NUM> is used, which is subject to change in the destination Compute Node since it is not unique across nodes
2. multipath_id in connnection_info is not maintained properly and may be lost during connection refreshing

The fix would be
1. Like iSCSI multipath, use /dev/mapper/<multipath_id> instead of /dev/dm-<NUM>
2. Since multipath_id is unique for a volume no matter where it is attached, add logic to preserve this information.

Jeegn Chen (jeegn-chen)
Changed in nova:
assignee: nobody → Jeegn Chen (jeegn-chen)
status: New → In Progress
description: updated
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/98738

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on nova (master)

Change abandoned by Jeegn Chen (<email address hidden>) on branch: master
Review: https://review.openstack.org/98738
Reason: Some changes are missing. Need revise

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/98738
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=3ea14e8a70a946dbb162ecafa848e4f2fa29772a
Submitter: Jenkins
Branch: master

commit 3ea14e8a70a946dbb162ecafa848e4f2fa29772a
Author: Jeegn Chen <email address hidden>
Date: Sun Jun 8 16:23:36 2014 +0800

    Fix live-migration failure in FC multipath case

    Currently, /dev/dm-<NUM> instead of /dev/mapper/<multipath_id> is
    used to access multipath FC volumes by Compute Node and
    multipath_id in connection_info is not maintained properly and
    may be lost during connection refreshing.

    This implementation will make source Compute Node and destination
    Compute Node fail to disconnect/connect to volumes properly and
    result in live-migration failure.

    To fix it, /dev/mapper<multipath_id> will be used instead of
    /dev/dm-<NUM> to access multipath devices, just like iSCSI multipath
    implementation, and logic to preserve the unique (across Compute
    Nodes) multipath_id is also added.

    Change-Id: I17f15852c098af88afd270084c62eb87693c60d4
    Closes-Bug: #1327497

Changed in nova:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in nova:
milestone: none → juno-3
status: Fix Committed → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/icehouse)

Fix proposed to branch: stable/icehouse
Review: https://review.openstack.org/123056

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/icehouse)

Reviewed: https://review.openstack.org/123056
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=74e0ba7e658fcd2c6d1b7a92dcee564098d0a1ff
Submitter: Jenkins
Branch: stable/icehouse

commit 74e0ba7e658fcd2c6d1b7a92dcee564098d0a1ff
Author: Jeegn Chen <email address hidden>
Date: Sun Jun 8 16:23:36 2014 +0800

    Fix live-migration failure in FC multipath case

    Currently, /dev/dm-<NUM> instead of /dev/mapper/<multipath_id> is
    used to access multipath FC volumes by Compute Node and
    multipath_id in connection_info is not maintained properly and
    may be lost during connection refreshing.

    This implementation will make source Compute Node and destination
    Compute Node fail to disconnect/connect to volumes properly and
    result in live-migration failure.

    To fix it, /dev/mapper<multipath_id> will be used instead of
    /dev/dm-<NUM> to access multipath devices, just like iSCSI multipath
    implementation, and logic to preserve the unique (across Compute
    Nodes) multipath_id is also added.

    Closes-Bug: #1327497
    (cherry picked from commit 3ea14e8a70a946dbb162ecafa848e4f2fa29772a)

    Conflicts:
     nova/storage/linuxscsi.py
     nova/tests/virt/libvirt/test_libvirt_volume.py
     nova/virt/block_device.py
     nova/virt/libvirt/volume.py

    This backport commit adjust oslo.i18n usage to oslo-incubator common code,
    due to we didn't have oslo.i18n in icehouse.
    And remove unused variable value dev_str in test_libvirt_volume.py,
    it should be deleted but not worth a specific commit in stable/icehouse.

    Change-Id: I17f15852c098af88afd270084c62eb87693c60d4

tags: added: in-stable-icehouse
Thierry Carrez (ttx)
Changed in nova:
milestone: juno-3 → 2014.2
Alan Pevec (apevec)
tags: removed: in-stable-icehouse
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.