Live Migration in XenServer fails with duplicated base_mirror key

[Antony Messerli]

Live migration on XenServer (SXM) platforms appears to start failing after a number of attempts with this error on the hypervisor logs:

Failure: ['INTERNAL_ERROR', 'Storage_interface.Internal_error("Storage_interface.Duplicated_key(\\"base_mirror\\")")']

This prevents any future live migrations from occuring. Apparently this is now fixed in xen-api trunk (https://github.com/xapi-project/xen-api/commit/141a8d01373b44c2cc625e2a25b0121cc39110b5) but not before XenServer 6.2 was cut.

Can we look into working around this issue until Citrix has a proper hotfix in place for this error?

[John Garbutt (johngarbutt)]

Makes sense.

[John Garbutt (johngarbutt)]

Looks to have been fixed upstream here:
https://github.com/xapi-project/xen-api/commit/141a8d01373b44c2cc625e2a25b0121cc39110b5

But that is unlikely to be released for a little while, so need something in the meantime.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/49715

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/49743

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/49758

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/49758
Committed: http://github.com/openstack/nova/commit/605455c8be241ddbb0fecae7f2e1b498070debd3
Submitter: Jenkins
Branch: master

commit 605455c8be241ddbb0fecae7f2e1b498070debd3
Author: John Garbutt <email address hidden>
Date: Fri Oct 4 10:33:15 2013 +0100

    xenapi: strip base_mirror after live-migrate

    There is a bug in XenServer CA-98606 that means you get base_mirror key
    left over after a block live-migrate.

    This change ensures to strip out that key after error live migrate, to
    ensure the key is never left around in a way that means the next live
    migrate would fail.

    Fixes bug 1234752
    Change-Id: Icd6ba17039b50ae20d252d7c052120abe7de0deb