Live migration to different host returns complete but nothing has happened

[Facundo Ciccioli]

I created a server group with anti-affinity policy:

$ os server group create test-group --policy anti-affinity -fvalue -cid
10573804-7127-426d-be6b-409fb5968735

Created four servers as members:

$ os server create --flavor $FLAVOR \
                   --image $IMAGE \
                   --network $NET \
                   --hint group=10573804-7127-426d-be6b-409fb5968735 \
                   --max 4 \
                   test-server

Chose one server randomly and tried to live-migrate it to a host hosting another member of the group:

$ os server group show 10573804-7127-426d-be6b-409fb5968735 -fvalue -cmembers | \
     tr -d ' ' | tr , '\n' | \
     xargs -n1 -P0 openstack server show -fvalue -cOS-EXT-SRV-ATTR:host
HOST1
HOST2
HOST3
HOST4

$ os server show 2edd275c-9f60-4191-a295-aaa0354f27b2 -fvalue -cOS-EXT-SRV-ATTR:host
HOST1

$os --os-compute-api-version 2.60 \
    server migrate --host HOST2 \
                   --live-migration \
                   --wait \
                   2edd275c-9f60-4191-a295-aaa0354f27b2
Complete

$ os server show 2edd275c-9f60-4191-a295-aaa0354f27b2 -fvalue -cOS-EXT-SRV-ATTR:host
HOST1

This is fairly misleading since nothing actually happens (rightfully), but the migrate commands doesn't give any indication of that fact.

The environment:

$ dpkg -l | grep nova
ii python-novaclient 2:3.3.1-2ubuntu1 all client library for OpenStack Compute API - Python 2.7
ii python3-novaclient 2:3.3.1-2ubuntu1 all client library for OpenStack Compute API - 3.x

Using QEMU hypervisor with KVM enabled:

$ qemu-system-x86_64 --version
QEMU emulator version 2.11.1(Debian 1:2.11+dfsg-1ubuntu7.15~cloud1)

Nova versions on the compute:

$ dpkg-query -l | grep nova
ii nova-common 2:17.0.10-0ubuntu2~cloud0 all OpenStack Compute - common files
ii nova-compute 2:17.0.10-0ubuntu2~cloud0 all OpenStack Compute - compute node base
ii nova-compute-kvm 2:17.0.10-0ubuntu2~cloud0 all OpenStack Compute - compute node (KVM)
ii nova-compute-libvirt 2:17.0.10-0ubuntu2~cloud0 all OpenStack Compute - compute node libvirt support
ii python-nova 2:17.0.10-0ubuntu2~cloud0 all OpenStack Compute Python libraries
ii python-novaclient 2:9.1.1-0ubuntu1~cloud0 all client library for OpenStack Compute API - Python 2.7

[Lee Yarwood (lyarwood)]

What is the actual state of the migration?

This smells more like an openstackclient bug than a nova bug if I'm honest.

$ openstack server migration list --instance 2edd275c-9f60-4191-a295-aaa0354f27b2

https://docs.openstack.org/python-openstackclient/latest/cli/command-objects/server-migration.html#server-migration-list

[Lee Yarwood (lyarwood)]

Apologies the actual command is:

$ openstack server migration list --server 2edd275c-9f60-4191-a295-aaa0354f27b2

[Stephen Finucane (stephenfinucane)]

Yeah, this does feel like an openstackclient (OSC) bug. For reference, OSC is checking if the server is in either the 'active' or 'verify_resize' state to determine whether the migration is complete [1]. The inclusion of the 'active' state is necessary to handle the auto-confirmation of migrations that can be configured by setting the 'resize_confirm_window' config option to a positive value in 'nova.conf'. Clearly this is flawed, as the server would remain in this state if the migration failed. We would be wiser to check server events or the migration record, though of course this could have impacts for policy.

Let's wait to see what the reporter suggests here, but assuming the migration records show a failed migration, then this should be moved to OSC.

[1] https://github.com/openstack/python-openstackclient/blob/6905e9756/openstackclient/compute/v2/server.py#L2279-L2284

[Launchpad Janitor (janitor)]

[Expired for OpenStack Compute (nova) because there has been no activity for 60 days.]