Hi, 
   Copy that. Thanks for your meeting-minutes
   Thanks 
                ChengDe. 



> yong hu <email address hidden> 於 2019年7月30日 下午9:58 寫道：
> 
> @chant, please make futher analysis based on the discussion in the
> meeting:
> 
> 1. CPU spec, RAM, any flavor extra information might be relevant, please provide this kind of info.
> 2. looked into the original log and had an analysis why in the first place.
>    - logs in Nova-* containers (pods or services) and 
>    - libvirtd related logs
> 
> -- 
> You received this bug notification because you are a bug assignee.
> https://bugs.launchpad.net/bugs/1836402
> 
> Title:
>  VM fail to live migrate after evacuation
> 
> Status in StarlingX:
>  Triaged
> 
> Bug description:
>  Brief Description
>  -----------------
> 
>                    It was observed after VM was evacuated live
>  migration was failing on same VM. Eg   VM UUID ab3048c4-b10f-49ad-
>  a93d-16dc24c97b93 evacuated successfully from compute-0 to compute-1
>  then fails Live migration any of the other computes from compute-1.
>  Below steps are showed on cli.
> 
>  ysadmin@controller-1 ~(keystone_admin)]$ openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show ab3048c4-b10f-49ad-a93d-16dc24c97b93
>  openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default -- 
>  os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show ab3048c4-b10f-49ad-a93d-16dc24c97b93
> 
>  +-------------------------------------+-------------------------------------------------------------+
> 
>  | Field                               | Value
>  |
> 
>  +-------------------------------------+-------------------------------------------------------------+
> 
>  | OS-DCF:diskConfig                   | MANUAL
>  |
> 
>  | OS-EXT-AZ:availability_zone         | nova
>  |
> 
>  | OS-EXT-SRV-ATTR:host                | compute-0
>  |
> 
>  | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0
>  |
> 
>  | OS-EXT-SRV-ATTR:instance_name       | instance-00000016
>  |
> 
>  | OS-EXT-STS:power_state              | Running
>  |
> 
>  | OS-EXT-STS:task_state               | None
>  |
> 
>  | OS-EXT-STS:vm_state                 | active
>  |
> 
>  | OS-SRV-USG:launched_at              | 2019-07-12T15:23:56.000000
>  |
> 
>  | OS-SRV-USG:terminated_at            | None
>  |
> 
>  | accessIPv4                          |
>  |
> 
>  | accessIPv6                          |
>  |
> 
>  | addresses                           | tenant1-mgmt-
>  net=192.168.146.55; tenant1-net1=172.16.1.159  |
> 
>  | config_drive                        |
>  |
> 
>  | created                             | 2019-07-12T15:23:45Z
>  |
> 
>  | flavor                              | flavor-default-size2 (0c7d0962
>  -bf0c-44fd-a5b9-ff6b9d0a93b3) |
> 
>  | hostId                              |
>  da8f07be2656cda0f4faff2389b7ee95d00879957c5b4bbea77fe622    |
> 
>  | id                                  | ab3048c4-b10f-49ad-a93d-
>  16dc24c97b93                        |
> 
>  | image                               |
>  |
> 
>  | key_name                            | keypair-tenant1
>  |
> 
>  | name                                | tenant1-vm-1
>  |
> 
>  | progress                            | 0
>  |
> 
>  | project_id                          |
>  196a1151724d4d5d8fe6e24bfd4acf5f                            |
> 
>  | properties                          |
>  |
> 
>  | security_groups                     | name='default'
>  |
> 
>  |                                     | name='default'
>  |
> 
>  | status                              | ACTIVE
>  |
> 
>  | updated                             | 2019-07-12T15:23:56Z
>  |
> 
>  | user_id                             |
>  ffd19119f10c4ab1aa99a9608a66923a                            |
> 
>  | volumes_attached                    | id='cedd17da-2480-4e04-999e-
>  2fc3de8274c3'                   |
> 
>  +-------------------------------------+-------------------------------------------------------------+
> 
>  [sysadmin@controller-1 ~(keystone_admin)]$ echo $?
> 
>  compute-0:~$ sudo reboot -f
>  sudo reboot -f
> 
>  Rebooting.
> 
>  packet_write_wait: Connection to 192.168.204.138 port 22: Broken pipe
> 
>  [sysadmin@controller-1 ~(keystone_admin)]$ system --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://192.168.204.2:5000/v3 --os-user-domain-name Default --os-project-domain-name Default  --os-endpoint-type internalURL --os-region-name RegionOne host-list
>  system --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://192.168.204.2:5000/v3 --os-user-domain-name Default --os-project-domain-name Default  --os-endpoint-type int 
>  ernalURL --os-region-name RegionOne host-list
> 
>  +----+--------------+-------------+----------------+-------------+--------------+
> 
>  | id | hostname     | personality | administrative | operational |
>  availability |
> 
>  +----+--------------+-------------+----------------+-------------+--------------+
> 
>  | 1  | controller-0 | controller  | unlocked       | enabled     |
>  available    |
> 
>  | 2  | controller-1 | controller  | unlocked       | enabled     |
>  available    |
> 
>  | 3  | compute-1    | worker      | unlocked       | enabled     |
>  available    |
> 
>  | 4  | compute-2    | worker      | unlocked       | enabled     |
>  available    |
> 
>  | 5  | compute-0    | worker      | unlocked       | disabled    |
>  offline      |
> 
>  | 6  | storage-0    | storage     | unlocked       | enabled     |
>  available    |
> 
>  | 7  | storage-1    | storage     | unlocked       | enabled     |
>  available    |
> 
>  +----+--------------+-------------+----------------+-------------+--------------+
> 
> 
>  +----+--------------+-------------+----------------+-------------+--------------+
> 
>  | id | hostname     | personality | administrative | operational |
>  availability |
> 
>  +----+--------------+-------------+----------------+-------------+--------------+
> 
>  | 1  | controller-0 | controller  | unlocked       | enabled     |
>  available    |
> 
>  | 2  | controller-1 | controller  | unlocked       | enabled     |
>  available    |
> 
>  | 3  | compute-1    | worker      | unlocked       | enabled     |
>  available    |
> 
>  | 4  | compute-2    | worker      | unlocked       | enabled     |
>  available    |
> 
>  | 5  | compute-0    | worker      | unlocked       | enabled     |
>  available    |
> 
>  | 6  | storage-0    | storage     | unlocked       | enabled     |
>  available    |
> 
>  | 7  | storage-1    | storage     | unlocked       | enabled     |
>  available    |
> 
>  +----+--------------+-------------+----------------+-------------+--------------+
>  ---------------------+-------------------------------------------+
> 
>  | Property            | Value                                     |
> 
>  +---------------------+-------------------------------------------+
> 
>  | action              | none                                      |
> 
>  | administrative      | unlocked                                  |
> 
>  | availability        | available                                 |
> 
>  | bm_ip               | 128.224.64.219                            |
> 
>  | bm_type             | bmc                                       |
> 
>  | bm_username         | root                                      |
> 
>  | boot_device         | /dev/disk/by-path/pci-0000:85:00.0-nvme-1 |
> 
>  | capabilities        | {}                                        |
> 
>  | config_applied      | 2dfb83d6-9616-4509-b8d4-e3c204d443ab      |
> 
>  | config_status       | None                                      |
> 
>  | config_target       | 2dfb83d6-9616-4509-b8d4-e3c204d443ab      |
> 
>  | console             | ttyS0,115200                              |
> 
>  | created_at          | 2019-07-11T19:47:49.078714+00:00          |
> 
>  | hostname            | compute-0                                 |
> 
>  | id                  | 5                                         |
> 
>  | install_output      | text                                      |
> 
>  | install_state       | completed                                 |
> 
>  | install_state_info  | None                                      |
> 
>  | invprovision        | provisioned                               |
> 
>  | location            | {}                                        |
> 
>  | mgmt_ip             | 192.168.204.138                           |
> 
>  | mgmt_mac            | 90:e2:ba:c6:90:60                         |
> 
>  | operational         | enabled                                   |
> 
>  | personality         | worker                                    |
> 
>  | reserved            | False                                     |
> 
>  | rootfs_device       | /dev/disk/by-path/pci-0000:85:00.0-nvme-1 |
> 
>  | serialid            | None                                      |
> 
>  | software_load       | 19.01                                     |
> 
>  | subfunctions        | worker,lowlatency                         |
> 
>  | task                |                                           |
> 
>  | tboot               | false                                     |
> 
>  | ttys_dcd            | None                                      |
> 
>  | updated_at          | 2019-07-12T15:30:39.811171+00:00          |
> 
>  | uptime              | 203                                       |
> 
>  | uuid                | fae1ee1f-a5ed-4ae5-b68b-ab3e9711ef24      |
> 
>  | vim_progress_status | services-enabled                          |
> 
>  +---------------------+-------------------------------------------+
> 
> 
>  ---------------------+-------------------------------------------+
> 
>  | Property            | Value                                     |
> 
>  +---------------------+-------------------------------------------+
> 
>  | action              | none                                      |
> 
>  | administrative      | unlocked                                  |
> 
>  | availability        | available                                 |
> 
>  | bm_ip               | 128.224.64.219                            |
> 
>  | bm_type             | bmc                                       |
> 
>  | bm_username         | root                                      |
> 
>  | boot_device         | /dev/disk/by-path/pci-0000:85:00.0-nvme-1 |
> 
>  | capabilities        | {}                                        |
> 
>  | config_applied      | 2dfb83d6-9616-4509-b8d4-e3c204d443ab      |
> 
>  | config_status       | None                                      |
> 
>  | config_target       | 2dfb83d6-9616-4509-b8d4-e3c204d443ab      |
> 
>  | console             | ttyS0,115200                              |
> 
>  | created_at          | 2019-07-11T19:47:49.078714+00:00          |
> 
>  | hostname            | compute-0                                 |
> 
>  | id                  | 5                                         |
> 
>  | install_output      | text                                      |
> 
>  | install_state       | completed                                 |
> 
>  | install_state_info  | None                                      |
> 
>  | invprovision        | provisioned                               |
> 
>  | location            | {}                                        |
> 
>  | mgmt_ip             | 192.168.204.138                           |
> 
>  | mgmt_mac            | 90:e2:ba:c6:90:60                         |
> 
>  | operational         | enabled                                   |
> 
>  | personality         | worker                                    |
> 
>  | reserved            | False                                     |
> 
>  | rootfs_device       | /dev/disk/by-path/pci-0000:85:00.0-nvme-1 |
> 
>  | serialid            | None                                      |
> 
>  | software_load       | 19.01                                     |
> 
>  | subfunctions        | worker,lowlatency                         |
> 
>  | task                |                                           |
> 
>  | tboot               | false                                     |
> 
>  | ttys_dcd            | None                                      |
> 
>  | updated_at          | 2019-07-12T15:30:39.811171+00:00          |
> 
>  | uptime              | 203                                       |
> 
>  | uuid                | fae1ee1f-a5ed-4ae5-b68b-ab3e9711ef24      |
> 
>  | vim_progress_status | services-enabled                          |
> 
>  +---------------------+-------------------------------------------+
> 
> 
>  [sysadmin@controller-1 ~(keystone_admin)]$ nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default  --os-endpoint-type internalURL --os-region-name RegionOne live-migration ab3048c4-b10f-49ad-a93d-16dc24c97b93 compute-0
>  nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default  --os-e 
>  ndpoint-type internalURL --os-region-name RegionOne live-migration ab3048c4-b10f-49ad-a93d-16dc24c97b93 compute-0
> 
>  sysadmin@controller-1 ~(keystone_admin)]$ openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show ab3048c4-b10f-49ad-a93d-16dc24c97b93
>  openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default -- 
>  os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show ab3048c4-b10f-49ad-a93d-16dc24c97b93
> 
>  +-------------------------------------+-------------------------------------------------------------+
> 
>  | Field                               | Value
>  |
> 
>  +-------------------------------------+-------------------------------------------------------------+
> 
>  | OS-DCF:diskConfig                   | MANUAL
>  |
> 
>  | OS-EXT-AZ:availability_zone         | nova
>  |
> 
>  | OS-EXT-SRV-ATTR:host                | compute-1
>  |
> 
>  | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1
>  |
> 
>  | OS-EXT-SRV-ATTR:instance_name       | instance-00000016
>  |
> 
>  | OS-EXT-STS:power_state              | Running
>  |
> 
>  | OS-EXT-STS:task_state               | None
>  |
> 
>  | OS-EXT-STS:vm_state                 | active
>  |
> 
>  | OS-SRV-USG:launched_at              | 2019-07-12T15:24:52.000000
>  |
> 
>  | OS-SRV-USG:terminated_at            | None
>  |
> 
>  | accessIPv4                          |
>  |
> 
>  | accessIPv6                          |
>  |
> 
>  | addresses                           | tenant1-mgmt-
>  net=192.168.146.55; tenant1-net1=172.16.1.159  |
> 
>  | config_drive                        |
>  |
> 
>  | created                             | 2019-07-12T15:23:45Z
>  |
> 
>  | flavor                              | flavor-default-size2 (0c7d0962
>  -bf0c-44fd-a5b9-ff6b9d0a93b3) |
> 
>  | hostId                              |
>  a223412c30650d0bbf5df2bd441a5a4828cf49b1020d63efb12cf90c    |
> 
>  | id                                  | ab3048c4-b10f-49ad-a93d-
>  16dc24c97b93                        |
> 
>  | image                               |
>  |
> 
>  | key_name                            | keypair-tenant1
>  |
> 
>  | name                                | tenant1-vm-1
>  |
> 
>  | progress                            | 0
>  |
> 
>  | project_id                          |
>  196a1151724d4d5d8fe6e24bfd4acf5f                            |
> 
>  | properties                          |
>  |
> 
>  | security_groups                     | name='default'
>  |
> 
>  |                                     | name='default'
>  |
> 
>  | status                              | ACTIVE
>  |
> 
>  | updated                             | 2019-07-12T15:32:01Z
>  |
> 
>  | user_id                             |
>  ffd19119f10c4ab1aa99a9608a66923a                            |
> 
>  | volumes_attached                    | id='cedd17da-2480-4e04-999e-
>  2fc3de8274c3'                   |
> 
>  +-------------------------------------+-------------------------------------------------------------
> 
>   [sysadmin@controller-1 ~(keystone_admin)]$ openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a
>  openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default -- 
>  os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a
> 
>  +--------------------------------------+--------------+--------+------------------------------------------------------------+-------+----------------------+
> 
>  | ID                                   | Name         | Status |
>  Networks                                                   | Image |
>  Flavor               |
> 
>  +--------------------------------------+--------------+--------+------------------------------------------------------------+-------+----------------------+
> 
>  | ab3048c4-b10f-49ad-a93d-16dc24c97b93 | tenant1-vm-1 | ACTIVE |
>  tenant1-mgmt-net=192.168.146.55; tenant1-net1=172.16.1.159 |       |
>  flavor-default-size2 |
> 
>  +--------------------------------------+--------------+--------+------------------------------------------------------------+-------+----------------------+
> 
> 
> 
>  ]$ nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default  --os-endpoint-type internalURL --os-region-name RegionOne migration-list
>  nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default  --os-e 
>  ndpoint-type internalURL --os-region-name RegionOne migration-list
> 
>  +----+--------------------------------------+-------------+-----------+----------------+--------------+-----------------+-----------+--------------------------------------+------------+------------+----------------------------+----------------------------+----------------+
> 
>  | Id | UUID                                 | Source Node | Dest Node
>  | Source Compute | Dest Compute | Dest Host       | Status    |
>  Instance UUID                        | Old Flavor | New Flavor |
>  Created At                 | Updated At                 | Type
>  |
> 
>  +----+--------------------------------------+-------------+-----------+----------------+--------------+-----------------+-----------+--------------------------------------+------------+------------+----------------------------+----------------------------+----------------+
> 
>  | 31 | 3fc2d997-3c9c-44f6-8991-39ad3c87ed12 | -           | -
>  | compute-1      | -            | -               | error     |
>  ab3048c4-b10f-49ad-a93d-16dc24c97b93 | 46         | 46         |
>  2019-07-12T15:31:59.000000 | 2019-07-12T15:32:01.000000 | live-
>  migration |
> 
>  | 28 | 5e82192b-f11f-4829-8a61-6ee196ecbb32 | compute-0   | compute-1
>  | compute-0      | compute-1    | 192.168.205.236 | completed |
>  ab3048c4-b10f-49ad-a93d-16dc24c97b93 | None       | None       |
>  2019-07-12T15:24:34.000000 | 2019-07-12T15:30:34.000000 | evacuation
>  |
> 
>  +----+--------------------------------------+-------------+-----------+----------------+--------------+-----------------+-----------+--------------------------------------+------------+------------+----------------------------+----------------------------+----------------+
> 
>  [sysadmin@controller-1 ~(keystone_admin)]$ echo $?
> 
>  Severity
>  --------
>  Major
>  Steps to Reproduce
>  ------------------
>  1. Launch VM . VM was created in compute-0
>  n)]$ openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show ab3048c4-b10f-49ad-a93d-16dc24c97b93
>  openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default -- 
>  os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show ab3048c4-b10f-49ad-a93d-16dc24c97b93
> 
>  +-------------------------------------+-------------------------------------------------------------+
> 
>  | Field                               | Value
>  |
> 
>  +-------------------------------------+-------------------------------------------------------------+
> 
>  | OS-DCF:diskConfig                   | MANUAL
>  |
> 
>  | OS-EXT-AZ:availability_zone         | nova
>  |
> 
>  | OS-EXT-SRV-ATTR:host                | compute-0
>  |
> 
>  | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0
>  |
> 
>  | OS-EXT-SRV-ATTR:instance_name       | instance-00000016
>  |
> 
>  | OS-EXT-STS:power_state              | Running
>  |
> 
>  | OS-EXT-STS:task_state               | None
>  |
> 
>  | OS-EXT-STS:vm_state                 | active
>  |
> 
>  | OS-SRV-USG:launched_at              | 2019-07-12T15:23:56.000000
>  |
> 
>  | OS-SRV-USG:terminated_at            | None
>  |
> 
>  | accessIPv4                          |
>  |
> 
>  | accessIPv6                          |
>  |
> 
>  | addresses                           | tenant1-mgmt-
>  net=192.168.146.55; tenant1-net1=172.16.1.159  |
> 
>  | config_drive                        |
>  |
> 
>  | created                             | 2019-07-12T15:23:45Z
>  |
> 
>  | flavor                              | flavor-default-size2 (0c7d0962
>  -bf0c-44fd-a5b9-ff6b9d0a93b3) |
> 
>  | hostId                              |
>  da8f07be2656cda0f4faff2389b7ee95d00879957c5b4bbea77fe622    |
> 
>  | id                                  | ab3048c4-b10f-49ad-a93d-
>  16dc24c97b93                        |
> 
>  | image                               |
>  |
> 
>  | key_name                            | keypair-tenant1
>  |
> 
>  | name                                | tenant1-vm-1
>  |
> 
>  | progress                            | 0
>  |
> 
>  | project_id                          |
>  196a1151724d4d5d8fe6e24bfd4acf5f                            |
> 
>  | properties                          |
>  |
> 
>  | security_groups                     | name='default'
>  |
> 
>  |                                     | name='default'
>  |
> 
>  | status                              | ACTIVE
>  |
> 
>  | updated                             | 2019-07-12T15:23:56Z
>  |
> 
>  | user_id                             |
>  ffd19119f10c4ab1aa99a9608a66923a                            |
> 
>  | volumes_attached                    | id='cedd17da-2480-4e04-999e-
>  2fc3de8274c3'                   |
> 
>  +-------------------------------------+-------------------------------------------------------------+
>  2. Reboot compute-0 sudo reboot –f
>  3. wait for compute  to come up.
>  +----+--------------+-------------+----------------+-------------+--------------+
> 
>  | id | hostname     | personality | administrative | operational |
>  availability |
> 
>  +----+--------------+-------------+----------------+-------------+--------------+
> 
>  | 1  | controller-0 | controller  | unlocked       | enabled     |
>  available    |
> 
>  | 2  | controller-1 | controller  | unlocked       | enabled     |
>  available    |
> 
>  | 3  | compute-1    | worker      | unlocked       | enabled     |
>  available    |
> 
>  | 4  | compute-2    | worker      | unlocked       | enabled     |
>  available    |
> 
>  | 5  | compute-0    | worker      | unlocked       | disabled    |
>  offline      |
> 
>  | 6  | storage-0    | storage     | unlocked       | enabled     |
>  available    |
> 
>  | 7  | storage-1    | storage     | unlocked       | enabled     |
>  available    |
> 
>  +----+--------------+-------------+----------------+-------------+--------------+
> 
> 
>  +----+--------------+-------------+----------------+-------------+--------------+
> 
>  | id | hostname     | personality | administrative | operational |
>  availability |
> 
>  +----+--------------+-------------+----------------+-------------+--------------+
> 
>  | 1  | controller-0 | controller  | unlocked       | enabled     |
>  available    |
> 
>  | 2  | controller-1 | controller  | unlocked       | enabled     |
>  available    |
> 
>  | 3  | compute-1    | worker      | unlocked       | enabled     |
>  available    |
> 
>  | 4  | compute-2    | worker      | unlocked       | enabled     |
>  available    |
> 
>  | 5  | compute-0    | worker      | unlocked       | enabled     |
>  available    |
> 
>  | 6  | storage-0    | storage     | unlocked       | enabled     |
>  available    |
> 
>  | 7  | storage-1    | storage     | unlocked       | enabled     |
>  available    |
> 
>  +----+--------------+-------------+----------------+-------------+--------------+
> 
>  4.Verify VM evacuation VM running on compute-1
> 
>  openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show ab3048c4-b10f-49ad-a93d-16dc24c97b93
>  openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --  os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show ab3048c4-b10f-49ad-a93d-16dc24c97b93 
>  +-------------------------------------+-------------------------------------------------------------+ 
>  | Field                               | Value                                                       | 
>  +-------------------------------------+-------------------------------------------------------------+ 
>  | OS-DCF:diskConfig                   | MANUAL                                                      | 
>  | OS-EXT-AZ:availability_zone         | nova                                                        | 
>  | OS-EXT-SRV-ATTR:host                | compute-1                                                   | 
>  | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1                                                   | 
>  | OS-EXT-SRV-ATTR:instance_name       | instance-00000016                                           | 
>  | OS-EXT-STS:power_state              | Running                                                     | 
>  | OS-EXT-STS:task_state               | None                                                        | 
>  | OS-EXT-STS:vm_state                 | active                                                      | 
>  | OS-SRV-USG:launched_at              | 2019-07-12T15:24:52.000000                                  | 
>  | OS-SRV-USG:terminated_at            | None                                                        | 
>  | accessIPv4                          |                                                             | 
>  | accessIPv6                          |                                                             | 
>  | addresses                           | tenant1-mgmt-net=192.168.146.55; tenant1-net1=172.16.1.159  | 
>  | config_drive                        |                                                             | 
>  | created                             | 2019-07-12T15:23:45Z                                        | 
>  | flavor                              | flavor-default-size2 (0c7d0962-bf0c-44fd-a5b9-ff6b9d0a93b3) | 
>  | hostId                              | a223412c30650d0bbf5df2bd441a5a4828cf49b1020d63efb12cf90c    | 
>  | id                                  | ab3048c4-b10f-49ad-a93d-16dc24c97b93                        | 
>  | image                               |                                                             | 
>  | key_name                            | keypair-tenant1                                             | 
>  | name                                | tenant1-vm-1                                                | 
>  | progress                            | 0                                                           | 
>  | project_id                          | 196a1151724d4d5d8fe6e24bfd4acf5f                            | 
>  | properties                          |                                                             | 
>  | security_groups                     | name='default'                                              | 
>  |                                     | name='default'                                              | 
>  | status                              | ACTIVE                                                      | 
>  | updated                             | 2019-07-12T15:30:33Z                                        | 
>  | user_id                             | ffd19119f10c4ab1aa99a9608a66923a                            | 
>  | volumes_attached                    | id='cedd17da-2480-4e04-999e-2fc3de8274c3'                   | 
>  +-------------------------------------+-------------------------------------------------------------+ 
>  [sysadmin@controller-1 ~(keystone_admin)]$ echo $?
> 
> 
> 
>  5. Livemigrate VM after evacute
> 
>  [sysadmin@controller-1 ~(keystone_admin)]$ nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default  --os-endpoint-type internalURL --os-region-name RegionOne live-migration ab3048c4-b10f-49ad-a93d-16dc24c97b93 compute-0
>  nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default  --os-e 
>  ndpoint-type internalURL --os-region-name RegionOne live-migration ab3048c4-b10f-49ad-a93d-16dc24c97b93 compute-0
> 
>  [sysadmin@controller-1 ~(keystone_admin)]$ echo $?
>  echo $?
> 
>  0
> 
>  [sysadmin@controller-1 ~(keystone_admin)]$ openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show ab3048c4-b10f-49ad-a93d-16dc24c97b93
>  openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default -- 
>  os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show ab3048c4-b10f-49ad-a93d-16dc24c97b93
>  openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show ab3048c4-b10f-49ad-a93d-16dc24c97b93
>  openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default -- 
>  os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show ab3048c4-b10f-49ad-a93d-16dc24c97b93
> 
>  +-------------------------------------+-------------------------------------------------------------+
> 
>  | Field                               | Value
>  |
> 
>  +-------------------------------------+-------------------------------------------------------------+
> 
>  | OS-DCF:diskConfig                   | MANUAL
>  |
> 
>  | OS-EXT-AZ:availability_zone         | nova
>  |
> 
>  | OS-EXT-SRV-ATTR:host                | compute-1
>  |
> 
>  | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1
>  |
> 
>  | OS-EXT-SRV-ATTR:instance_name       | instance-00000016
>  |
> 
>  | OS-EXT-STS:power_state              | Running
>  |
> 
>  | OS-EXT-STS:task_state               | migrating
>  |
> 
>  | OS-EXT-STS:vm_state                 | active
>  |
> 
>  | OS-SRV-USG:launched_at              | 2019-07-12T15:24:52.000000
>  |
> 
>  | OS-SRV-USG:terminated_at            | None
>  |
> 
>  | accessIPv4                          |
>  |
> 
>  | accessIPv6                          |
>  |
> 
>  | addresses                           | tenant1-mgmt-
>  net=192.168.146.55; tenant1-net1=172.16.1.159  |
> 
>  | config_drive                        |
>  |
> 
>  | created                             | 2019-07-12T15:23:45Z
>  |
> 
>  | flavor                              | flavor-default-size2 (0c7d0962
>  -bf0c-44fd-a5b9-ff6b9d0a93b3) |
> 
>  | hostId                              |
>  a223412c30650d0bbf5df2bd441a5a4828cf49b1020d63efb12cf90c    |
> 
>  | id                                  | ab3048c4-b10f-49ad-a93d-
>  16dc24c97b93                        |
> 
>  | image                               |
>  |
> 
>  | key_name                            | keypair-tenant1
>  |
> 
>  | name                                | tenant1-vm-1
>  |
> 
>  | progress                            | 0
>  |
> 
>  | project_id                          |
>  196a1151724d4d5d8fe6e24bfd4acf5f                            |
> 
>  | properties                          |
>  |
> 
>  | security_groups                     | name='default'
>  |
> 
>  |                                     | name='default'
>  |
> 
>  | status                              | MIGRATING
>  |
> 
>  | updated                             | 2019-07-12T15:31:59Z
>  |
> 
>  | user_id                             |
>  ffd19119f10c4ab1aa99a9608a66923a                            |
> 
>  | volumes_attached                    | id='cedd17da-2480-4e04-999e-
>  2fc3de8274c3'                   |
> 
>  	
>  openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show ab3048c4-b10f-49ad-a93d-16dc24c97b93
>  openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default -- 
>  os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show ab3048c4-b10f-49ad-a93d-16dc24c97b93
> 
>  +-------------------------------------+-------------------------------------------------------------+
> 
>  | Field                               | Value
>  |
> 
>  +-------------------------------------+-------------------------------------------------------------+
> 
>  | OS-DCF:diskConfig                   | MANUAL
>  |
> 
>  | OS-EXT-AZ:availability_zone         | nova
>  |
> 
>  | OS-EXT-SRV-ATTR:host                | compute-1
>  |
> 
>  | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1
>  |
> 
>  | OS-EXT-SRV-ATTR:instance_name       | instance-00000016
>  |
> 
>  | OS-EXT-STS:power_state              | Running
>  |
> 
>  | OS-EXT-STS:task_state               | None
>  |
> 
>  | OS-EXT-STS:vm_state                 | active
>  |
> 
>  | OS-SRV-USG:launched_at              | 2019-07-12T15:24:52.000000
>  |
> 
>  | OS-SRV-USG:terminated_at            | None
>  |
> 
>  | accessIPv4                          |
>  |
> 
>  | accessIPv6                          |
>  |
> 
>  | addresses                           | tenant1-mgmt-
>  net=192.168.146.55; tenant1-net1=172.16.1.159  |
> 
>  | config_drive                        |
>  |
> 
>  | created                             | 2019-07-12T15:23:45Z
>  |
> 
>  | flavor                              | flavor-default-size2 (0c7d0962
>  -bf0c-44fd-a5b9-ff6b9d0a93b3) |
> 
>  | hostId                              |
>  a223412c30650d0bbf5df2bd441a5a4828cf49b1020d63efb12cf90c    |
> 
>  | id                                  | ab3048c4-b10f-49ad-a93d-
>  16dc24c97b93                        |
> 
>  | image                               |
>  |
> 
>  | key_name                            | keypair-tenant1
>  |
> 
>  | name                                | tenant1-vm-1
>  |
> 
>  | progress                            | 0
>  |
> 
>  | project_id                          |
>  196a1151724d4d5d8fe6e24bfd4acf5f                            |
> 
>  | properties                          |
>  |
> 
>  | security_groups                     | name='default'
>  |
> 
>  |                                     | name='default'
>  |
> 
>  | status                              | ACTIVE
>  |
> 
>  | updated                             | 2019-07-12T15:32:01Z
>  |
> 
>  | user_id                             |
>  ffd19119f10c4ab1aa99a9608a66923a                            |
> 
>  | volumes_attached                    | id='cedd17da-2480-4e04-999e-
>  2fc3de8274c3'                   |
> 
>  +-------------------------------------+-------------------------------------------------------------
> 
>  6.Migration list shows error to migrate from  compute-1
> 
> 
>  nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default  --os-endpoint-type internalURL --os-region-name RegionOne migration-list
>  nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default  --os-e 
>  ndpoint-type internalURL --os-region-name RegionOne migration-list
> 
>  +----+--------------------------------------+-------------+-----------+----------------+--------------+-----------------+-----------+--------------------------------------+------------+------------+----------------------------+----------------------------+----------------+
> 
>  | Id | UUID                                 | Source Node | Dest Node
>  | Source Compute | Dest Compute | Dest Host       | Status    |
>  Instance UUID                        | Old Flavor | New Flavor |
>  Created At                 | Updated At                 | Type
>  |
> 
>  +----+--------------------------------------+-------------+-----------+----------------+--------------+-----------------+-----------+--------------------------------------+------------+------------+----------------------------+----------------------------+----------------+
> 
>  | 31 | 3fc2d997-3c9c-44f6-8991-39ad3c87ed12 | -           | -
>  | compute-1      | -            | -               | error     |
>  ab3048c4-b10f-49ad-a93d-16dc24c97b93 | 46         | 46         |
>  2019-07-12T15:31:59.000000 | 2019-07-12T15:32:01.000000 | live-
>  migration |
> 
>  | 28 | 5e82192b-f11f-4829-8a61-6ee196ecbb32 | compute-0   | compute-1
>  | compute-0      | compute-1    | 192.168.205.236 | completed |
>  ab3048c4-b10f-49ad-a93d-16dc24c97b93 | None       | None       |
>  2019-07-12T15:24:34.000000 | 2019-07-12T15:30:34.000000 | evacuation
>  |
> 
>  +----+--------------------------------------+-------------+-----------+----------------+--------------+-----------------+-----------+--------------------------------------+------------+------------+----------------------------+----------------------------+----------------+
> 
>  System Configuration
>  --------------------
>  Regular system
> 
> 
>  Expected Behavior
>  ------------------
>  Able to migrate after evacuate. 
>  Actual Behavior
>  ----------------
>  Failure on migration after evacuate.
>  Reproducibility
>  ---------------
>  100% reproducible
>  System Configuration
>  --------------------
>  Regular system
>  Load 
>  ----
>  2019-07-11 01:30:00 +0000
>  Last Pass
>  ---------
>  Not sure in this lab.
> 
>  Timestamp/Logs
>  --------------
>  2019-07-12T15:31:59.000000
> 
>  Test Activity
>  -------------
>  Regression test
> 
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/starlingx/+bug/1836402/+subscriptions