VM fail to live migrate after evacuation
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
StarlingX |
Fix Released
|
High
|
YU CHENGDE |
Bug Description
Brief Description
-----------------
ysadmin@
openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://
os-identity-
+------
| Field | Value |
+------
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-
| OS-EXT-
| OS-EXT-
| OS-EXT-
| OS-EXT-
| OS-EXT-
| OS-EXT-STS:vm_state | active |
| OS-SRV-
| OS-SRV-
| accessIPv4 | |
| accessIPv6 | |
| addresses | tenant1-
| config_drive | |
| created | 2019-07-
| flavor | flavor-
| hostId | da8f07be2656cda
| id | ab3048c4-
| image | |
| key_name | keypair-tenant1 |
| name | tenant1-vm-1 |
| progress | 0 |
| project_id | 196a1151724d4d5
| properties | |
| security_groups | name='default' |
| | name='default' |
| status | ACTIVE |
| updated | 2019-07-
| user_id | ffd19119f10c4ab
| volumes_attached | id='cedd17da-
+------
[sysadmin@
compute-0:~$ sudo reboot -f
sudo reboot -f
Rebooting.
packet_write_wait: Connection to 192.168.204.138 port 22: Broken pipe
[sysadmin@
system --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://
ernalURL --os-region-name RegionOne host-list
+----+-
| id | hostname | personality | administrative | operational | availability |
+----+-
| 1 | controller-0 | controller | unlocked | enabled | available |
| 2 | controller-1 | controller | unlocked | enabled | available |
| 3 | compute-1 | worker | unlocked | enabled | available |
| 4 | compute-2 | worker | unlocked | enabled | available |
| 5 | compute-0 | worker | unlocked | disabled | offline |
| 6 | storage-0 | storage | unlocked | enabled | available |
| 7 | storage-1 | storage | unlocked | enabled | available |
+----+-
+----+-
| id | hostname | personality | administrative | operational | availability |
+----+-
| 1 | controller-0 | controller | unlocked | enabled | available |
| 2 | controller-1 | controller | unlocked | enabled | available |
| 3 | compute-1 | worker | unlocked | enabled | available |
| 4 | compute-2 | worker | unlocked | enabled | available |
| 5 | compute-0 | worker | unlocked | enabled | available |
| 6 | storage-0 | storage | unlocked | enabled | available |
| 7 | storage-1 | storage | unlocked | enabled | available |
+----+-
-------
| Property | Value |
+------
| action | none |
| administrative | unlocked |
| availability | available |
| bm_ip | 128.224.64.219 |
| bm_type | bmc |
| bm_username | root |
| boot_device | /dev/disk/
| capabilities | {} |
| config_applied | 2dfb83d6-
| config_status | None |
| config_target | 2dfb83d6-
| console | ttyS0,115200 |
| created_at | 2019-07-
| hostname | compute-0 |
| id | 5 |
| install_output | text |
| install_state | completed |
| install_state_info | None |
| invprovision | provisioned |
| location | {} |
| mgmt_ip | 192.168.204.138 |
| mgmt_mac | 90:e2:ba:c6:90:60 |
| operational | enabled |
| personality | worker |
| reserved | False |
| rootfs_device | /dev/disk/
| serialid | None |
| software_load | 19.01 |
| subfunctions | worker,lowlatency |
| task | |
| tboot | false |
| ttys_dcd | None |
| updated_at | 2019-07-
| uptime | 203 |
| uuid | fae1ee1f-
| vim_progress_status | services-enabled |
+------
-------
| Property | Value |
+------
| action | none |
| administrative | unlocked |
| availability | available |
| bm_ip | 128.224.64.219 |
| bm_type | bmc |
| bm_username | root |
| boot_device | /dev/disk/
| capabilities | {} |
| config_applied | 2dfb83d6-
| config_status | None |
| config_target | 2dfb83d6-
| console | ttyS0,115200 |
| created_at | 2019-07-
| hostname | compute-0 |
| id | 5 |
| install_output | text |
| install_state | completed |
| install_state_info | None |
| invprovision | provisioned |
| location | {} |
| mgmt_ip | 192.168.204.138 |
| mgmt_mac | 90:e2:ba:c6:90:60 |
| operational | enabled |
| personality | worker |
| reserved | False |
| rootfs_device | /dev/disk/
| serialid | None |
| software_load | 19.01 |
| subfunctions | worker,lowlatency |
| task | |
| tboot | false |
| ttys_dcd | None |
| updated_at | 2019-07-
| uptime | 203 |
| uuid | fae1ee1f-
| vim_progress_status | services-enabled |
+------
[sysadmin@
nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://
ndpoint-type internalURL --os-region-name RegionOne live-migration ab3048c4-
sysadmin@
openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://
os-identity-
+------
| Field | Value |
+------
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-
| OS-EXT-
| OS-EXT-
| OS-EXT-
| OS-EXT-
| OS-EXT-
| OS-EXT-STS:vm_state | active |
| OS-SRV-
| OS-SRV-
| accessIPv4 | |
| accessIPv6 | |
| addresses | tenant1-
| config_drive | |
| created | 2019-07-
| flavor | flavor-
| hostId | a223412c30650d0
| id | ab3048c4-
| image | |
| key_name | keypair-tenant1 |
| name | tenant1-vm-1 |
| progress | 0 |
| project_id | 196a1151724d4d5
| properties | |
| security_groups | name='default' |
| | name='default' |
| status | ACTIVE |
| updated | 2019-07-
| user_id | ffd19119f10c4ab
| volumes_attached | id='cedd17da-
+------
[sysadmin@
openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://
os-identity-
+------
| ID | Name | Status | Networks | Image | Flavor |
+------
| ab3048c4-
+------
]$ nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://
nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://
ndpoint-type internalURL --os-region-name RegionOne migration-list
+----+-
| Id | UUID | Source Node | Dest Node | Source Compute | Dest Compute | Dest Host | Status | Instance UUID | Old Flavor | New Flavor | Created At | Updated At | Type |
+----+-
| 31 | 3fc2d997-
| 28 | 5e82192b-
+----+-
[sysadmin@
Severity
--------
Major
Steps to Reproduce
------------------
1. Launch VM . VM was created in compute-0
n)]$ openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://
openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://
os-identity-
+------
| Field | Value |
+------
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-
| OS-EXT-
| OS-EXT-
| OS-EXT-
| OS-EXT-
| OS-EXT-
| OS-EXT-STS:vm_state | active |
| OS-SRV-
| OS-SRV-
| accessIPv4 | |
| accessIPv6 | |
| addresses | tenant1-
| config_drive | |
| created | 2019-07-
| flavor | flavor-
| hostId | da8f07be2656cda
| id | ab3048c4-
| image | |
| key_name | keypair-tenant1 |
| name | tenant1-vm-1 |
| progress | 0 |
| project_id | 196a1151724d4d5
| properties | |
| security_groups | name='default' |
| | name='default' |
| status | ACTIVE |
| updated | 2019-07-
| user_id | ffd19119f10c4ab
| volumes_attached | id='cedd17da-
+------
2. Reboot compute-0 sudo reboot –f
3. wait for compute to come up.
+----+-
| id | hostname | personality | administrative | operational | availability |
+----+-
| 1 | controller-0 | controller | unlocked | enabled | available |
| 2 | controller-1 | controller | unlocked | enabled | available |
| 3 | compute-1 | worker | unlocked | enabled | available |
| 4 | compute-2 | worker | unlocked | enabled | available |
| 5 | compute-0 | worker | unlocked | disabled | offline |
| 6 | storage-0 | storage | unlocked | enabled | available |
| 7 | storage-1 | storage | unlocked | enabled | available |
+----+-
+----+-
| id | hostname | personality | administrative | operational | availability |
+----+-
| 1 | controller-0 | controller | unlocked | enabled | available |
| 2 | controller-1 | controller | unlocked | enabled | available |
| 3 | compute-1 | worker | unlocked | enabled | available |
| 4 | compute-2 | worker | unlocked | enabled | available |
| 5 | compute-0 | worker | unlocked | enabled | available |
| 6 | storage-0 | storage | unlocked | enabled | available |
| 7 | storage-1 | storage | unlocked | enabled | available |
+----+-
4.Verify VM evacuation VM running on compute-1
openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://
openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://
+------
| Field | Value |
+------
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-
| OS-EXT-
| OS-EXT-
| OS-EXT-
| OS-EXT-
| OS-EXT-
| OS-EXT-STS:vm_state | active |
| OS-SRV-
| OS-SRV-
| accessIPv4 | |
| accessIPv6 | |
| addresses | tenant1-
| config_drive | |
| created | 2019-07-
| flavor | flavor-
| hostId | a223412c30650d0
| id | ab3048c4-
| image | |
| key_name | keypair-tenant1 |
| name | tenant1-vm-1 |
| progress | 0 |
| project_id | 196a1151724d4d5
| properties | |
| security_groups | name='default' |
| | name='default' |
| status | ACTIVE |
| updated | 2019-07-
| user_id | ffd19119f10c4ab
| volumes_attached | id='cedd17da-
+------
[sysadmin@
5. Livemigrate VM after evacute
[sysadmin@
nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://
ndpoint-type internalURL --os-region-name RegionOne live-migration ab3048c4-
[sysadmin@
echo $?
0
[sysadmin@
openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://
os-identity-
openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://
openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://
os-identity-
+------
| Field | Value |
+------
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-
| OS-EXT-
| OS-EXT-
| OS-EXT-
| OS-EXT-
| OS-EXT-
| OS-EXT-STS:vm_state | active |
| OS-SRV-
| OS-SRV-
| accessIPv4 | |
| accessIPv6 | |
| addresses | tenant1-
| config_drive | |
| created | 2019-07-
| flavor | flavor-
| hostId | a223412c30650d0
| id | ab3048c4-
| image | |
| key_name | keypair-tenant1 |
| name | tenant1-vm-1 |
| progress | 0 |
| project_id | 196a1151724d4d5
| properties | |
| security_groups | name='default' |
| | name='default' |
| status | MIGRATING |
| updated | 2019-07-
| user_id | ffd19119f10c4ab
| volumes_attached | id='cedd17da-
openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://
openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://
os-identity-
+------
| Field | Value |
+------
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-
| OS-EXT-
| OS-EXT-
| OS-EXT-
| OS-EXT-
| OS-EXT-
| OS-EXT-STS:vm_state | active |
| OS-SRV-
| OS-SRV-
| accessIPv4 | |
| accessIPv6 | |
| addresses | tenant1-
| config_drive | |
| created | 2019-07-
| flavor | flavor-
| hostId | a223412c30650d0
| id | ab3048c4-
| image | |
| key_name | keypair-tenant1 |
| name | tenant1-vm-1 |
| progress | 0 |
| project_id | 196a1151724d4d5
| properties | |
| security_groups | name='default' |
| | name='default' |
| status | ACTIVE |
| updated | 2019-07-
| user_id | ffd19119f10c4ab
| volumes_attached | id='cedd17da-
+------
6.Migration list shows error to migrate from compute-1
nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://
nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://
ndpoint-type internalURL --os-region-name RegionOne migration-list
+----+-
| Id | UUID | Source Node | Dest Node | Source Compute | Dest Compute | Dest Host | Status | Instance UUID | Old Flavor | New Flavor | Created At | Updated At | Type |
+----+-
| 31 | 3fc2d997-
| 28 | 5e82192b-
+----+-
System Configuration
-------
Regular system
Expected Behavior
------------------
Able to migrate after evacuate.
Actual Behavior
----------------
Failure on migration after evacuate.
Reproducibility
---------------
100% reproducible
System Configuration
-------
Regular system
Load
----
2019-07-11 01:30:00 +0000
Last Pass
---------
Not sure in this lab.
Timestamp/Logs
--------------
2019-07-
Test Activity
-------------
Regression test
summary: |
- After VM was evacuated failing to livemigate + VM fail to live migrate after evacuation |
tags: | added: stx.2.0 |
Changed in starlingx: | |
importance: | Undecided → High |
status: | New → Triaged |
Changed in starlingx: | |
assignee: | zhipeng liu (zhipengs) → Shuquan Huang (shuquan) |
Changed in starlingx: | |
assignee: | Shuquan Huang (shuquan) → hutianhao27 (hutianhao) |
Changed in starlingx: | |
assignee: | hutianhao27 (hutianhao) → YU CHENGDE (chant) |
Assigning to Yong Hu as this appears to be a nova issue. Given that the issue is 100% reproducible, my recommendation is that this should be gating stx.2.0