When live-migrate failed with error, lun-id of connection_info column in Nova's block_deivce_mapping table couldn't be rollback.
and failed VM can have others volume.
my test environment is following :
Openstack Version : Havana ( 2013.2.3)
Compute Node OS : 3.5.0-23-generic #35~precise1-Ubuntu SMP Fri Jan 25 17:13:26 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
Compute Node multipath : multipath-tools 0.4.9-3ubuntu7.2
test step is :
1) create 2 Compute node (host#1 and host#2)
2) create 1 VM on host#1 (vm01)
3) create 1 cinder volume (vol01)
4) attach 1 volume to vm01 (/dev/vdb)
5) live-migrate vm01 from host#1 to host#2
6) live-migrate success
- please check the mapper by using multipath command in host#1 (# multipath -ll), then you can find mapper is not deleted.
and the status of devices is "failed faulty"
- please check the lun-id of vol01
7) Again, live-migrate vm01 from host#2 to host#1 (vm01 was migrated to host#2 at step 4)
8) live-migrate fail
- please check the mapper in host#1
- please check the lun-id of vol01, then you can find the lun hav "two" igroups
- please check the connection_info column in Nova's block_deivce_mapping table, then you can find lun-id couldn't be rollback
This Bug is critical security issue because the failed VM can have others volume.
and every backend storage of cinder-volume can have same problem because this is the bug of live-migration's rollback process.
I suggest bellow methods to solve issue :
1) when live-migrate is complete, nova should delete mapper devices at origin host
2) when live-migrate is failed, nova should rollback lun-id in connection_info column
3) when live-migrate is failed, cinder should delete the mapping between lun and host (Netapp : igroup, EMC : storage_group ...)
4) when volume-attach is requested , cinder volume driver of vendors should make lun-id randomly for reduce of probability of mis-mapping
Hi, guys
When live-migrate failed with error, lun-id of connection_info column in Nova's block_deivce_ mapping table couldn't be rollback.
and failed VM can have others volume.
my test environment is following :
Openstack Version : Havana ( 2013.2.3)
Compute Node OS : 3.5.0-23-generic #35~precise1-Ubuntu SMP Fri Jan 25 17:13:26 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
Compute Node multipath : multipath-tools 0.4.9-3ubuntu7.2
test step is :
1) create 2 Compute node (host#1 and host#2) mapping table, then you can find lun-id couldn't be rollback
2) create 1 VM on host#1 (vm01)
3) create 1 cinder volume (vol01)
4) attach 1 volume to vm01 (/dev/vdb)
5) live-migrate vm01 from host#1 to host#2
6) live-migrate success
- please check the mapper by using multipath command in host#1 (# multipath -ll), then you can find mapper is not deleted.
and the status of devices is "failed faulty"
- please check the lun-id of vol01
7) Again, live-migrate vm01 from host#2 to host#1 (vm01 was migrated to host#2 at step 4)
8) live-migrate fail
- please check the mapper in host#1
- please check the lun-id of vol01, then you can find the lun hav "two" igroups
- please check the connection_info column in Nova's block_deivce_
This Bug is critical security issue because the failed VM can have others volume.
and every backend storage of cinder-volume can have same problem because this is the bug of live-migration's rollback process.
I suggest bellow methods to solve issue :
1) when live-migrate is complete, nova should delete mapper devices at origin host
2) when live-migrate is failed, nova should rollback lun-id in connection_info column
3) when live-migrate is failed, cinder should delete the mapping between lun and host (Netapp : igroup, EMC : storage_group ...)
4) when volume-attach is requested , cinder volume driver of vendors should make lun-id randomly for reduce of probability of mis-mapping
please check this bug.
Thank you.