Activity log for bug #1419577

Date Who What changed Old value New value Message
2015-02-09 01:29:56 Hyun Ha bug added bug
2015-02-09 01:35:37 Hyun Ha description Hi, guys When live-migrate failed with error, lun-id of connection_info column in Nova's block_deivce_mapping table couldn't be rollback. and failed VM can have others volume. my test environment is following : Openstack Version : Havana ( 2013.2.3) Compute Node OS : 3.5.0-23-generic #35~precise1-Ubuntu SMP Fri Jan 25 17:13:26 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux Compute Node multipath : multipath-tools 0.4.9-3ubuntu7.2 test step is : 1) create 2 Compute node (host#1 and host#2) 2) create 1 VM on host#1 (vm01) 3) create 1 cinder volume (vol01) 4) attach 1 volume to vm01 (/dev/vdb) 5) live-migrate vm01 from host#1 to host#2 6) live-migrate success - please check the mapper by using multipath command in host#1 (# multipath -ll), then you can find mapper is not deleted. and the status of devices is "failed faulty" - please check the lun-id of vol01 7) Again, live-migrate vm01 from host#2 to host#1 (vm01 was migrated to host#2 at step 4) 8) live-migrate fail - please check the mapper in host#1 - please check the lun-id of vol01, then you can find the lun hav "two" igroups - please check the connection_info column in Nova's block_deivce_mapping table, then you can find lun-id couldn't be rollback This Bug is critical security issue because the failed VM can have others volume. and every backend storage of cinder-volume can have same problem because this is the bug of live-migration's rollback process. I suggest bellow methods to solve issue : 1) when live-migrate is complete, nova should delete mapper devices at origin host 2) when live-migrate is failed, nova should rollback lun-id in connection_info column 3) when live-migrate is failed, cinder should delete the mapping between lun and host (Netapp : igroup, EMC : storage_group ...) 4) when volume-attach is requested , cinder volume driver of vendors should make lun-id randomly for reduce of probability of mis-mapping please check this bug. Thank you. Hi, guys When live-migrate failed with error, lun-id of connection_info column in Nova's block_deivce_mapping table couldn't be rollback. and failed VM can have others volume. my test environment is following : Openstack Version : Havana ( 2013.2.3) Compute Node OS : 3.5.0-23-generic #35~precise1-Ubuntu SMP Fri Jan 25 17:13:26 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux Compute Node multipath : multipath-tools 0.4.9-3ubuntu7.2 test step is : 1) create 2 Compute node (host#1 and host#2) 2) create 1 VM on host#1 (vm01) 3) create 1 cinder volume (vol01) 4) attach 1 volume to vm01 (/dev/vdb) 5) live-migrate vm01 from host#1 to host#2 6) live-migrate success      - please check the mapper by using multipath command in host#1 (# multipath -ll), then you can find mapper is not deleted.        and the status of devices is "failed faulty"      - please check the lun-id of vol01 7) Again, live-migrate vm01 from host#2 to host#1 (vm01 was migrated to host#2 at step 4) 8) live-migrate fail      - please check the mapper in host#1      - please check the lun-id of vol01, then you can find the lun hav "two" igroups      - please check the connection_info column in Nova's block_deivce_mapping table, then you can find lun-id couldn't be rollback This Bug is critical security issue because the failed VM can have others volume. and every backend storage of cinder-volume can have same problem because this is the bug of live-migration's rollback process. I suggest below methods to solve issue : 1) when live-migrate is complete, nova should delete mapper devices at origin host 2) when live-migrate is failed, nova should rollback lun-id in connection_info column 3) when live-migrate is failed, cinder should delete the mapping between lun and host (Netapp : igroup, EMC : storage_group ...) 4) when volume-attach is requested , cinder volume driver of vendors should make lun-id randomly for reduce of probability of mis-mapping please check this bug. Thank you.
2015-02-09 01:37:21 Hyun Ha affects cinder nova-project
2015-02-09 02:17:28 Hyun Ha information type Private Security Public Security
2015-02-09 03:22:12 Robert Esker bug added subscriber Robert Esker
2015-02-09 14:10:35 Tristan Cacqueray bug task added ossa
2015-02-09 14:10:42 Tristan Cacqueray ossa: status New Incomplete
2015-02-26 15:29:29 Robert Esker bug added subscriber NetApp
2015-03-02 15:26:56 Thierry Carrez nova-project: status New Confirmed
2015-03-02 15:31:28 Thierry Carrez affects nova-project nova
2015-03-02 15:32:12 Thierry Carrez nova: importance Undecided High
2015-03-09 14:08:40 Jeremy Stanley information type Public Security Public
2015-03-09 14:08:52 Jeremy Stanley tags security
2015-03-09 14:13:08 Jeremy Stanley ossa: status Incomplete Won't Fix
2015-04-03 22:37:53 Jeremy Stanley cve linked 2015-2687
2015-05-26 20:02:40 Matt Riedemann tags security live-migration security volumes
2015-05-29 07:53:39 Pawel Koniszewski tags live-migration security volumes live-migrate security volumes
2015-11-06 16:50:18 Paul Murray tags live-migrate security volumes live-migration security volumes
2015-11-17 13:53:18 lvmxh nova: assignee lvmxh (shaohef)
2015-12-03 09:07:44 lvmxh nova: assignee lvmxh (shaohef)
2015-12-11 08:15:40 Pawel Koniszewski bug added subscriber Pawel Koniszewski
2016-01-27 12:18:07 lvmxh information type Public Public Security
2016-01-27 13:44:38 Jeremy Stanley information type Public Security Public
2016-07-07 12:12:46 Lee Yarwood bug watch added https://bugzilla.redhat.com/show_bug.cgi?id=1353147
2016-07-07 12:15:07 OpenStack Infra nova: status Confirmed In Progress
2016-07-07 12:15:07 OpenStack Infra nova: assignee Lee Yarwood (lyarwood)
2016-10-26 08:24:46 OpenStack Infra nova: assignee Lee Yarwood (lyarwood) Dan Smith (danms)
2016-10-26 14:43:01 OpenStack Infra nova: assignee Dan Smith (danms) Lee Yarwood (lyarwood)
2016-12-09 16:12:07 Matt Riedemann nova: status In Progress Confirmed
2016-12-09 16:12:11 Matt Riedemann nova: assignee Lee Yarwood (lyarwood)
2016-12-09 16:14:49 Sean Dague nova: importance High Medium
2017-01-17 13:20:19 OpenStack Infra nova: status Confirmed In Progress
2017-01-17 13:20:19 OpenStack Infra nova: assignee Lee Yarwood (lyarwood)
2017-06-27 15:55:06 Sean Dague tags live-migration security volumes live-migration openstack-version.havana security volumes