Detaching multipath volume doesn't work properly when using different targets with same portal for each multipath device
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Cinder |
Invalid
|
Undecided
|
Unassigned | ||
Kilo |
Fix Released
|
Undecided
|
Patrick East | ||
OpenStack Compute (nova) |
Fix Released
|
High
|
Hiroyuki Eguchi | ||
Icehouse |
Fix Released
|
High
|
Hiroyuki Eguchi | ||
Juno |
Fix Released
|
High
|
Sean Dague | ||
os-brick |
Fix Released
|
High
|
Danny Wilson |
Bug Description
Overview:
On Icehouse(2014.1.2) with "iscsi_
multipath volume doesn't work properly. When we use different targets(IQNs)
associated with same portal for each different multipath device, all of
the targets will be deleted via disconnect_
This problem is not yet fixed in upstream. However, the attached patch
fixes this problem.
Steps to Reproduce:
We can easily reproduce this issue without any special storage
system in the following Steps:
1. configure "iscsi_
2. configure "volume_
in cinder.conf on cinder node.
2. create an instance.
3. create 3 volumes and attach them to the instance.
4. detach one of these volumes.
5. check "multipath -ll" and "iscsiadm --mode session".
Detail:
This problem was introduced with the following patch which modified
attaching and detaching volume operations for different targets
associated with different portals for the same multipath device.
commit 429ac4dedd617f8
Author: Xing Yang <email address hidden>
Date: Date: Mon Jan 6 17:27:28 2014 -0500
Fixed a problem in iSCSI multipath
We found out that:
> # Do a discovery to find all targets.
> # Targets for multiple paths for the same multipath device
> # may not be the same.
> out = self._run_
> 'discovery',
> '-t',
> 'sendtargets',
> '-p',
> iscsi_propertie
> check_exit_code=[0, 255])[0] \
> or ""
>
> ips_iqns = self._get_
...
> # If no other multipath device attached has the same iqn
> # as the current device
> if not in_use:
> # disconnect if no other multipath devices with same iqn
> self._disconnec
> return
> elif multipath_device not in devices:
> # delete the devices associated w/ the unused multipath
> self._delete_
When we use different targets(IQNs) associated with same portal for each different
multipath device, the ips_iqns has all targets in compute node from the result of
"iscsiadm -m discovery -t sendtargets -p <the same portal>".
Then, the _delete_mpath() deletes all of the targets in the ips_iqns
via /sys/block/
For example, we create an instance and attach 3 volumes to the instance:
# iscsiadm --mode session
tcp: [17] 192.168.0.55:3260,1 iqn.2010-
tcp: [18] 192.168.0.55:3260,1 iqn.2010-
tcp: [19] 192.168.0.55:3260,1 iqn.2010-
# multipath -ll
33000000300000001 dm-7 IET,VIRTUAL-DISK
size=4.0G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
`- 23:0:0:1 sdd 8:48 active ready running
33000000100000001 dm-5 IET,VIRTUAL-DISK
size=2.0G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
`- 21:0:0:1 sdb 8:16 active ready running
33000000200000001 dm-6 IET,VIRTUAL-DISK
size=3.0G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
`- 22:0:0:1 sdc 8:32 active ready running
Then we detach one of these volumes:
# nova volume-detach 95f959cd-
As a result of detaching the volume, the compute node remains 3 iSCSI sessions
and the instance fails to access the attached multipath devices:
# iscsiadm --mode session
tcp: [17] 192.168.0.55:3260,1 iqn.2010-
tcp: [18] 192.168.0.55:3260,1 iqn.2010-
tcp: [19] 192.168.0.55:3260,1 iqn.2010-
# multipath -ll
33000000300000001 dm-7 ,
size=4.0G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=0 status=enabled
`- #:#:#:# - #:# failed faulty running
33000000200000001 dm-6 ,
size=3.0G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=0 status=enabled
`- #:#:#:# - #:# failed faulty running
Changed in nova: | |
assignee: | nobody → Hiroyuki Eguchi (h-eguchi) |
tags: | added: volumes |
tags: | added: icehouse-backport-potential juno-backport-potential |
Changed in nova: | |
status: | Fix Committed → In Progress |
Changed in nova: | |
milestone: | none → kilo-1 |
status: | Fix Committed → Fix Released |
Changed in nova: | |
status: | Fix Released → In Progress |
Changed in nova: | |
importance: | Undecided → High |
tags: | removed: icehouse-backport-potential juno-backport-potential |
Changed in nova: | |
milestone: | kilo-1 → 2015.1.0 |
Changed in nova: | |
status: | Fix Committed → Fix Released |
Changed in os-brick: | |
assignee: | nobody → Danny Wilson (daniel-wilson) |
Changed in os-brick: | |
importance: | Undecided → High |
status: | New → Confirmed |
Changed in os-brick: | |
status: | Confirmed → In Progress |
Changed in os-brick: | |
milestone: | none → 0.3.0 |
status: | Fix Committed → Fix Released |
Changed in cinder: | |
status: | New → Invalid |
We need to push a patch on the gerrit. /wiki.openstack .org/wiki/ How_To_ Contribute ?
Could you check https:/