os-brick's iscsi initiator unexpectedly reverts node.startup from "automatic" to "manual".

Bug #1670237 reported by Rikimaru Honjo
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
os-brick
Fix Released
Medium
Rikimaru Honjo

Bug Description

os-brick's iscsi initiator unexpectedly reverts node.startup from "automatic" to "manual" in some specific cases if we use multipath.
As a result, automatic nodes and manual nodes are mixed on that host.

[Premises]
* Use multipath.
* os-brick's iscsi initiator changes node.startup from "manual" to "automatic" when it makes iscsi connections.
* If "iscsiadm -m discovery -t sendtargets -p ..." command is executed, node.startup is reverted from "automatic" to "manual".
* In current os-brick's implementation, it won't login iscsi session and change node.startup if sessions are already existed.
  (That logic is implemented in os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal() )

[Reproduce steps - 1]
1. User attaches a volume to a instance with iscsi.
2. iscsi sessions are made.
   And, node.startup is changed from "manual" to "automatic".
3. User attaches other volume to a instance with iscsi.
4. In attaching volume process, os-brick will execute "iscsiadm -m discovery -t sendtargets ..." if multipath is used.
5. As a result, node.startup of 1st attaching is reverted from "automatic" to "manual". And automatic nodes and manual nodes are mixed on that host.

[Reproduce steps - 2]

1. User attaches a volume to a instance with iscsi.
2. As a result, iscsi sessions are made.
   And, node.startup is changed from "manual" to "automatic".
3. The host of the instance is rebooted by with some reason.
4. After operating system is launched, iscsi sessions are automatically recovered.
   Because node.startup is "automatic".
5. The instance has stopped because the host was rebooted.
   User calls "Start Server" API.
6. Nova try to re-make iscsi sessions.
7. In starting server process, os-brick will execute "iscsiadm -m discovery -t sendtargets ..." if multipath is used.
   As a result, node.startup is reverted from "automatic" to "manual".
8. But os-brick doesn't change node.startup to "automatic" after that because iscsi sessions are already existed.
9. node.startup is still "manual" when server start is completed.

[My opinion]
Changing node.startup from "manual" to "automatic" is unnecessary.
Current os-brick users[1] create/re-create iSCSI sessions when they need it.
If someone needs node.startup=automatic, he should set default value as "automatic" in iscsi.conf.

[1]e.g. nova,cinder...

description: updated
description: updated
summary: - os-brick's iscsi initiator unexpectedly reverts node.startup from "auto"
- to "manual".
+ os-brick's iscsi initiator unexpectedly reverts node.startup from
+ "automatic" to "manual".
description: updated
Revision history for this message
Rikimaru Honjo (honjo-rikimaru-c6) wrote :

I fixed the bug description.
Sorry, I missed to write the following condition.

* Use multipath.

description: updated
description: updated
Changed in os-brick:
assignee: nobody → Rikimaru Honjo (honjo-rikimaru-c6)
Revision history for this message
Walt Boring (walter-boring) wrote :

So, the problem here is that every time we logout of an iSCSI portal the node.startup is forced back to manual. We should only do that if there are no more iSCSI volumes on the host at all, not just from 1 particular portal.

Changed in os-brick:
status: New → Confirmed
importance: Undecided → Medium
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on os-brick (master)

Change abandoned by Rikimaru Honjo (<email address hidden>) on branch: master
Review: https://review.openstack.org/446401
Reason: I abandon this patch for retrying review process.

Changed in os-brick:
status: Confirmed → In Progress
Revision history for this message
Rikimaru Honjo (honjo-rikimaru-c6) wrote :

I improved bug description.

description: updated
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to os-brick (master)

Fix proposed to branch: master
Review: https://review.openstack.org/466146

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to os-brick (master)

Reviewed: https://review.openstack.org/466146
Committed: https://git.openstack.org/cgit/openstack/os-brick/commit/?id=3266fb51a53156e4fa103c222e0226b23f785dd3
Submitter: Zuul
Branch: master

commit 3266fb51a53156e4fa103c222e0226b23f785dd3
Author: Rikimaru Honjo <email address hidden>
Date: Tue Nov 28 15:16:41 2017 +0900

    Recover node.startup values after discovering

    os_brick updates node.startup values from default value to
    "automatic" when it creates iscsi connection.
    But existing target's node.startup values will be reverted from
    "automatic" to default value in creating iscsi connection process
    if multipath is used.
    When using multipath with a discovery type of backend, the
    "iscsiadm -m discovery -t sendtargets -p ..." command will recreate
    all target information of specified node.[1] node.startup value wil
    be reverted to default value of existing targets by recreating.
    As a result, "automatic" targets and default value targets will be
    mixed on the host.

    So this patch recovers node.startup values after discovering.

    [1]
    This behavior was explained in following page:
    https://github.com/open-iscsi/open-iscsi/issues/58

    Change-Id: I30b736ae3b916f77fc0778f5364c5f6ed6fecc60
    closes-bug: #1670237

Changed in os-brick:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/os-brick 2.2.0

This issue was fixed in the openstack/os-brick 2.2.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.