Can not take a node offline for maintenance

Bug #1260099 reported by aeva black
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ironic
Fix Released
High
Yuriy Zveryanskyy

Bug Description

Admins sometimes need to take a node out of service temporarily, eg. to fix some hardware problem. Right now, the only way to stop Ironic from attempting to manage a node is to delete it. There should be a means to mark it as temporarily unavailable, eg, "in maintenance mode."

Furthermore, this should be possible while an instance is provisioned on the node -- and it should not immediately or adversely affect the management or operation of that instance. Maintenance-mode should only prevent certain actions, eg:
- the association of a new instance
- deployment of an image associated with an instance

It should not prevent the admin from doing such things as:
- accessing serial console
- managing power state
- deleting an instance
- [future] updating firmware
- [future] booting into a recovery or utility ramdisk, eg. to triage the hardware fault

aeva black (tenbrae)
Changed in ironic:
status: New → Confirmed
importance: Undecided → High
Revision history for this message
Robert Collins (lifeless) wrote :

Ideally this would remove the node from the scheduler immediately to avoid retry-fail loops against it.

aeva black (tenbrae)
description: updated
Changed in ironic:
assignee: nobody → Yuriy Zveryanskyy (yzveryanskyy)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to ironic (master)

Fix proposed to branch: master
Review: https://review.openstack.org/62711

Changed in ironic:
status: Confirmed → In Progress
Changed in ironic:
assignee: Yuriy Zveryanskyy (yzveryanskyy) → Roman Prykhodchenko (romcheg)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to ironic (master)

Reviewed: https://review.openstack.org/62711
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=9bc5f92fb88169acdac48b367e9ea71930cfaf38
Submitter: Jenkins
Branch: master

commit 9bc5f92fb88169acdac48b367e9ea71930cfaf38
Author: Yuriy Zveryanskyy <email address hidden>
Date: Tue Dec 17 16:36:55 2013 +0200

    Add RPC method for node maintenance mode

    Method 'change_node_maintenance_mode' added to manager and rpcapi.
    This method triggered maintenance mode for a node.
    New column 'maintenance' added to nodes table.

    Partial-Bug: #1260099
    Change-Id: I945a1ce72c04e5ee2a9427a58dae72b0719c160f

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Reviewed: https://review.openstack.org/63936
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=3c10b567ce07e9633b53acf6e51d182ce40c14ab
Submitter: Jenkins
Branch: master

commit 3c10b567ce07e9633b53acf6e51d182ce40c14ab
Author: Yuriy Zveryanskyy <email address hidden>
Date: Tue Dec 24 14:49:28 2013 +0200

    Disallow new provision for nodes in maintenance

    New deploy operation will not allowed for nodes in
    'maintenance' mode.

    Partial-Bug: #1260099
    Change-Id: Idaf8c3d70bd78b2c95bbdcff734e235eb8063d91

Changed in ironic:
assignee: Roman Prykhodchenko (romcheg) → Yuriy Zveryanskyy (yzveryanskyy)
aeva black (tenbrae)
Changed in ironic:
milestone: none → icehouse-3
Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Reviewed: https://review.openstack.org/63937
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=487f6738bcb331115b49a88f21520b412cdcc49c
Submitter: Jenkins
Branch: master

commit 487f6738bcb331115b49a88f21520b412cdcc49c
Author: Yuriy Zveryanskyy <email address hidden>
Date: Tue Dec 24 15:41:54 2013 +0200

    Add parameter for filtering nodes by maintenance mode

    Boolean parameter 'maintenance' added to API for filtering
    nodes by maintenance mode, example:
    /nodes?maintenance=false
    strutils.bool_from_string() used for converting 'associated'
    and 'maintenance' parameters, accepted values now for True
    ('1', 't', 'true', 'on', 'y', 'yes'), for False
    ('0', 'f', 'false', 'off', 'n', 'no') case-insensitive.
    Refactored dbapi method get_node_list with filters used for
    _get_nodes_collection in nodes controller.

    Partial-Bug: #1260099
    Partial-Bug: #1271291
    Change-Id: I01a62eb641316e890d191cf569bb3cf00b7619f2

Revision history for this message
aeva black (tenbrae) wrote :

Also addressed by the following patches

Allow clients to mark a node as in maintenance
  https://review.openstack.org/#/c/76614/5

Prevent a node in maintenance from being deployed
  https://review.openstack.org/#/c/77129/

Changed in ironic:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in ironic:
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in ironic:
milestone: icehouse-3 → 2014.1
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers