Can not take a node offline for maintenance

Bug #1260099 reported by aeva black
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ironic
Fix Released
High
Yuriy Zveryanskyy

Bug Description

Admins sometimes need to take a node out of service temporarily, eg. to fix some hardware problem. Right now, the only way to stop Ironic from attempting to manage a node is to delete it. There should be a means to mark it as temporarily unavailable, eg, "in maintenance mode."

Furthermore, this should be possible while an instance is provisioned on the node -- and it should not immediately or adversely affect the management or operation of that instance. Maintenance-mode should only prevent certain actions, eg:
- the association of a new instance
- deployment of an image associated with an instance

It should not prevent the admin from doing such things as:
- accessing serial console
- managing power state
- deleting an instance
- [future] updating firmware
- [future] booting into a recovery or utility ramdisk, eg. to triage the hardware fault

aeva black (tenbrae)
Changed in ironic:
status: New → Confirmed
importance: Undecided → High
Revision history for this message
Robert Collins (lifeless) wrote :

Ideally this would remove the node from the scheduler immediately to avoid retry-fail loops against it.

aeva black (tenbrae)
description: updated
Changed in ironic:
assignee: nobody → Yuriy Zveryanskyy (yzveryanskyy)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to ironic (master)

Fix proposed to branch: master
Review: https://review.openstack.org/62711

Changed in ironic:
status: Confirmed → In Progress
Changed in ironic:
assignee: Yuriy Zveryanskyy (yzveryanskyy) → Roman Prykhodchenko (romcheg)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to ironic (master)

Reviewed: https://review.openstack.org/62711
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=9bc5f92fb88169acdac48b367e9ea71930cfaf38
Submitter: Jenkins
Branch: master

commit 9bc5f92fb88169acdac48b367e9ea71930cfaf38
Author: Yuriy Zveryanskyy <email address hidden>
Date: Tue Dec 17 16:36:55 2013 +0200

    Add RPC method for node maintenance mode

    Method 'change_node_maintenance_mode' added to manager and rpcapi.
    This method triggered maintenance mode for a node.
    New column 'maintenance' added to nodes table.

    Partial-Bug: #1260099
    Change-Id: I945a1ce72c04e5ee2a9427a58dae72b0719c160f

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Reviewed: https://review.openstack.org/63936
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=3c10b567ce07e9633b53acf6e51d182ce40c14ab
Submitter: Jenkins
Branch: master

commit 3c10b567ce07e9633b53acf6e51d182ce40c14ab
Author: Yuriy Zveryanskyy <email address hidden>
Date: Tue Dec 24 14:49:28 2013 +0200

    Disallow new provision for nodes in maintenance

    New deploy operation will not allowed for nodes in
    'maintenance' mode.

    Partial-Bug: #1260099
    Change-Id: Idaf8c3d70bd78b2c95bbdcff734e235eb8063d91

Changed in ironic:
assignee: Roman Prykhodchenko (romcheg) → Yuriy Zveryanskyy (yzveryanskyy)
aeva black (tenbrae)
Changed in ironic:
milestone: none → icehouse-3
Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Reviewed: https://review.openstack.org/63937
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=487f6738bcb331115b49a88f21520b412cdcc49c
Submitter: Jenkins
Branch: master

commit 487f6738bcb331115b49a88f21520b412cdcc49c
Author: Yuriy Zveryanskyy <email address hidden>
Date: Tue Dec 24 15:41:54 2013 +0200

    Add parameter for filtering nodes by maintenance mode

    Boolean parameter 'maintenance' added to API for filtering
    nodes by maintenance mode, example:
    /nodes?maintenance=false
    strutils.bool_from_string() used for converting 'associated'
    and 'maintenance' parameters, accepted values now for True
    ('1', 't', 'true', 'on', 'y', 'yes'), for False
    ('0', 'f', 'false', 'off', 'n', 'no') case-insensitive.
    Refactored dbapi method get_node_list with filters used for
    _get_nodes_collection in nodes controller.

    Partial-Bug: #1260099
    Partial-Bug: #1271291
    Change-Id: I01a62eb641316e890d191cf569bb3cf00b7619f2

Revision history for this message
aeva black (tenbrae) wrote :

Also addressed by the following patches

Allow clients to mark a node as in maintenance
  https://review.openstack.org/#/c/76614/5

Prevent a node in maintenance from being deployed
  https://review.openstack.org/#/c/77129/

Changed in ironic:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in ironic:
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in ironic:
milestone: icehouse-3 → 2014.1
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.