[RFE] Manual cleaning

[Vladyslav Drok]

Manual cleaning (as opposed to automated cleaning) encompasses all long running, manual, destructive tasks an operator may want to take either between workloads, or before the first workload has been assigned to a node.

[Ruby Loo (rloo)]

Specification has been approved and is available at http://specs.openstack.org/openstack/ironic-specs/specs/approved/manual-cleaning.html.

Copying from old BP (https://blueprints.launchpad.net/ironic/+spec/manual-cleaning):

Gerrit topic: https://review.openstack.org/#q,topic:bp/manual-cleaning,n,z

Addressed by: https://review.openstack.org/231288
    Manual cleaning

Addressed by: https://review.openstack.org/247128
    Specifying target provision states in fsm

Addressed by: https://review.openstack.org/247201
    Support arguments to clean step methods

Addressed by: https://review.openstack.org/247213
    Add manual cleaning to state machine

Addressed by: https://review.openstack.org/247285
    Conductor handles manual cleaning

Addressed by: https://review.openstack.org/247695
    Agent vendor handles manual cleaning

Addressed by: https://review.openstack.org/247701
    Replace config 'clean_nodes' with 'automated_clean'

Addressed by: https://review.openstack.org/251602
    Validate arguments to clean_step() decorator

Addressed by: https://review.openstack.org/251995
    API to manually clean nodes

Addressed by: https://review.openstack.org/257662
    Prevent 'manage' from CLEANING

[OpenStack Infra (hudson-openstack)]

Change abandoned by Ruby Loo (<email address hidden>) on branch: master
Review: https://review.openstack.org/257662
Reason: Not needed since it should return 409 (locked node) to be consistent with other states that are *ING.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/264266

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/247285
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=16786b6abd4ac74699c29c3af92208f9660629e9
Submitter: Jenkins
Branch: master

commit 16786b6abd4ac74699c29c3af92208f9660629e9
Author: Ruby Loo <email address hidden>
Date: Thu Nov 19 03:36:54 2015 +0000

    Conductor handles manual cleaning

    The conductor (manager) supports manual cleaning via:
    - do_node_clean() entry point
    - modifying cleaning flow/methods to support both automated and manual
      cleaning
    - validates the user-specified cleaning steps against the cleaning
      steps supported by the driver

    The RPC API version is bumped to 1.32.

    Change-Id: I0d8ab185f1a5553e02dff87a6d8a16ab9af4050e
    Partial-Bug: #1526290

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/267918

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/247695
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=1f82783a9e103e367706ee3e5a1be7d5f9408a02
Submitter: Jenkins
Branch: master

commit 1f82783a9e103e367706ee3e5a1be7d5f9408a02
Author: Ruby Loo <email address hidden>
Date: Thu Nov 19 17:14:55 2015 +0000

    Agent vendor handles manual cleaning

    Agent base vendor now supports manual cleaning. In the case of a
    clean version mismatch (agent rebooted to a new version after manual
    cleaning started), we continue from the current clean step instead
    of (in automated cleaning) restarting from the first clean step.

    Change-Id: I89726f3b32f96befba9a2e1bee67b2a7cc7eeb57
    Partial-Bug: #1526290

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/251995
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=edc37cbe1db539372a5e96296d7fbbebe9c50b2f
Submitter: Jenkins
Branch: master

commit edc37cbe1db539372a5e96296d7fbbebe9c50b2f
Author: Ruby Loo <email address hidden>
Date: Tue Dec 1 17:22:48 2015 +0000

    API to manually clean nodes

    This adds an API to allow manual cleaning of nodes, via
    PUT /v1/nodes/<node_ident>/states/provision. The argument
    'target' is 'clean', and the argument 'clean_steps' (the
    list of clean steps to be performed on the node) must be
    specified.

    The API version is bumped to 1.15.

    Change-Id: I0e34407133684e34c4ab9446b3521a24f3038f92
    Partial-Bug: #1526290

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/278135

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/278135
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=0b3da478a24a85df392ad383acea15bb559b80dc
Submitter: Jenkins
Branch: master

commit 0b3da478a24a85df392ad383acea15bb559b80dc
Author: Ruby Loo <email address hidden>
Date: Tue Feb 9 22:36:10 2016 +0000

    Remove 'zapping' from code

    The term 'zapping' has been replaced by 'manual cleaning'. This
    removes 'zapping' from the code. Since one of the changes is in
    the help for a config option, ironic.conf.sample was updated
    (note that it was out of date).

    (References to 'zapping' in the documentation is addressed in a
    separate patch.)

    Change-Id: Icc0303cf26b464b84ba97636b41d9edfa16b8dbd
    Partial-Bug: #1526290

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/278566

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/278936

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/247701
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=9220cbd385d8bfd41c2465306a5dc1b56de570b6
Submitter: Jenkins
Branch: master

commit 9220cbd385d8bfd41c2465306a5dc1b56de570b6
Author: Ruby Loo <email address hidden>
Date: Thu Nov 19 17:38:32 2015 +0000

    Replace config 'clean_nodes' with 'automated_clean'

    Since we have automated cleaning and manual cleaning, the config
    'clean_nodes' is a bit confusing since it only refers to automated
    cleaning. It is replaced with 'automated_clean'.

    This deprecates the 'clean_nodes' config; the plan is to delete it
    in Newton cycle.

    Updates to the documentation to explicitly mention automated vs
    manual cleaning is done in a separate patch.

    This includes a reno-style release note about the upgrade impact.

    Change-Id: I5b4fbac1753cb4213eb151b663490f381b751597
    Partial-Bug: #1526290

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/267918
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=74f66614042f8b32cadf7d6f171907c6ed1cefcb
Submitter: Jenkins
Branch: master

commit 74f66614042f8b32cadf7d6f171907c6ed1cefcb
Author: Ruby Loo <email address hidden>
Date: Fri Jan 15 03:11:14 2016 +0000

    Cache agent clean steps on node

    In order to make getting clean steps a synchronous call, instead of
    one that blocks on communication with the agent, the agent clean steps
    are cached in the node's driver_internal_info. Any time cleaning is
    started, the steps will be fetched from the agent and cached.

    This is needed for the 'GET /nodes/<node_ident>/cleaning/steps'
    API, which is not yet implemented but see the spec:
    (http://specs.openstack.org/openstack/ironic-specs/specs/approved/manual-cleaning.html#get-nodes-node-ident-cleaning-steps)

    Change-Id: I26288802d06683fd99138bfea488233c88260a7f
    Partial-Bug: #1526290
    Co-Authored-By: Josh Gachnang <email address hidden>

[Jim Rollenhagen (jim-rollenhagen)]

Two more left here:

Clean steps API: https://review.openstack.org/#/c/278566/
Docs: https://review.openstack.org/#/c/264266/

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/264266
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=f2d9886f991c5c8c07383c1af82823b3b5dcf067
Submitter: Jenkins
Branch: master

commit f2d9886f991c5c8c07383c1af82823b3b5dcf067
Author: Ruby Loo <email address hidden>
Date: Wed Jan 6 16:45:13 2016 +0000

    Add manual cleaning to documentation

    This updates the documentation to include manual cleaning.

    Change-Id: I8f91214911e8916c329c20a140e1d0957b1cc137
    Partial-Bug: #1526290