Deleted nodes after provision task not became bootstrap

[Anastasia Palkina]

1. Create new environment by default
2. Add "Controller" node
3. Start provision task for this node via CLI
4. Wait until provisioning is finished
5. Check this node and click "Delete" button in UI. Also click this button in confirmation dialogue

Expected result:
Node deleted from environment.
Node rebooted and became bootstrap

Actual result:
Node deleted from environment.
Node not rebooted and not became bootstrap.

Logs are here: https://drive.google.com/a/mirantis.com/file/d/0B6SjzarTGFxaaEczblA0akI0bFk/view?usp=sharing

I have 2 provisioned and not rebooted after deletion nodes: node-1,2

VERSION:
  feature_groups:
    - mirantis
  production: "docker"
  release: "8.0"
  openstack_version: "2015.1.0-8.0"
  api: "1.0"
  build_number: "243"
  build_id: "243"
  fuel-nailgun_sha: "1b6ae6c7fd4766f4a57d90afc1cd0f2315255a42"
  python-fuelclient_sha: "91474bd8c526f4f536ab13368feb4a5c1b84d185"
  fuel-agent_sha: "660c6514caa8f5fcd482f1cc4008a6028243e009"
  fuel-nailgun-agent_sha: "a33a58d378c117c0f509b0e7badc6f0910364154"
  astute_sha: "b60624ee2c5f1d6d805619b6c27965a973508da1"
  fuel-library_sha: "22ed1ebd7804457a02b51b57c7019d853c528611"
  fuel-ostf_sha: "a98973482f839554d90cc1c071d625a01e018cfe"
  fuel-mirror_sha: "5a1efffd411611b87cbd45af59b9733b7e170cd4"
  fuelmenu_sha: "fcb15df4fd1a790b17dd78cf675c11c279040941"
  shotgun_sha: "25a0cc461a9fa4f7684f04cef0ff4ad9aa99a64d"
  network-checker_sha: "a3534f8885246afb15609c54f91d3b23d599a5b1"
  fuel-upgrade_sha: "1e894e26d4e1423a9b0d66abd6a79505f4175ff6"
  fuelmain_sha: "ff6eec402830ea8aa9898249b5eeb928576962ff"

[Dmitry Pyzhov (dpyzhov)]

This feature is experimental. Setting priority to Medium.

[slava valyavskiy (slava-val-al)]

May be related to https://bugs.launchpad.net/fuel/+bug/1528574

[Valentin Kaplov (vkaplov)]

Web UI doesn't send request DELETE api/tasks/<task_id> if node has status 'provisioned'.

[Valentin Kaplov (vkaplov)]

More details:
- if node is deployed, then UI sends requests DELETE api/tasks/<task_id> and Node moves to bootstrap state.
- if node is provisioned, then UI doesn't sends the DELETE api/tasks/<task_id> and Node leaves in provisioned state.

Expected behaviour:
- DELETE request should be sends if node was provisioned.

[Julia Aranovich (jkirnosova)]

When deleting a node from UI (node with any status), there are no any DELETE /api/tasks/<task_id> requests, only PUT /api/nodes/ with corresponding node data.

Valentin, could you please provide more details what sould be done for provisioned node to be bootstrapped?

[Valentin Kaplov (vkaplov)]

I beleave, that requeast on DELETE of provision task will make nodes in bootstraped state. But I did not checked this case. So if it would not help, please, contact me.

[Julia Aranovich (jkirnosova)]

So, when we click Delete button on UI for deployed node, there are no any DELETE requests for tasks. We just mark the node as "pending deletion". To remove it from environment completely, we need to run redeployment for the environment. In this case, all previous deployment tasks will be deleted, yes.

But in the case of provisioned and not deployed nodes, we do not need to redeploy environment. Provision task won't be deleted.

Does UI need to send both requests when deleting a provisioned node: PUT /api/nodes/ and DELETE /api/tasks/<provision_task_id> ?

[Julia Aranovich (jkirnosova)]

In result, we have an agreement, that UI should:

1) send DELETE /api/nodes/<node_id> request to delete discovered or provisioned nodes. This request will trigger appropriate tasks for the node (like bootstrapping, etc.)

2) send PUT /api/nodes request with data {id: <node_id>, pending_deletion: true} to mark deployed node for deletion (this needs further redeployment)

BUT it also should be Nailgun change here:

1) when UI sends DELETE request for discovered node, Nailgun should check the 'discovered' node status and NOT to bootstrap node again

[Julia Aranovich (jkirnosova)]

Sending of a DELETE request for a node runs node_deletion task and this blocks a batch node deletion on UI (DELETE for the second node will return 500 because of not completed previous node_deletion task).

So, we still need to think about the way to delete provisioned nodes. The deletion flow of deployed nodes (pending deletion node flag and redeployment) is not suitable for provisioned nodes too, because it can be the case, that we need to deploy some controller to delete another provisioned node from the environment (at least 1 controller is required to deploy environment changes).

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/274647
Committed: https://git.openstack.org/cgit/openstack/fuel-web/commit/?id=99946e37223bd5bbde1e6186de0457e635069c9f
Submitter: Jenkins
Branch: master

commit 99946e37223bd5bbde1e6186de0457e635069c9f
Author: Julia Aranovich <email address hidden>
Date: Mon Feb 1 12:18:43 2016 +0300

    Separate provisioning of nodes in Fuel UI

    Separate provisioning task can be started
    for all environment nodes in Fuel UI

    Implements: blueprint allow-choosing-nodes-for-provisioning-and-deployment

    Closes-Bug: #1522444

    Change-Id: I30b902da2f6134d0b229f23aa99c7e8c7a362873

[Maksym Strukov (unbelll)]

Verified as fixed in 9.0-mos-448