Issues with adding/removing nodes via CLI vs UI

[Bogdan Dobrelya]

{"build_id": "2013-12-25_01-01-41", "ostf_sha": "a27665008e961b773861b95969acf61d25f0f977", "build_number": "202", "nailgun_sha": "d4aecc4f1b60d66021bac195800952fb94cbc627", "fuelmain_sha": "15f575db2d35e3374a217e29a38091152801cfe5", "astute_sha": "9000b12aa0f3639bd4a18efeb82ef554fdbde9c0", "release": "4.0", "fuellib_sha": "cc0301c5f0399e7567230d5cf261f0796005d91c"}

Steps to reproduce:
- From UI, create 2 environments env1 and env2,
- From UI add 3 nodes to env1 and 3 nodes to env2,
- provision some nodes in envs using CLI:
fuel --env 1 node --provision --node-id x,y,z
fuel --env 2 node --provision --node-id u,v,t
- make nodes in env2 to have an ERROR state, e.g. interrupt deployment of Openstack
- remove all nodes from its envs using CLI:
fuel node remove --node-id x,y,z,u,v,t

issue 1
- Check the nodes in UI: the UI-state is PENDING DELETION and never changes (see screenshot)
- Check the nodes in CLI, the CLI-states would remain as they were before removal. e.g.:

[root@fuel ~]# fuel node list
id | status | name | cluster | mac | roles | pending_roles | online
---|-------------|-------------------|---------|-------------------|-------|---------------|-------
1 | provisioned | Untitled (EB:4E) | None | 00:13:72:fa:eb:4c | [] | [] | True
4 | error | mc0n5-msk (56:C8) | None | 00:25:90:94:56:c8 | [] | [] | True
5 | provisioned | Untitled (2E:44) | None | f4:ce:46:c3:2e:44 | [] | [] | True
6 | provisioned | Untitled (7A:D4) | None | f4:ce:46:c3:7a:d4 | [] | [] | True
2 | error | mc0n3-msk (57:7A) | None | 00:25:90:94:57:7a | [] | [] | True
3 | error | mc0n4-msk (5A:38) | None | 00:25:90:94:5a:38 | [] | [] | True

- But, if you re-add the nodes back and delete them again from UI, the states in UI would be correct ('OS installed' and 'error')

issue 2
- From UI & CLI, you are able to add CLI-state "provisioned" nodes to any env, despite its 'pending for deletion' UI-state (see screenshot)
[root@fuel ~]# fuel --env-id=1 node set --node-id=1,5,6 --role=controller
[root@fuel ~]# fuel node list
id | status | name | cluster | mac | roles | pending_roles | online
---|-------------|-------------------|---------|-------------------|-------|-----------------|-------
6 | provisioned | Untitled (7A:D4) | 1 | f4:ce:46:c3:7a:d4 | [] | [u'controller'] | True
5 | provisioned | Untitled (2E:44) | 1 | f4:ce:46:c3:2e:44 | [] | [u'controller'] | True
1 | provisioned | Untitled (EB:4E) | 1 | 00:13:72:fa:eb:4c | [] | [u'controller'] | True
2 | error | mc0n3-msk (57:7A) | None | 00:25:90:94:57:7a | [] | [] | True
3 | error | mc0n4-msk (5A:38) | None | 00:25:90:94:5a:38 | [] | [] | True
4 | error | mc0n5-msk (56:C8) | None | 00:25:90:94:56:c8 | [] | [] | True

- From UI, you *cannnot* add CLI-state "error" nodes to any env.
- But you still can do it from CLI:
[root@fuel ~]# fuel --env-id=2 node set --node-id=2,3,4 --role=controller
[root@fuel ~]# fuel node list
id | status | name | cluster | mac | roles | pending_roles | online
---|-------------|-------------------|---------|-------------------|-------|-----------------|-------
3 | error | mc0n4-msk (5A:38) | 2 | 00:25:90:94:5a:38 | [] | [u'controller'] | True
5 | provisioned | Untitled (2E:44) | 1 | f4:ce:46:c3:2e:44 | [] | [u'controller'] | True
6 | provisioned | Untitled (7A:D4) | 1 | f4:ce:46:c3:7a:d4 | [] | [u'controller'] | True
2 | error | mc0n3-msk (57:7A) | 2 | 00:25:90:94:57:7a | [] | [u'controller'] | True
4 | error | mc0n5-msk (56:C8) | 2 | 00:25:90:94:56:c8 | [] | [u'controller'] | True
1 | provisioned | Untitled (EB:4E) | 1 | 00:13:72:fa:eb:4c | [] | [u'controller'] | True

Fix:
- UI-state should be logically equal to CLI-state by its behavior and by possible actions to do with it...
- UI-state should not hang for ever in 'pending deletion'.
- CLI-state should be updated appropriately on every node's removal / addition.

[Evgeniy L (rustyrobot)]

>> - Check the nodes in UI: the UI-state is PENDING DELETION and never changes (see screenshot)

It should be in PENDING DELETION state unless you run deployment.

>> - Check the nodes in CLI, the CLI-states would remain as they were before removal. e.g.:

Here need to show pending_deletion flag in CLI

So, in my opinion the main problem of this ticket is that we delete deployed nodes from cluster via CLI directly without erasing and rebooting nodes.

https://github.com/stackforge/fuel-web/blob/master/fuelclient/fuel#L504-L512

It should be resolved on nailgun side (api refactoring is required i.e. changes in ui and cli required too).

- create new handler for nodes deletion, this handler should accept nodes ids
- if nodes weren't provisioned then just delete them from cluster
- if nodes were provisioned then set up pending_deletion flag and nodes with such flags will be deleted after deployment

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/74102

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/72337
Committed: https://git.openstack.org/cgit/stackforge/fuel-web/commit/?id=530146a76fcc30bfb09a2205812d771ef863cbd3
Submitter: Jenkins
Branch: master

commit 530146a76fcc30bfb09a2205812d771ef863cbd3
Author: Alexandr Notchenko <email address hidden>
Date: Mon Feb 10 12:09:19 2014 +0400

    Created Node (Un)Assignment Handler

     * Created handler, validator,
     added url to handler mapping.
     * Created test for assignment
     and unassignment.
     * Moved role validation from
     fuelclient to assignment validator
     * new handlers used in fuelclient
     * refactored fueclient for new
     assignment and unassignment API

     Closes-bug: #1265036
     Implements: blueprint create-separate-handler-for-node-assignment

    Change-Id: Ie1dce1401e0f43cc336662bf29a4e5e72bd80db1

[Andrey Sledzinskiy (asledzinskiy)]

Bug is reproduced on fuel_5_0_iso #21

Steps:
1. Create env - fuel env create --name Test --rel 1
2. Add 3 nodes
- fuel node set --node 1 --role controller --env 1
- fuel node set --node 2 --role compute --env 1
- fuel node set --node 3 --role cinder --env 1
3. Run provisioning on 3 nodes - fuel --env 1 node --provision --node 1,2,3
4. Delete 3 nodes - fuel node remove --node 1,2,3
5. Check nodes status - fuel nodes

Expected - nodes statuses are 'pending deletion'
Actual - nodes statuses are 'provisioned'

[Andrey Sledzinskiy (asledzinskiy)]

Issue was discussed. Pending deletion status should occur on UI only.
Issue is verified on fuel_5_0_iso #21

[Vladimir Kuklin (vkuklin)]

should be backported to stable/4.1

[Dmitry Pyzhov (dpyzhov)]

Already in stable/4.1 branch

[Meg McRoberts (dreidellhasa)]

Marked as Fixed Issue in 5.0 Release Notes

[Egor Kotko (ykotko)]

{"build_id": "2014-06-05_15-25-50", "mirantis": "yes", "build_number": "344", "nailgun_sha": "a828d6b7610f872980d5a2113774f1cda6f6810b", "ostf_sha": "2b7b39e4b6ea89751b65171f24a8e80b5cac56aa", "fuelmain_sha": "9964da7dec34d3100419c1c77c8f5235d8e30f14", "astute_sha": "55df06b2e84fa5d71a1cc0e78dbccab5db29d968", "release": "4.1B", "fuellib_sha": "3511461a2b529619a787a6306441d9039699e71d"}
Checked on ISO#344