Issues with adding/removing nodes via CLI vs UI

Bug #1265036 reported by Bogdan Dobrelya on 2013-12-30
16
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Medium
Alexandr Notchenko

Bug Description

{"build_id": "2013-12-25_01-01-41", "ostf_sha": "a27665008e961b773861b95969acf61d25f0f977", "build_number": "202", "nailgun_sha": "d4aecc4f1b60d66021bac195800952fb94cbc627", "fuelmain_sha": "15f575db2d35e3374a217e29a38091152801cfe5", "astute_sha": "9000b12aa0f3639bd4a18efeb82ef554fdbde9c0", "release": "4.0", "fuellib_sha": "cc0301c5f0399e7567230d5cf261f0796005d91c"}

Steps to reproduce:
- From UI, create 2 environments env1 and env2,
- From UI add 3 nodes to env1 and 3 nodes to env2,
- provision some nodes in envs using CLI:
fuel --env 1 node --provision --node-id x,y,z
fuel --env 2 node --provision --node-id u,v,t
- make nodes in env2 to have an ERROR state, e.g. interrupt deployment of Openstack
- remove all nodes from its envs using CLI:
fuel node remove --node-id x,y,z,u,v,t

issue 1
- Check the nodes in UI: the UI-state is PENDING DELETION and never changes (see screenshot)
- Check the nodes in CLI, the CLI-states would remain as they were before removal. e.g.:

[root@fuel ~]# fuel node list
id | status | name | cluster | mac | roles | pending_roles | online
---|-------------|-------------------|---------|-------------------|-------|---------------|-------
1 | provisioned | Untitled (EB:4E) | None | 00:13:72:fa:eb:4c | [] | [] | True
4 | error | mc0n5-msk (56:C8) | None | 00:25:90:94:56:c8 | [] | [] | True
5 | provisioned | Untitled (2E:44) | None | f4:ce:46:c3:2e:44 | [] | [] | True
6 | provisioned | Untitled (7A:D4) | None | f4:ce:46:c3:7a:d4 | [] | [] | True
2 | error | mc0n3-msk (57:7A) | None | 00:25:90:94:57:7a | [] | [] | True
3 | error | mc0n4-msk (5A:38) | None | 00:25:90:94:5a:38 | [] | [] | True

- But, if you re-add the nodes back and delete them again from UI, the states in UI would be correct ('OS installed' and 'error')

issue 2
- From UI & CLI, you are able to add CLI-state "provisioned" nodes to any env, despite its 'pending for deletion' UI-state (see screenshot)
[root@fuel ~]# fuel --env-id=1 node set --node-id=1,5,6 --role=controller
[root@fuel ~]# fuel node list
id | status | name | cluster | mac | roles | pending_roles | online
---|-------------|-------------------|---------|-------------------|-------|-----------------|-------
6 | provisioned | Untitled (7A:D4) | 1 | f4:ce:46:c3:7a:d4 | [] | [u'controller'] | True
5 | provisioned | Untitled (2E:44) | 1 | f4:ce:46:c3:2e:44 | [] | [u'controller'] | True
1 | provisioned | Untitled (EB:4E) | 1 | 00:13:72:fa:eb:4c | [] | [u'controller'] | True
2 | error | mc0n3-msk (57:7A) | None | 00:25:90:94:57:7a | [] | [] | True
3 | error | mc0n4-msk (5A:38) | None | 00:25:90:94:5a:38 | [] | [] | True
4 | error | mc0n5-msk (56:C8) | None | 00:25:90:94:56:c8 | [] | [] | True

- From UI, you *cannnot* add CLI-state "error" nodes to any env.
- But you still can do it from CLI:
[root@fuel ~]# fuel --env-id=2 node set --node-id=2,3,4 --role=controller
[root@fuel ~]# fuel node list
id | status | name | cluster | mac | roles | pending_roles | online
---|-------------|-------------------|---------|-------------------|-------|-----------------|-------
3 | error | mc0n4-msk (5A:38) | 2 | 00:25:90:94:5a:38 | [] | [u'controller'] | True
5 | provisioned | Untitled (2E:44) | 1 | f4:ce:46:c3:2e:44 | [] | [u'controller'] | True
6 | provisioned | Untitled (7A:D4) | 1 | f4:ce:46:c3:7a:d4 | [] | [u'controller'] | True
2 | error | mc0n3-msk (57:7A) | 2 | 00:25:90:94:57:7a | [] | [u'controller'] | True
4 | error | mc0n5-msk (56:C8) | 2 | 00:25:90:94:56:c8 | [] | [u'controller'] | True
1 | provisioned | Untitled (EB:4E) | 1 | 00:13:72:fa:eb:4c | [] | [u'controller'] | True

Fix:
- UI-state should be logically equal to CLI-state by its behavior and by possible actions to do with it...
- UI-state should not hang for ever in 'pending deletion'.
- CLI-state should be updated appropriately on every node's removal / addition.

Bogdan Dobrelya (bogdando) wrote :
description: updated
description: updated
description: updated
summary: - Issues with removing nodes via CLI
+ Issues with adding/removing nodes via CLI vs UI
description: updated
Evgeniy L (rustyrobot) wrote :

>> - Check the nodes in UI: the UI-state is PENDING DELETION and never changes (see screenshot)

It should be in PENDING DELETION state unless you run deployment.

>> - Check the nodes in CLI, the CLI-states would remain as they were before removal. e.g.:

Here need to show pending_deletion flag in CLI

So, in my opinion the main problem of this ticket is that we delete deployed nodes from cluster via CLI directly without erasing and rebooting nodes.

https://github.com/stackforge/fuel-web/blob/master/fuelclient/fuel#L504-L512

It should be resolved on nailgun side (api refactoring is required i.e. changes in ui and cli required too).

- create new handler for nodes deletion, this handler should accept nodes ids
- if nodes weren't provisioned then just delete them from cluster
- if nodes were provisioned then set up pending_deletion flag and nodes with such flags will be deleted after deployment

Changed in fuel:
status: New → Triaged
importance: Undecided → High
assignee: nobody → Fuel Python Team (fuel-python)
tags: added: nailgun
Changed in fuel:
assignee: Fuel Python Team (fuel-python) → Alexandr Notchenko (anotchenko)
status: Triaged → In Progress
Changed in fuel:
importance: High → Medium

Reviewed: https://review.openstack.org/72337
Committed: https://git.openstack.org/cgit/stackforge/fuel-web/commit/?id=530146a76fcc30bfb09a2205812d771ef863cbd3
Submitter: Jenkins
Branch: master

commit 530146a76fcc30bfb09a2205812d771ef863cbd3
Author: Alexandr Notchenko <email address hidden>
Date: Mon Feb 10 12:09:19 2014 +0400

    Created Node (Un)Assignment Handler

     * Created handler, validator,
     added url to handler mapping.
     * Created test for assignment
     and unassignment.
     * Moved role validation from
     fuelclient to assignment validator
     * new handlers used in fuelclient
     * refactored fueclient for new
     assignment and unassignment API

     Closes-bug: #1265036
     Implements: blueprint create-separate-handler-for-node-assignment

    Change-Id: Ie1dce1401e0f43cc336662bf29a4e5e72bd80db1

Changed in fuel:
status: In Progress → Fix Committed
Changed in fuel:
status: Fix Committed → New

Bug is reproduced on fuel_5_0_iso #21

Steps:
1. Create env - fuel env create --name Test --rel 1
2. Add 3 nodes
- fuel node set --node 1 --role controller --env 1
- fuel node set --node 2 --role compute --env 1
- fuel node set --node 3 --role cinder --env 1
3. Run provisioning on 3 nodes - fuel --env 1 node --provision --node 1,2,3
4. Delete 3 nodes - fuel node remove --node 1,2,3
5. Check nodes status - fuel nodes

Expected - nodes statuses are 'pending deletion'
Actual - nodes statuses are 'provisioned'

Changed in fuel:
milestone: 4.1 → 4.1.1
status: New → Confirmed

Issue was discussed. Pending deletion status should occur on UI only.
Issue is verified on fuel_5_0_iso #21

Changed in fuel:
status: Confirmed → Fix Committed
status: Fix Committed → Fix Released
Vladimir Kuklin (vkuklin) wrote :

should be backported to stable/4.1

Changed in fuel:
status: Fix Released → In Progress
Dmitry Pyzhov (dpyzhov) wrote :

Already in stable/4.1 branch

Changed in fuel:
status: In Progress → Fix Committed
Mike Scherbakov (mihgen) on 2014-05-08
tags: added: release-notes
Meg McRoberts (dreidellhasa) wrote :

Marked as Fixed Issue in 5.0 Release Notes

Egor Kotko (ykotko) wrote :

{"build_id": "2014-06-05_15-25-50", "mirantis": "yes", "build_number": "344", "nailgun_sha": "a828d6b7610f872980d5a2113774f1cda6f6810b", "ostf_sha": "2b7b39e4b6ea89751b65171f24a8e80b5cac56aa", "fuelmain_sha": "9964da7dec34d3100419c1c77c8f5235d8e30f14", "astute_sha": "55df06b2e84fa5d71a1cc0e78dbccab5db29d968", "release": "4.1B", "fuellib_sha": "3511461a2b529619a787a6306441d9039699e71d"}
Checked on ISO#344

Changed in fuel:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers