Error if try to delete 2 offline nodes in 9.2

Bug #1657350 reported by Sergey Galkin
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Committed
Medium
Alexander Kislitsky
Mitaka
Fix Released
Medium
Denis Meltsaykin
Newton
Fix Committed
Medium
Georgy Kibardin

Bug Description

Chrome - Version 55.0.2883.87 (64-bit) Linux
Steps to reproduce:
1. Install 9.0
2. Upgrade to 9.2 from http://mirror.fuel-infra.org/mos-repos/centos/mos9.0-centos7/snapshots/proposed-2017-01-13-184421/x86_64
3. Start deploying cluster (~400 nodes in my case)
4. During deployment 2 nodes gone offline

Try to remove first node (successes), after this try to remove second node and get error
After disappear first node I can remove second node without any problem

Revision history for this message
Sergey Galkin (sgalkin) wrote :
Changed in fuel:
status: New → Confirmed
importance: Undecided → Critical
importance: Critical → Medium
assignee: nobody → Fuel Sustaining (fuel-sustaining-team)
milestone: none → 11.0
tags: added: area-python
Revision history for this message
Julia Aranovich (jkirnosova) wrote :

First, the following issue should be fixed:
api call to delete the second node returns error, but this error isn't shown in the error dialog. It happens because of an invalid format of api response. Now it's a simple string (error message), but should be an object of the following format:
{message: 'Error message ...', errors: []}

Revision history for this message
Julia Aranovich (jkirnosova) wrote :

Then we should consider UX of a few nodes deletion. We need to get user information that it is possible to remove 1 node only at the same time:

1) it can be described in the error message
2) or Remove button can be hidden on UI if some node already removing (cluster has running 'node_deletion' task).
3) Does 'node_deletion' task support deletion of a few nodes at the same time? If yes, we can think about batch node removal from UI.

tags: added: area-ui
Revision history for this message
Julia Aranovich (jkirnosova) wrote :

As far as we have a handler for bulk node removal (https://github.com/openstack/fuel-web/blob/master/nailgun/nailgun/api/v1/handlers/node.py#L163), it was decided to move Remove button from a node panel to node management toolbar to support the bulk action. User will be able to remove a group of offline nodes at once.

tags: added: feature
Changed in fuel:
assignee: Fuel Sustaining (fuel-sustaining-team) → Julia Aranovich (jkirnosova)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-ui (master)

Fix proposed to branch: master
Review: https://review.openstack.org/425099

Changed in fuel:
status: Confirmed → In Progress
tags: removed: area-python
Revision history for this message
Julia Aranovich (jkirnosova) wrote :

Bulk offline node removal added here: https://review.openstack.org/#/c/425099/
The other part of the issue fix is to update a format of error message (which tells that node_deletion task is already running), it should be:
{message: 'Error message ...', errors: []} (it's a single string now)

tags: added: area-python
removed: feature
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-web (master)

Fix proposed to branch: master
Review: https://review.openstack.org/425684

Changed in fuel:
assignee: Julia Aranovich (jkirnosova) → Alexander Kislitsky (akislitsky)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-ui (stable/newton)

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/425696

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-ui (master)

Reviewed: https://review.openstack.org/425099
Committed: https://git.openstack.org/cgit/openstack/fuel-ui/commit/?id=be80843f05c64a1258d334b006d3ed638fa7218b
Submitter: Jenkins
Branch: master

commit be80843f05c64a1258d334b006d3ed638fa7218b
Author: Julia Aranovich <email address hidden>
Date: Wed Jan 25 13:04:22 2017 +0300

    Add bulk offline node removal

    Now user is able to select a group of offline nodes
    and remove them from Fuel by clicking
    'Remove Offline Nodes' button in the toolbar and confirming
    the removal.

    Partial-Bug: #1657350

    Change-Id: I0b3b50a29e8abed7c4cfac0c46a0f053cb82bb09

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-ui (stable/mitaka)

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/425743

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-ui (stable/newton)

Reviewed: https://review.openstack.org/425696
Committed: https://git.openstack.org/cgit/openstack/fuel-ui/commit/?id=17495ac4676b03e5b44a843836bcbe009232fd45
Submitter: Jenkins
Branch: stable/newton

commit 17495ac4676b03e5b44a843836bcbe009232fd45
Author: Julia Aranovich <email address hidden>
Date: Wed Jan 25 13:04:22 2017 +0300

    Add bulk offline node removal

    Now user is able to select a group of offline nodes
    and remove them from Fuel by clicking
    'Remove Offline Nodes' button in the toolbar and confirming
    the removal.

    Partial-Bug: #1657350

    Change-Id: I0b3b50a29e8abed7c4cfac0c46a0f053cb82bb09

tags: added: in-stable-newton
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-ui (stable/mitaka)

Reviewed: https://review.openstack.org/425743
Committed: https://git.openstack.org/cgit/openstack/fuel-ui/commit/?id=de101edf0338fe70eb445d6f7a9508a142c5ae22
Submitter: Jenkins
Branch: stable/mitaka

commit de101edf0338fe70eb445d6f7a9508a142c5ae22
Author: Julia Aranovich <email address hidden>
Date: Wed Jan 25 13:04:22 2017 +0300

    Add bulk offline node removal

    Now user is able to select a group of offline nodes
    and remove them from Fuel by clicking
    'Remove Offline Nodes' button in the toolbar and confirming
    the removal.

    Partial-Bug: #1657350

    Change-Id: I0b3b50a29e8abed7c4cfac0c46a0f053cb82bb09

tags: added: in-stable-mitaka
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-web (stable/newton)

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/426175

Changed in fuel:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on fuel-web (stable/newton)

Change abandoned by Julia Aranovich (<email address hidden>) on branch: stable/newton
Review: https://review.openstack.org/426175

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-web (master)

Reviewed: https://review.openstack.org/425684
Committed: https://git.openstack.org/cgit/openstack/fuel-web/commit/?id=e46b57befc0032448e12581dc64458170280d465
Submitter: Jenkins
Branch: master

commit e46b57befc0032448e12581dc64458170280d465
Author: Alexander Kislitsky <email address hidden>
Date: Thu Jan 26 16:11:10 2017 +0300

    Message format on node(-s) deletion error changed

    In case of already running task we are returning JSON instead
    of plain text.

    Change-Id: Ic3998b8e534b2fa67813faf033335e849abe55af
    Closes-Bug: #1657350

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-web (stable/mitaka)

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/427593

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-web (stable/newton)

Reviewed: https://review.openstack.org/426175
Committed: https://git.openstack.org/cgit/openstack/fuel-web/commit/?id=836061341e636ef70d9ea0d7ad871bfdf1a79e9f
Submitter: Jenkins
Branch: stable/newton

commit 836061341e636ef70d9ea0d7ad871bfdf1a79e9f
Author: Alexander Kislitsky <email address hidden>
Date: Thu Jan 26 16:11:10 2017 +0300

    Message format on node(-s) deletion error changed

    In case of already running task we are returning JSON instead
    of plain text.

    Change-Id: Ic3998b8e534b2fa67813faf033335e849abe55af
    Closes-Bug: #1657350

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/fuel-web 11.0.0.0rc1

This issue was fixed in the openstack/fuel-web 11.0.0.0rc1 release candidate.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-web (stable/mitaka)

Reviewed: https://review.openstack.org/427593
Committed: https://git.openstack.org/cgit/openstack/fuel-web/commit/?id=e6c922a82a7450afdbf7c75f6fbdfe3a9918ea7c
Submitter: Jenkins
Branch: stable/mitaka

commit e6c922a82a7450afdbf7c75f6fbdfe3a9918ea7c
Author: Alexander Kislitsky <email address hidden>
Date: Thu Jan 26 16:11:10 2017 +0300

    Message format on node(-s) deletion error changed

    In case of already running task we are returning JSON instead
    of plain text.

    Change-Id: Ic3998b8e534b2fa67813faf033335e849abe55af
    Closes-Bug: #1657350
    (cherry picked from commit e46b57befc0032448e12581dc64458170280d465)

Revision history for this message
Denis Meltsaykin (dmeltsaykin) wrote :

It turns out that this patch https://review.openstack.org/#/c/427566/ has to be also merged in order to finally fix the bug. Moving bug to 9.2-mu-3 to fix it properly.

Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Fix proposed to openstack/fuel-ui (9.0/mitaka)

Fix proposed to branch: 9.0/mitaka
Change author: Julia Aranovich <email address hidden>
Review: https://review.fuel-infra.org/36967

Revision history for this message
Alexey Stupnikov (astupnikov) wrote :

Fuel-ui patch works bad in our lab: after node removal it shows a dialogue window that hangs up and doesn't show OK button. It is possible to close it by simply clicking near this window, but it is still not ok.

On the other hand, I wasn't able to reproduce reproduce reported issue in our lab without this fuel-ui fix.

Revision history for this message
Alexey Stupnikov (astupnikov) wrote :

I was unable to reproduce this issue in a lab without fuel-ui fix. I believe that this bug was fixed with fuel-web patch and should be closed as Fix Released.

Please move it back to Confirmed if you will be able to reproduce it in a lab.

Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Change abandoned on openstack/fuel-ui (9.0/mitaka)

Change abandoned by Alexey Stupnikov <email address hidden> on branch: 9.0/mitaka
Review: https://review.fuel-infra.org/36967
Reason: It looks like the reported issue was fixed with fuel-web patch and there is no need to merge anything to fuel-ui.

Feel free to restore and merge this patch if the issue will be detected again.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.