Nailgun does not check nodes roles before the first deployment

Bug #1605472 reported by Sergey Kreys
14
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Committed
High
Georgy Kibardin
Mitaka
Fix Released
High
Georgy Kibardin

Bug Description

It happens because function _check_nodes_roles uses method TaskHelper.nodes_to_deploy(cluster) to get the list of nodes. Function nodes_to_deploy returns an empty list when all nodes in the cluster are in 'discovered' state. Thus, no checks are done.

Steps to reproduce:
  - create cluster with role 'cinder' on some node;
  - define a restriction for role 'cinder', e.g. 'cinder' role should be hidden or disabled according to restriction's condition;
  - deploy cluster.

Expected results:
  check_before_deployment task should fail with error that restriction was hit: "Role 'cinder' restrictions mismatch: <message_specified_in_restriction>".

Actual result:
  Deployment passed. As a result, we have a cluster with node role that doesn't satisfy roles_metadata.
  And further attempts to deploy changes for this cluster will fail with error described in "Expected results" above.
  Even if we just add a new node with not-affected role 'compute' and run deploy, function _check_nodes_roles will go through all the nodes and will fail on node that has affected role 'cinder'.

Reproducibility:
  100%

Description of the environment:
  fuel-web branch is stable/mitaka

Additional information:
  commit, where restrictions check was introduced: https://review.openstack.org/#/c/316870/2
  commit, where the logic of nodes_to_deploy(cluster) was rewritten: https://review.openstack.org/#/c/309769/4

Revision history for this message
Dmitry Pyzhov (dpyzhov) wrote :

Let's postpone fixing the bug until end of next week. There is an refactoring in progress in this area that will make the fix easier.

Changed in fuel:
milestone: none → 10.0
assignee: nobody → Fuel Sustaining (fuel-sustaining-team)
status: New → Confirmed
importance: Undecided → High
tags: added: area-python
Changed in fuel:
assignee: Fuel Sustaining (fuel-sustaining-team) → Georgy Kibardin (gkibardin)
Changed in fuel:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-web (master)

Fix proposed to branch: master
Review: https://review.openstack.org/347694

Dmitry Pyzhov (dpyzhov)
tags: added: 9.1-proposed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-web (master)

Reviewed: https://review.openstack.org/347694
Committed: https://git.openstack.org/cgit/openstack/fuel-web/commit/?id=bb37dbd4a9472c7a40ea8eb768bdbe5d161a0425
Submitter: Jenkins
Branch: master

commit bb37dbd4a9472c7a40ea8eb768bdbe5d161a0425
Author: Georgy Kibardin <email address hidden>
Date: Wed Jul 27 11:33:36 2016 +0300

    Check role restriction also before provisioning

    Currently the nodes which need provisioning and then deployment avoid role
    restrictions check because of their 'not ready for deployment'
    status. This commit adds unprovisioned nodes to the list of checked
    nodes.

    Change-Id: I7b19d1beeb71849e2def465127873821a4f93052
    Closes-Bug: #1605472

Changed in fuel:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-web (stable/mitaka)

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/357916

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-web (stable/mitaka)

Reviewed: https://review.openstack.org/357916
Committed: https://git.openstack.org/cgit/openstack/fuel-web/commit/?id=7b6e2c65b54133fa5014453c37ac245b515c79ae
Submitter: Jenkins
Branch: stable/mitaka

commit 7b6e2c65b54133fa5014453c37ac245b515c79ae
Author: Georgy Kibardin <email address hidden>
Date: Wed Jul 27 11:33:36 2016 +0300

    Check role restriction also before provisioning

    Currently the nodes which need provisioning and then deployment avoid role
    restrictions check because of their 'not ready for deployment'
    status. This commit adds unprovisioned nodes to the list of checked
    nodes.

    Change-Id: I7b19d1beeb71849e2def465127873821a4f93052
    Closes-Bug: #1605472
    (cherry picked from commit bb37dbd4a9472c7a40ea8eb768bdbe5d161a0425)

tags: added: on-verification
tags: removed: on-verification
Oleksandr (oivashchenko)
tags: added: on-verification
Oleksandr (oivashchenko)
tags: added: hard-to-verify
removed: on-verification
Revision history for this message
Maksym Strukov (unbelll) wrote :

Actual repro steps is:
1. Create cluster with default settings
2. Add controller, compute and cinder nodes
3. GET /api/clusters/1/attributes and save json data to file.
4. Edit /editable/storage/volumes_lvm/value to 'false'
5. POST -X PUT -d @file_with_edited_json_data
6. Open Cluster Settings tab and ensure Cinder is disabled
7. Start add Cinder node to cluster and ensure you can't do that
8. Deploy changes

Actual/Expected:
Error
Role 'cinder' restrictions mismatch: Cinder LVM should be enabled in the environment settings.

Verified as fixed in 9.1 snapshot proposed-2016-09-22-224322

tags: added: on-verification
tags: removed: on-verification
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/fuel-web 10.0.0rc1

This issue was fixed in the openstack/fuel-web 10.0.0rc1 release candidate.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/fuel-web 10.0.0

This issue was fixed in the openstack/fuel-web 10.0.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.