need to rework filling_root test

Bug #1584097 reported by Artem Hrechanychenko on 2016-05-20
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
High
Unassigned
Mitaka
High
Unassigned
Newton
High
Unassigned

Bug Description

Detailed bug description:
need rework filling_root test because number of running resources on controller may be not equal as at beginning of the test:
root@node-1:~# pcs status xml
<?xml version="1.0"?>
<crm_mon version="1.1.14">
    <summary>
        <last_update time="Fri May 20 14:51:32 2016" />
        <last_change time="Fri May 20 14:40:21 2016" user="root" client="cibadmin" origin="node-1.test.domain.local" />
        <stack type="corosync" />
        <current_dc present="true" version="1.1.14-70404b0" name="node-1.test.domain.local" id="1" with_quorum="true" />
        <nodes_configured number="3" expected_votes="unknown" />
        <resources_configured number="46" />
        <cluster_options stonith-enabled="false" symmetric-cluster="false" no-quorum-policy="stop" />
    </summary>
    <nodes>
        <node name="node-1.test.domain.local" id="1" online="true" standby="false" standby_onfail="false" maintenance="false" pending="false" unclean="false" shutdown="false" expected_up="true" is_dc="true" resources_running="14" type="member" />
        <node name="node-2.test.domain.local" id="2" online="true" standby="false" standby_onfail="false" maintenance="false" pending="false" unclean="false" shutdown="false" expected_up="true" is_dc="false" resources_running="16" type="member" />
        <node name="node-3.test.domain.local" id="3" online="true" standby="false" standby_onfail="false" maintenance="false" pending="false" unclean="false" shutdown="false" expected_up="true" is_dc="false" resources_running="16" type="member" />

we see that resources_configured number="46" and number of resources_running on nodes equal 46 = 14+16+16

at beginning of the test - 2016-05-19 23:49:14,793 - INFO strength_actions.py:352 -- Checking for 'running_resources' attribute have 15 value on node node-1.test.domain.local

node-1 have 15 running resources, but at the end - have 14 after recover space on root partition and it's not a failure because 0 resources not started

Steps to reproduce:
Scenario:
    0. Setup master node
    1. Configuration releases
    2. Bootstrap slave and make snapshot
    3. Revert bootstrapped nodes
    4. Create Fuel Environment
    5. Add nodes to environment
    6. Run network checker
    7. Deploy environment
    8. Run network checker
    9. Run health checker
    10. Load existent environment from snapshot or save it
    11. Get controllers initial status in pacemaker
    12. Filling root filesystem on primary controller
    13. Check stopping pacemaker resources
    14. Run health checker
    15. Fill root more to below rabbit disk free limit
    16. Check stopping pacemaker resources
    17. Run health checker
    18. Clean up space on root filesystem on primary controller
    19. Check starting pacemaker resources <<<<<<<<<<<failed here
    20. Run health checker Sanity, Smoke and HA

Expected results:
  test passed

Actual result:
  TimeoutError: Waiting timed out when waiting for number of available resources

Reproducibility:
  https://product-ci.infra.mirantis.net/job/9.0.system_test.ubuntu.filling_root/112/console

Workaround:
  - - -

summary: - need rework filling_root test
+ need to rework filling_root test
Changed in fuel:
status: New → Confirmed
tags: added: swarm-fail

Fix proposed to branch: master
Review: https://review.openstack.org/339609

Changed in fuel:
assignee: Fuel QA Team (fuel-qa) → Maksym Strukov (unbelll)
status: Confirmed → In Progress

Reviewed: https://review.openstack.org/339609
Committed: https://git.openstack.org/cgit/openstack/fuel-qa/commit/?id=951d250b1dadce94d56b23aade301755119a6a98
Submitter: Jenkins
Branch: master

commit 951d250b1dadce94d56b23aade301755119a6a98
Author: Maksym Strukov <email address hidden>
Date: Fri Jul 8 18:16:58 2016 +0300

    Fix filling_root test

    As running resources on controller may be not equal as at beginning
    of the test, let's use 'Check pacemaker status' sanity OSTF test.

    Change-Id: I6ad59702fb04d5f7b2fb97cc257b763c53ee2fc7
    Closes-Bug: #1584097

Changed in fuel:
status: In Progress → Fix Committed
tags: added: swarm-blocker
removed: swarm-fail
Maksym Strukov (unbelll) on 2016-07-20
no longer affects: fuel/8.0.x

Reviewed: https://review.openstack.org/344708
Committed: https://git.openstack.org/cgit/openstack/fuel-qa/commit/?id=154869be2f01527687b683edb363cfe4fca8a91d
Submitter: Jenkins
Branch: stable/mitaka

commit 154869be2f01527687b683edb363cfe4fca8a91d
Author: Maksym Strukov <email address hidden>
Date: Fri Jul 8 18:16:58 2016 +0300

    Fix filling_root test

    As running resources on controller may be not equal as at beginning
    of the test, let's use 'Check pacemaker status' sanity OSTF test.

    Change-Id: I6ad59702fb04d5f7b2fb97cc257b763c53ee2fc7
    Closes-Bug: #1584097

Maksym Strukov (unbelll) on 2017-04-06
Changed in fuel:
assignee: Maksym Strukov (unbelll) → nobody
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers