fuel-ostf heat-autoscaling fails on complicated environments with timeout error
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Fuel for OpenStack |
Fix Released
|
High
|
Timur Nurlygayanov | ||
8.0.x |
Fix Released
|
High
|
MOS Maintenance | ||
Mitaka |
Fix Released
|
High
|
Timur Nurlygayanov | ||
Newton |
Fix Released
|
High
|
Timur Nurlygayanov |
Bug Description
Hello.
In our environments ostf fails on heat autoscaling test with timeout. Apparently when there is workload on the server 720 seconds is not enough to realise workload on VMs have changed. I'd suggest to change it to 1300 or something like this.
"""Create an AIC large HA environment with Contrail.
Scenario:
1. Revert snapshot "ready".
2. Install plugins.
3. Create a cluster.
4. Bootstrap 21 nodes.
5. Add 3 nodes with "aic-haproxy" role.
6. Add 3 nodes with "aic-controller", "aic-swift-proxy" and
"cinder" roles.
7. Add 2 nodes with "aic-identity" role.
8. Add 3 nodes with "aic-dbng" role.
9. Add 3 nodes with "mongo" role.
10. Add 3 nodes with "aic-swift-storage" role.
11. Add 1 node with "aic-compute" role.
12. Add 3 nodes with "contrail-config", "contrail-control",
"contrail-db" roles
13. Deploy the created cluster
14. Run OSTF <------ Fails here
Changed in fuel: | |
milestone: | none → 9.0 |
assignee: | nobody → Fuel QA Team (fuel-qa) |
importance: | Undecided → High |
status: | New → Confirmed |
tags: | added: area-qa |
tags: |
added: area-ostf removed: area-qa |
tags: | added: on-verification |
tags: | removed: on-verification |
tags: | added: on-verification |
Note that it's intermittent issue. But it's really annoying to hit this issue "from time to time".
Here is exact trace from this error:
2016-05-17 22:29:40 FAILURE Check stack autoscaling (fuel_health. tests.tests_ platform. test_heat. HeatSmokeTests. test_autoscalin g) Time limit exceeded while waiting for terminating the 2nd instance per autoscaling alarm to finish. Please refer to OpenStack logs for more details. File "/usr/lib/ python2. 7/site- packages/ unittest2/ case.py" , line 67, in testPartExecutor python2. 7/site- packages/ unittest2/ case.py" , line 601, in run python2. 7/site- packages/ fuel_health/ tests/tests_ platform/ test_heat. py", line 704, in test_autoscaling python2. 7/site- packages/ fuel_health/ common/ test_mixins. py", line 183, in verify python2. 7/site- packages/ unittest2/ case.py" , line 666, in fail eption( msg)
yield
File "/usr/lib/
testMethod()
File "/usr/lib/
len(instances) + 1, 720, 10, reduced_stack_name
File "/usr/lib/
" Please refer to OpenStack logs for more details.")
File "/usr/lib/
raise self.failureExc
Step 11 failed: Time limit exceeded while waiting for terminating the 2nd instance per autoscaling alarm to finish. Please refer to OpenStack logs for more details.