Heat ostf Typical stack actions: create, delete, show details failed by timeout

Bug #1525200 reported by Tatyanka
22
This bug affects 4 people
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Released
High
Alexander Nagovitsyn
8.0.x
Won't Fix
High
Alexander Nagovitsyn

Bug Description

Deploy HA environment with Cinder, Neutron and network template

Scenario:
1. Revert snapshot with 3 slaves
2. Create cluster (HA) with Neutron VLAN/VXLAN/GRE
3. Add 1 controller + cinder nodes
4. Add 2 compute + cinder nodes
5. Upload 'cinder' network template'
6. Create custom network groups basing
on template endpoints assignments
7. Run network verification
8. Deploy cluster
9. Run network verification
10. Run health checks (OSTF)
11. Check L3 network configuration on slaves
12. Check that services are listening on their networks only

Actual:
heat test stack creation fails:
 - Typical stack actions: create, delete, show details, etc. (failure) Time limit exceeded while waiting for stack creation to finish. Please refer to OpenStack logs for more details.

VERSION:
  feature_groups:
    - mirantis
  production: "docker"
  release: "8.0"
  api: "1.0"
  build_number: "283"
  build_id: "283"
  fuel-nailgun_sha: "073beb9f02e103e8f78abe9425d4c9feb5fab6e4"
  python-fuelclient_sha: "f96659066e522e28b389de3cc685f6f2aacca3da"
  fuel-agent_sha: "2f18b7596bc7da79d2f28c34f42620b2090d8a35"
  fuel-nailgun-agent_sha: "a33a58d378c117c0f509b0e7badc6f0910364154"
  astute_sha: "f7df18ab6b2089649abcfa8cf667ffd6b332be1a"
  fuel-library_sha: "66fc43373039a23fb63c26e7919a60b3560e0db1"
  fuel-ostf_sha: "632730169e8c01afe7fd5d78a898f00d4646358b"
  fuel-mirror_sha: "31b9df814960ec69b644ca9b689dacec0c7e10a1"
  fuelmenu_sha: "680b720291ff577f4c058cee25f85e563c96312e"
  shotgun_sha: "a0bd06508067935f2ae9be2523ed0d1717b995ce"
  network-checker_sha: "a3534f8885246afb15609c54f91d3b23d599a5b1"
  fuel-upgrade_sha: "1e894e26d4e1423a9b0d66abd6a79505f4175ff6"
  fuelmain_sha: "1577a306c2c9e7bd12f28c0e16cf3652997da004"

Tags: area-qa heat
Revision history for this message
Tatyanka (tatyana-leontovich) wrote :
summary: - Heat ostf Typical stack actions: create, delete, show details аailed by
+ Heat ostf Typical stack actions: create, delete, show details failed by
timeout
Changed in fuel:
status: New → Confirmed
tags: added: heat
Revision history for this message
Alexander Nagovitsyn (gluk12189) wrote :

When I increased timeouts for this ostf tests - the test is successful
Default timeouts = 20 s
I change it to 50 s
For 7.0 quite enough defaults

Revision history for this message
Sergey Kraynev (skraynev) wrote :

Tatyanka: I think, that it's common issue for all services. I don;t know why, but response from each command in CLI takes more time, then expected.

Revision history for this message
Alexander Nagovitsyn (gluk12189) wrote :

As I see - it is so
I see a problem at all services, not only on the hit

Revision history for this message
Timur Nurlygayanov (tnurlygayanov) wrote :

It looks like this issue will be reproduced only slow virtual environments.
We need to increase timeout for this tests in the future.

Changed in fuel:
milestone: 8.0 → 9.0
Revision history for this message
Tatyanka (tatyana-leontovich) wrote :

+1 :)

Revision history for this message
Alexander Nagovitsyn (gluk12189) wrote :

ok! i rewrite this tests

Changed in fuel:
status: Confirmed → In Progress
Revision history for this message
Timur Nurlygayanov (tnurlygayanov) wrote :

We don't have a fix on review for MOS 9.0, status changed to Confirmed for 9.0 release.
Status should be Won't Fix for 8.0 release because we are in HCF now.

Changed in fuel:
status: In Progress → Confirmed
Changed in fuel:
assignee: MOS QA Team (mos-qa) → Alexander Nagovitsyn (gluk12189)
Changed in fuel:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-ostf (master)

Reviewed: https://review.openstack.org/287229
Committed: https://git.openstack.org/cgit/openstack/fuel-ostf/commit/?id=aa8375acf44f730df5de1df4202ada2c7a7be4f2
Submitter: Jenkins
Branch: master

commit aa8375acf44f730df5de1df4202ada2c7a7be4f2
Author: Alexandr Nagovitsyn <email address hidden>
Date: Wed Mar 2 17:32:16 2016 +0300

    Add RAM filters and increased the timers in heat tests

    Recently, a few new bugs associated with timers and error
    'failed by timeout on step wait stack CREATE_COMPLETE status'
    Research has shown that an error in most cases occurs due to a
    slow environments.
    Typically, a small amount of RAM (<3 GB).
    Tests autoscaling and wait condition require more than 7 GB
    Also increased the other timers in (+ 120s in verify steps,
    + 30s in create stack steps, +10s in some other.

    Change-Id: I8066581e6358e84655ac3b419dcaa324e4f57aca
    Closes-bug:#1544179
    Closes-bug:#1539721
    Closes-bug:#1525200

Changed in fuel:
status: In Progress → Fix Committed
Revision history for this message
Tatyanka (tatyana-leontovich) wrote :

Fix was merged only yesterday , so we need fresh iso to verify this issue

Revision history for this message
Alexander Nagovitsyn (gluk12189) wrote :

verified on new build ( 2016-03-23 #101) - test passed

Changed in fuel:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.