Sahara OSTF test failed without UI notification

Bug #1401036 reported by Dennis Dmitriev
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Invalid
Medium
Yaroslav Lobankov
5.1.x
Won't Fix
Medium
Yaroslav Lobankov
6.0.x
Invalid
Medium
Yaroslav Lobankov
6.1.x
Invalid
Medium
Yaroslav Lobankov

Bug Description

Cluster configuration:
    - Ubuntu/simple,
    - Neutron/GRE,
    - Network without tagging, eth0:admin, eth1:public, eth2:storage,
          eth3+eth4 = ovs-bond0 : management
    - Ceph for volumes
    - Ceph for images
    - Sahara
    - Murano

Nodes configuration:
    - 1 controller (+ ceph) / 4 CPU, 4096Mb RAM
    - 2 compute / 3 CPU, 6144Mb RAM
    - 2 ceph / 1 CPU, 3072Mb RAM

Deployment of the cluster was successful, all OSTF tests were passed except two platform tests for Sahara and Murano.
Murano showed ''success' != u'completed_w_errors' Please refer to OpenStack logs for more details.' error.

Sahara showed no messages in UI at all. Two instances have been running for ~14 hours, but Sahara cluster is still in 'Waiting' status.

Sahara error in OSTF log: http://paste.openstack.org/show/148473/

Revision history for this message
Dennis Dmitriev (ddmitriev) wrote :
Revision history for this message
Dennis Dmitriev (ddmitriev) wrote :
Revision history for this message
Dennis Dmitriev (ddmitriev) wrote :
Revision history for this message
Dennis Dmitriev (ddmitriev) wrote :
Revision history for this message
Dennis Dmitriev (ddmitriev) wrote :

{"build_id": "2014-12-03_01-07-36", "ostf_sha": "64cb59c681658a7a55cc2c09d079072a41beb346", "build_number": "48", "auth_required": true, "api": "1.0", "nailgun_sha": "500e36d08a45dbb389bf2bd97673d9bff48ee84d", "production": "docker", "fuelmain_sha": "7626c5aeedcde77ad22fc081c25768944697d404", "astute_sha": "ef8aa0fd0e3ce20709612906f1f0551b5682a6ce", "feature_groups": ["mirantis"], "release": "5.1.1", "release_versions": {"2014.1.3-5.1.1": {"VERSION": {"build_id": "2014-12-03_01-07-36", "ostf_sha": "64cb59c681658a7a55cc2c09d079072a41beb346", "build_number": "48", "api": "1.0", "nailgun_sha": "500e36d08a45dbb389bf2bd97673d9bff48ee84d", "production": "docker", "fuelmain_sha": "7626c5aeedcde77ad22fc081c25768944697d404", "astute_sha": "ef8aa0fd0e3ce20709612906f1f0551b5682a6ce", "feature_groups": ["mirantis"], "release": "5.1.1", "fuellib_sha": "a3043477337b4a0a8fd166dc83d6cd5d504f5da8"}}}, "fuellib_sha": "a3043477337b4a0a8fd166dc83d6cd5d504f5da8"}

Changed in fuel:
importance: Undecided → Medium
assignee: nobody → Yaroslav Lobankov (ylobankov)
status: New → Confirmed
Revision history for this message
Yaroslav Lobankov (ylobankov) wrote :

Fuel cluster nodes were KVM virtual machines? Or you used VBox scripts to build cluster?

Revision history for this message
Dennis Dmitriev (ddmitriev) wrote :

I used qemu/kvm with fuel-devops to create the environment. But resources for slaves were changed manually.

Changed in fuel:
milestone: 5.1.2 → 6.1
status: Confirmed → New
Changed in fuel:
status: New → Confirmed
Revision history for this message
Dmitry Mescheryakov (dmitrymex) wrote :

That is a medium bug, we are not going to fix it in maintenance releases 5.1.2 and 6.0.1

Revision history for this message
Dmitry Mescheryakov (dmitrymex) wrote :

Pushing the bug to 7.0 as we re close to SCF

no longer affects: fuel/7.0.x
Changed in fuel:
milestone: 6.1 → 7.0
tags: added: release-notes
tags: added: qa-agree-7.0
Revision history for this message
Yaroslav Lobankov (ylobankov) wrote :

It looks like the problem was in security groups. In MOS 5.1.1 before we run Sahara test we have to open some security group ports manually, for example 22 (SSH port). If we forget to do this, the Sahara cluster will hang in "Waiting" state, trying to connect to instances via SSH. In logs I see the following:

2014-12-10T07:00:39.738002+00:00 debug: 2014-12-10 07:00:39.709 4234 DEBUG sahara.service.engine [-] Can't login to node ostf-cluster-1378389495-ostf-test-master-001 (10.108.30.152), reason error: [Errno 110] Connection timed out _wait_until_accessible /usr/lib/python2.7/dist-packages/sahara/service/engine.py:95

Anyway, now these ports are opened automatically and this issue is no longer actual since MOS 6.0.1.

Changed in fuel:
status: Confirmed → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.