KeyError: 'deployment_graph_task_name' in assert_all_tasks_completed in fuel-qa

Bug #1623123 reported by Alexandr Kostrikov
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Committed
High
Oleksandr
Nominated for Ocata by Alexander Kurenyshev
Mitaka
Fix Released
High
Oleksandr
Newton
Fix Released
High
Oleksandr

Bug Description

At custom job run [0] there were error:
ERROR: proboscis.case.MethodTest (gate_fuel_web)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jenkins/venv-nailgun-tests-2.9/local/lib/python2.7/site-packages/proboscis/case.py", line 296, in testng_method_mistake_capture_func
    compatability.capture_type_error(s_func)
  File "/home/jenkins/venv-nailgun-tests-2.9/local/lib/python2.7/site-packages/proboscis/compatability/exceptions_2_6.py", line 27, in capture_type_error
    func()
  File "/home/jenkins/venv-nailgun-tests-2.9/local/lib/python2.7/site-packages/proboscis/case.py", line 350, in func
    func(test_case.state.get_state())
  File "/home/jenkins/workspace/master.fuel-web.pkgs.ubuntu.review_fuel_web_deploy/fuel-qa/fuelweb_test/helpers/decorators.py", line 121, in wrapper
    result = func(*args, **kwargs)
  File "/home/jenkins/workspace/master.fuel-web.pkgs.ubuntu.review_fuel_web_deploy/fuel-qa/gates_tests/tests/test_review_fuel_web.py", line 87, in gate_fuel_web
    self.fuel_web.deploy_cluster_wait(cluster_id)
  File "/home/jenkins/workspace/master.fuel-web.pkgs.ubuntu.review_fuel_web_deploy/fuel-qa/fuelweb_test/helpers/decorators.py", line 463, in wrapper
    result = func(*args, **kwargs)
  File "/home/jenkins/workspace/master.fuel-web.pkgs.ubuntu.review_fuel_web_deploy/fuel-qa/fuelweb_test/helpers/decorators.py", line 448, in wrapper
    result = func(*args, **kwargs)
  File "/home/jenkins/workspace/master.fuel-web.pkgs.ubuntu.review_fuel_web_deploy/fuel-qa/fuelweb_test/helpers/decorators.py", line 499, in wrapper
    return func(*args, **kwargs)
  File "/home/jenkins/workspace/master.fuel-web.pkgs.ubuntu.review_fuel_web_deploy/fuel-qa/fuelweb_test/helpers/decorators.py", line 506, in wrapper
    result = func(*args, **kwargs)
  File "/home/jenkins/workspace/master.fuel-web.pkgs.ubuntu.review_fuel_web_deploy/fuel-qa/fuelweb_test/helpers/decorators.py", line 390, in wrapper
    return func(*args, **kwargs)
  File "/home/jenkins/workspace/master.fuel-web.pkgs.ubuntu.review_fuel_web_deploy/fuel-qa/fuelweb_test/models/fuel_web_client.py", line 950, in deploy_cluster_wait
    self.check_deploy_state(cluster_id, check_services, check_tasks)
  File "/home/jenkins/workspace/master.fuel-web.pkgs.ubuntu.review_fuel_web_deploy/fuel-qa/fuelweb_test/models/fuel_web_client.py", line 898, in check_deploy_state
    self.assert_all_tasks_completed(cluster_id=cluster_id)
  File "/home/jenkins/workspace/master.fuel-web.pkgs.ubuntu.review_fuel_web_deploy/fuel-qa/core/helpers/log_helpers.py", line 32, in wrapped
    result = func(*args, **kwargs)
  File "/home/jenkins/workspace/master.fuel-web.pkgs.ubuntu.review_fuel_web_deploy/fuel-qa/fuelweb_test/models/fuel_web_client.py", line 423, in assert_all_tasks_completed
    checkers.fail_deploy(not_ready_transactions)
  File "/home/jenkins/workspace/master.fuel-web.pkgs.ubuntu.review_fuel_web_deploy/fuel-qa/fuelweb_test/helpers/checkers.py", line 1435, in fail_deploy
    for cluster, records in sorted(not_ready_transactions.items())
  File "/home/jenkins/workspace/master.fuel-web.pkgs.ubuntu.review_fuel_web_deploy/fuel-qa/fuelweb_test/helpers/checkers.py", line 1435, in <genexpr>
    for cluster, records in sorted(not_ready_transactions.items())
  File "/home/jenkins/workspace/master.fuel-web.pkgs.ubuntu.review_fuel_web_deploy/fuel-qa/fuelweb_test/helpers/checkers.py", line 1433, in <genexpr>
    for node_id, tasks in sorted(records.items())
  File "/home/jenkins/workspace/master.fuel-web.pkgs.ubuntu.review_fuel_web_deploy/fuel-qa/fuelweb_test/helpers/checkers.py", line 1429, in <genexpr>
    for task in sorted(
KeyError: 'deployment_graph_task_name'

Seems, that value is searched in dicts like
{u'status': u'skipped', u'time_start': u'2016-09-13T16:15:52.830638', u'groups': [u'primary-controller'], u'parameters': {u'puppet_modules': u'/etc/puppet/modules', u'puppet_manifest': u'/etc/puppet/modules/osnailyfacter/modular/ceph/radosgw_keystone.pp', u'timeout': 1800}, u'requires': [u'keystone', u'primary-mon-update'], u'cross-depends': [{u'name': u'keystone'}], u'time_end': u'2016-09-13T16:15:52.830653', u'custom': {}, u'version': u'2.1.0', u'condition': {u'yaql_exp': u"$.storage.objects_ceph and changedAny($.storage, $.network_metadata.vips, $.get('region', 'RegionOne'), $.public_ssl, $.get('use_ssl'))"}, u'task_name': u'radosgw-keystone', u'required_for': [u'ceph-radosgw'], u'type': u'puppet', u'cross_depends': [{u'name': u'keystone'}], u'node_id': u'3'}

[0] https://ci.fuel-infra.org/job/master.fuel-web.pkgs.ubuntu.review_fuel_web_deploy/4369/console

Tags: area-qa
Changed in fuel:
status: New → In Progress
Revision history for this message
Aleksei Stepanov (penguinolog) wrote :

https://custom-ci.infra.mirantis.net/job/10.0.custom.system_test/1225/console - debug test run was not reproduced this bug.
Will monitor until failure.

Maksym Strukov (unbelll)
Changed in fuel:
assignee: Alexey Stepanov (astepanov-m) → Fuel QA Team (fuel-qa)
status: In Progress → Confirmed
Revision history for this message
Dmitry Belyaninov (dbelyaninov) wrote :
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-qa (master)

Fix proposed to branch: master
Review: https://review.openstack.org/391794

Changed in fuel:
assignee: Fuel QA Team (fuel-qa) → Oleksandr (oivashchenko)
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-qa (master)

Reviewed: https://review.openstack.org/391794
Committed: https://git.openstack.org/cgit/openstack/fuel-qa/commit/?id=96100c74d120216e8065caa875da6dbdc90e505b
Submitter: Jenkins
Branch: master

commit 96100c74d120216e8065caa875da6dbdc90e505b
Author: Oleksandr Ivashchenko <email address hidden>
Date: Mon Oct 31 13:27:53 2016 +0200

    Change key-name

    Parameter name was changed to the correct

    Closes-Bug: #1623123
    Change-Id: I00a409d9102b565c39ef04dca232d6990b759d07

Changed in fuel:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-qa (stable/mitaka)

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/393743

Changed in fuel:
milestone: 10.0 → 11.0
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-qa (stable/newton)

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/394359

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-qa (stable/mitaka)

Reviewed: https://review.openstack.org/393743
Committed: https://git.openstack.org/cgit/openstack/fuel-qa/commit/?id=4a7943d97d609b7528689d0a6b3746f5c454e508
Submitter: Jenkins
Branch: stable/mitaka

commit 4a7943d97d609b7528689d0a6b3746f5c454e508
Author: Oleksandr Ivashchenko <email address hidden>
Date: Mon Oct 31 13:27:53 2016 +0200

    Change key-name

    Parameter name was changed to the correct

    Closes-Bug: #1623123
    Change-Id: I00a409d9102b565c39ef04dca232d6990b759d07
    (cherry picked from commit 96100c74d120216e8065caa875da6dbdc90e505b)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-qa (stable/newton)

Reviewed: https://review.openstack.org/394359
Committed: https://git.openstack.org/cgit/openstack/fuel-qa/commit/?id=5d464d1528eaba612ea3cdcead25da2d08e2f231
Submitter: Jenkins
Branch: stable/newton

commit 5d464d1528eaba612ea3cdcead25da2d08e2f231
Author: Oleksandr Ivashchenko <email address hidden>
Date: Mon Oct 31 13:27:53 2016 +0200

    Change key-name

    Parameter name was changed to the correct

    Closes-Bug: #1623123
    Change-Id: I00a409d9102b565c39ef04dca232d6990b759d07
    (cherry picked from commit 96100c74d120216e8065caa875da6dbdc90e505b)

tags: added: on-verification
Revision history for this message
Ekaterina Shutova (eshutova) wrote :

Verified on snapshot #495.
No failures with this simptoms were found.

tags: removed: on-verification
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/fuel-qa 11.0.0.0rc1

This issue was fixed in the openstack/fuel-qa 11.0.0.0rc1 release candidate.

Revision history for this message
Alexander Kurenyshev (akurenyshev) wrote :

The last run was successful [0]
Thus bug moved to Fix released
[0] https://ci.fuel-infra.org/job/master.fuel-web.pkgs.ubuntu.review_fuel_web_deploy/6207/

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.