master fuel-agent job is failed: got an error at ironic-conductor deployment task

[Dmitry Kaigarodеsev]

URL to canary job: https://ci.fuel-infra.org/view/deployment%20tests/job/master.fuel-agent.pkgs.ubuntu.review_fuel_agent_ironic_deploy/526/console

======================================================================
FAIL: Revert snapshot, update fuel-agent, bootstrap from review
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jenkins/venv-nailgun-tests-2.9/local/lib/python2.7/site-packages/proboscis/case.py", line 296, in testng_method_mistake_capture_func
    compatability.capture_type_error(s_func)
  File "/home/jenkins/venv-nailgun-tests-2.9/local/lib/python2.7/site-packages/proboscis/compatability/exceptions_2_6.py", line 27, in capture_type_error
    func()
  File "/home/jenkins/venv-nailgun-tests-2.9/local/lib/python2.7/site-packages/proboscis/case.py", line 350, in func
    func(test_case.state.get_state())
  File "/home/jenkins/workspace/master.fuel-agent.pkgs.ubuntu.review_fuel_agent_ironic_deploy/fuel-qa/fuelweb_test/helpers/decorators.py", line 117, in wrapper
    result = func(*args, **kwargs)
  File "/home/jenkins/workspace/master.fuel-agent.pkgs.ubuntu.review_fuel_agent_ironic_deploy/fuel-qa/gates_tests/tests/test_review_in_fuel_agent.py", line 114, in gate_patch_fuel_agent
    cluster_id = self._deploy_ironic_cluster(settings=data, nodes=nodes)
  File "/home/jenkins/workspace/master.fuel-agent.pkgs.ubuntu.review_fuel_agent_ironic_deploy/fuel-qa/fuelweb_test/tests/test_ironic_base.py", line 118, in _deploy_ironic_cluster
    self.fuel_web.deploy_cluster_wait(cluster_id)
  File "/home/jenkins/workspace/master.fuel-agent.pkgs.ubuntu.review_fuel_agent_ironic_deploy/fuel-qa/fuelweb_test/helpers/decorators.py", line 459, in wrapper
    result = func(*args, **kwargs)
  File "/home/jenkins/workspace/master.fuel-agent.pkgs.ubuntu.review_fuel_agent_ironic_deploy/fuel-qa/fuelweb_test/helpers/decorators.py", line 444, in wrapper
    result = func(*args, **kwargs)
  File "/home/jenkins/workspace/master.fuel-agent.pkgs.ubuntu.review_fuel_agent_ironic_deploy/fuel-qa/fuelweb_test/helpers/decorators.py", line 495, in wrapper
    return func(*args, **kwargs)
  File "/home/jenkins/workspace/master.fuel-agent.pkgs.ubuntu.review_fuel_agent_ironic_deploy/fuel-qa/fuelweb_test/helpers/decorators.py", line 502, in wrapper
    result = func(*args, **kwargs)
  File "/home/jenkins/workspace/master.fuel-agent.pkgs.ubuntu.review_fuel_agent_ironic_deploy/fuel-qa/fuelweb_test/helpers/decorators.py", line 386, in wrapper
    return func(*args, **kwargs)
  File "/home/jenkins/workspace/master.fuel-agent.pkgs.ubuntu.review_fuel_agent_ironic_deploy/fuel-qa/fuelweb_test/models/fuel_web_client.py", line 934, in deploy_cluster_wait
    self.assert_task_success(task, interval=interval, timeout=timeout)
  File "/home/jenkins/workspace/master.fuel-agent.pkgs.ubuntu.review_fuel_agent_ironic_deploy/fuel-qa/fuelweb_test/__init__.py", line 60, in wrapped
    result = func(*args, **kwargs)
  File "/home/jenkins/workspace/master.fuel-agent.pkgs.ubuntu.review_fuel_agent_ironic_deploy/fuel-qa/fuelweb_test/models/fuel_web_client.py", line 336, in assert_task_success
    task["name"], task['status'], 'ready', _message(task)
AssertionError: Task 'deploy' has incorrect status. error != ready, 'Deployment has failed. All nodes are finished. Failed tasks: Task[controller_remaining_tasks/1], Task[ironic-conductor/2] Stopping the deployment process!'

[Alexander Gordeev (a-gordeev)]

Updated original description, it's failing at the deployment step, not at the authentification.

[Alexander Gordeev (a-gordeev)]

it seems like the first failure with the same traces was the following one:

https://ci.fuel-infra.org/job/master.fuel-agent.pkgs.ubuntu.review_fuel_agent_ironic_deploy/508/console

[Nastya Urlapova (aurlapova)]

@Alex, why did you assign QA team on issue with deployment? Assignment was changed to fuel-sustaining.

[Oleksiy Molchanov (omolchanov)]

<178>Aug 11 13:45:47 node-1 nova-cert: 2016-08-11 13:45:47.145 22890 CRITICAL nova [req-a815fcce-e3b0-4e88-b1d4-64a4e6210d26 - - - - -] ConfigFileValueError: Value for option scheduler_host_manager is not valid: Valid values are [host_manager, ironic_host_manager], but found 'nova.scheduler.ironic_host_manager.IronicHostManager'

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/354126

[Alexander Gordeev (a-gordeev)]

@Nastya, sorry, it wasn't me, i didn't touch the assignments, only updated the information.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/354126
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=6fb4e9f761798af2616c237baf07f4cf20e4faaa
Submitter: Jenkins
Branch: master

commit 6fb4e9f761798af2616c237baf07f4cf20e4faaa
Author: Oleksiy Molchanov <email address hidden>
Date: Thu Aug 11 17:42:38 2016 +0300

    Change value for Ironic scheduler_host_manager

    Change-Id: I4f25f74d186be97e363e37264d49d7d8ecf34beb
    Closes-Bug: 1611689

[Oleksiy Molchanov (omolchanov)]

Seems it is not affecting mitaka - https://ci.fuel-infra.org/view/deployment%20tests/job/mitaka.fuel-agent.pkgs.ubuntu.review_fuel_agent_ironic_deploy/

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/fuel-library 10.0.0rc1 release candidate.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/fuel-library 10.0.0 release.