9.x BVT jobs [0], [1], [2] failed with the following trace:
2016-12-09 09:02:21 - ERROR decorators.py:126 -- Traceback (most recent call last):
File "/home/jenkins/workspace/9.x.main.ubuntu.bvt_2/fuelweb_test/helpers/decorators.py", line 120, in wrapper
result = func(*args, **kwargs)
File "/home/jenkins/workspace/9.x.main.ubuntu.bvt_2/fuelweb_test/tests/test_ceph.py", line 507, in ceph_rados_gw
self.fuel_web.deploy_cluster_wait(cluster_id)
File "/home/jenkins/workspace/9.x.main.ubuntu.bvt_2/fuelweb_test/helpers/decorators.py", line 460, in wrapper
result = func(*args, **kwargs)
File "/home/jenkins/workspace/9.x.main.ubuntu.bvt_2/fuelweb_test/helpers/decorators.py", line 445, in wrapper
result = func(*args, **kwargs)
File "/home/jenkins/workspace/9.x.main.ubuntu.bvt_2/fuelweb_test/helpers/decorators.py", line 496, in wrapper
return func(*args, **kwargs)
File "/home/jenkins/workspace/9.x.main.ubuntu.bvt_2/fuelweb_test/helpers/decorators.py", line 503, in wrapper
result = func(*args, **kwargs)
File "/home/jenkins/workspace/9.x.main.ubuntu.bvt_2/fuelweb_test/helpers/decorators.py", line 387, in wrapper
return func(*args, **kwargs)
File "/home/jenkins/workspace/9.x.main.ubuntu.bvt_2/fuelweb_test/models/fuel_web_client.py", line 953, in deploy_cluster_wait
self.assert_task_success(task, interval=interval, timeout=timeout)
File "/home/jenkins/workspace/9.x.main.ubuntu.bvt_2/core/helpers/log_helpers.py", line 215, in wrapped
result = func(*args, **kwargs)
File "/home/jenkins/workspace/9.x.main.ubuntu.bvt_2/fuelweb_test/models/fuel_web_client.py", line 354, in assert_task_success
task["name"], task['status'], 'ready', _message(task)
File "/home/jenkins/venv-nailgun-tests-2.9/local/lib/python2.7/site-packages/proboscis/asserts.py", line 55, in assert_equal
raise ASSERTION_ERROR(message)
AssertionError: Task 'deploy' has incorrect status. error != ready, 'Deployment has failed. All nodes are finished. Failed tasks: Task[tools/1], Task[tools/3], Task[tools/2], Task[tools/5], Task[tools/4], Task[tools/6] Stopping the deployment process!'
Errors in astute log are the same for each of the nodes:
2016-12-09 08:59:59 ERROR [15379] Node 4, task tools, manifest /etc/puppet/modules/osnailyfacter/modular/tools/tools.pp, status: stopped
2016-12-09 08:59:59 ERROR [15379] Node has failed to deploy. There is no more retries for puppet run. Node 4, task tools, manifest /etc/puppet/modules/osnailyfacter/modular/tools/tools.pp
Errors in puppet-apply.log on nodes [3] show that execution of `/bin/systemctl enable atop` and `/bin/systemctl enable postfix` fail with return code 1: `Failed to issue method call: No such file or directory`.
[0] https://product-ci.infra.mirantis.net/job/9.x.main.ubuntu.bvt_2/614
[1] https://product-ci.infra.mirantis.net/job/9.x.main.ubuntu.bvt_2/613/
[2] https://product-ci.infra.mirantis.net/job/9.x.main.ubuntu.bvt_2/615/
[3] https://paste.mirantis.net/show/2847/
Also reproduced on murano plugin jobs (9.0, 9.1). So, in looks this is problem not in fuel/mos. Maybe smth wrong with mirrors.