| 2016-03-01 16:35:22 |
Dmitry Teselkin |
description |
During the executing 8.0-pkg-systest-ubuntu job for https://review.fuel-infra.org/#/c/14261/ there was deployment error occured
======================================================================
ERROR: Deploy ceph HA with RadosGW for objects
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/jenkins/venv-nailgun-tests-2.9/local/lib/python2.7/site-packages/proboscis/case.py", line 296, in testng_method_mistake_capture_func
compatability.capture_type_error(s_func)
File "/home/jenkins/venv-nailgun-tests-2.9/local/lib/python2.7/site-packages/proboscis/compatability/exceptions_2_6.py", line 27, in capture_type_error
func()
File "/home/jenkins/venv-nailgun-tests-2.9/local/lib/python2.7/site-packages/proboscis/case.py", line 350, in func
func(test_case.state.get_state())
File "/home/jenkins/workspace/8.0-pkg-systest-ubuntu/fuel-qa/fuelweb_test/helpers/decorators.py", line 80, in wrapper
result = func(*args, **kwargs)
File "/home/jenkins/workspace/8.0-pkg-systest-ubuntu/fuel-qa/fuelweb_test/tests/test_ceph.py", line 504, in ceph_rados_gw
self.fuel_web.deploy_cluster_wait(cluster_id)
File "/home/jenkins/workspace/8.0-pkg-systest-ubuntu/fuel-qa/fuelweb_test/helpers/decorators.py", line 428, in wrapper
result = func(*args, **kwargs)
File "/home/jenkins/workspace/8.0-pkg-systest-ubuntu/fuel-qa/fuelweb_test/helpers/decorators.py", line 413, in wrapper
result = func(*args, **kwargs)
File "/home/jenkins/workspace/8.0-pkg-systest-ubuntu/fuel-qa/fuelweb_test/helpers/decorators.py", line 464, in wrapper
return func(*args, **kwargs)
File "/home/jenkins/workspace/8.0-pkg-systest-ubuntu/fuel-qa/fuelweb_test/helpers/decorators.py", line 474, in wrapper
result = func(*args, **kwargs)
File "/home/jenkins/workspace/8.0-pkg-systest-ubuntu/fuel-qa/fuelweb_test/helpers/decorators.py", line 355, in wrapper
return func(*args, **kwargs)
File "/home/jenkins/workspace/8.0-pkg-systest-ubuntu/fuel-qa/fuelweb_test/models/fuel_web_client.py", line 695, in deploy_cluster_wait
self.assert_task_success(task, interval=interval, timeout=timeout)
File "/home/jenkins/workspace/8.0-pkg-systest-ubuntu/fuel-qa/fuelweb_test/__init__.py", line 57, in wrapped
result = func(*args, **kwargs)
File "/home/jenkins/workspace/8.0-pkg-systest-ubuntu/fuel-qa/fuelweb_test/models/fuel_web_client.py", line 312, in assert_task_success
task = self.task_wait(task, timeout, interval)
File "/home/jenkins/workspace/8.0-pkg-systest-ubuntu/fuel-qa/fuelweb_test/__init__.py", line 57, in wrapped
result = func(*args, **kwargs)
File "/home/jenkins/workspace/8.0-pkg-systest-ubuntu/fuel-qa/fuelweb_test/models/fuel_web_client.py", line 1050, in task_wait
timeout=timeout
File "/home/jenkins/venv-nailgun-tests-2.9/local/lib/python2.7/site-packages/devops/helpers/helpers.py", line 92, in wait
while not predicate():
File "/home/jenkins/workspace/8.0-pkg-systest-ubuntu/fuel-qa/fuelweb_test/models/fuel_web_client.py", line 1047, in <lambda>
lambda: (self.client.get_task(task['id'])['status']
File "/home/jenkins/workspace/8.0-pkg-systest-ubuntu/fuel-qa/fuelweb_test/__init__.py", line 57, in wrapped
result = func(*args, **kwargs)
File "/home/jenkins/workspace/8.0-pkg-systest-ubuntu/fuel-qa/fuelweb_test/helpers/decorators.py", line 130, in wrapped
response = func(*args, **kwargs)
File "/home/jenkins/workspace/8.0-pkg-systest-ubuntu/fuel-qa/fuelweb_test/models/nailgun_client.py", line 160, in get_task
return self.client.get("/api/tasks/{}".format(task_id))
File "/home/jenkins/workspace/8.0-pkg-systest-ubuntu/fuel-qa/fuelweb_test/helpers/http.py", line 68, in get
return self._open(req)
File "/home/jenkins/workspace/8.0-pkg-systest-ubuntu/fuel-qa/fuelweb_test/helpers/http.py", line 92, in _open
return self._get_response(req)
File "/home/jenkins/workspace/8.0-pkg-systest-ubuntu/fuel-qa/fuelweb_test/helpers/http.py", line 116, in _get_response
return self.opener.open(req)
File "/usr/lib/python2.7/urllib2.py", line 404, in open
response = self._open(req, data)
File "/usr/lib/python2.7/urllib2.py", line 422, in _open
'_open', req)
File "/usr/lib/python2.7/urllib2.py", line 382, in _call_chain
result = func(*args)
File "/usr/lib/python2.7/urllib2.py", line 1214, in http_open
return self.do_open(httplib.HTTPConnection, req)
File "/usr/lib/python2.7/urllib2.py", line 1184, in do_open
raise URLError(err)
URLError: <urlopen error [Errno 113] No route to host>
in job https://packaging-ci.infra.mirantis.net/job/8.0-pkg-systest-ubuntu/1353/console |
During the executing 8.0-pkg-systest-ubuntu job for https://review.fuel-infra.org/#/c/14261/ there was deployment error occured
======================================================================
ERROR: Deploy ceph HA with RadosGW for objects
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/jenkins/venv-nailgun-tests-2.9/local/lib/python2.7/site-packages/proboscis/case.py", line 296, in testng_method_mistake_capture_func
compatability.capture_type_error(s_func)
File "/home/jenkins/venv-nailgun-tests-2.9/local/lib/python2.7/site-packages/proboscis/compatability/exceptions_2_6.py", line 27, in capture_type_error
func()
File "/home/jenkins/venv-nailgun-tests-2.9/local/lib/python2.7/site-packages/proboscis/case.py", line 350, in func
func(test_case.state.get_state())
...
File "/home/jenkins/workspace/8.0-pkg-systest-ubuntu/fuel-qa/fuelweb_test/helpers/http.py", line 116, in _get_response
return self.opener.open(req)
File "/usr/lib/python2.7/urllib2.py", line 404, in open
response = self._open(req, data)
File "/usr/lib/python2.7/urllib2.py", line 422, in _open
'_open', req)
File "/usr/lib/python2.7/urllib2.py", line 382, in _call_chain
result = func(*args)
File "/usr/lib/python2.7/urllib2.py", line 1214, in http_open
return self.do_open(httplib.HTTPConnection, req)
File "/usr/lib/python2.7/urllib2.py", line 1184, in do_open
raise URLError(err)
URLError: <urlopen error [Errno 113] No route to host>
in job https://packaging-ci.infra.mirantis.net/job/8.0-pkg-systest-ubuntu/1353/console
UPDATE (2016-03-01)
===================
The root cause here is broken driver for e1000 interface. This issue is not CentOS related, it happened (and still happening) on almost every platform. There are only workarounds for this issue and no *real* solution, like 'fix the driver'.
There is an interesting commit that looks like a solution, it was merged Sep 8, 2015 - https://github.com/torvalds/linux/commit/9eab46b7cb8d0b0dcf014bf7b25e0e72b9e4d929
Unfortunately, it doesn't help.
Workarounds
-----------
* Disable TSO offloading
* Use 'virtio' driver
Related Bugs
------------
kernel - https://bugzilla.kernel.org/show_bug.cgi?id=47331
novell - http://www.novell.com/support/kb/doc.php?id=7000925
redhat - https://access.redhat.com/solutions/25528
redhat - https://bugzilla.redhat.com/show_bug.cgi?id=398921
centos - https://bugs.centos.org/view.php?id=6168 |
|
