Fuel for OpenStack

ubuntu staging 6.1 bvt job failed on step: Deploy ceph HA with RadosGW for objects

Bug #1457586 reported by Vadim Rovachev
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Invalid
High
MOS Maintenance
Fuel for OpenStack 6.0-updates
6.1.x
Invalid
High
Oleksiy Molchanov
Fuel for OpenStack 6.1-updates

Bug Description

http://jenkins-product.srt.mirantis.net:8080/view/6.1/job/6.1.staging.ubuntu.bvt_2/352/console

======================================================================
ERROR: Deploy ceph HA with RadosGW for objects
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jenkins/venv-nailgun-tests-2.9/local/lib/python2.7/site-packages/proboscis/case.py", line 296, in testng_method_mistake_capture_func
    compatability.capture_type_error(s_func)
  File "/home/jenkins/venv-nailgun-tests-2.9/local/lib/python2.7/site-packages/proboscis/compatability/exceptions_2_6.py", line 27, in capture_type_error
    func()
  File "/home/jenkins/venv-nailgun-tests-2.9/local/lib/python2.7/site-packages/proboscis/case.py", line 350, in func
    func(test_case.state.get_state())
  File "/home/jenkins/workspace/6.1.staging.ubuntu.bvt_2/fuelweb_test/helpers/decorators.py", line 70, in wrapper
    result = func(*args, **kwargs)
  File "/home/jenkins/workspace/6.1.staging.ubuntu.bvt_2/fuelweb_test/tests/test_ceph.py", line 417, in ceph_rados_gw
    self.fuel_web.deploy_cluster_wait(cluster_id)
  File "/home/jenkins/workspace/6.1.staging.ubuntu.bvt_2/fuelweb_test/helpers/decorators.py", line 429, in wrapper
    result = func(*args, **kwargs)
  File "/home/jenkins/workspace/6.1.staging.ubuntu.bvt_2/fuelweb_test/helpers/decorators.py", line 414, in wrapper
    result = func(*args, **kwargs)
  File "/home/jenkins/workspace/6.1.staging.ubuntu.bvt_2/fuelweb_test/helpers/decorators.py", line 463, in wrapper
    return func(*args, **kwargs)
  File "/home/jenkins/workspace/6.1.staging.ubuntu.bvt_2/fuelweb_test/helpers/decorators.py", line 470, in wrapper
    result = func(*args, **kwargs)
  File "/home/jenkins/workspace/6.1.staging.ubuntu.bvt_2/fuelweb_test/helpers/decorators.py", line 356, in wrapper
    return func(*args, **kwargs)
  File "/home/jenkins/workspace/6.1.staging.ubuntu.bvt_2/fuelweb_test/models/fuel_web_client.py", line 727, in deploy_cluster_wait
    self.assert_task_success(task, interval=interval)
  File "/home/jenkins/workspace/6.1.staging.ubuntu.bvt_2/fuelweb_test/__init__.py", line 48, in wrapped
    result = func(*args, **kwargs)
  File "/home/jenkins/workspace/6.1.staging.ubuntu.bvt_2/fuelweb_test/models/fuel_web_client.py", line 263, in assert_task_success
    task = self.task_wait(task, timeout, interval)
  File "/home/jenkins/workspace/6.1.staging.ubuntu.bvt_2/fuelweb_test/__init__.py", line 48, in wrapped
    result = func(*args, **kwargs)
  File "/home/jenkins/workspace/6.1.staging.ubuntu.bvt_2/fuelweb_test/models/fuel_web_client.py", line 1008, in task_wait
    "was exceeded: ".format(task=task["name"], timeout=timeout))
TimeoutError: Waiting task "deploy" timeout 7800 sec was exceeded:

Tags: non-release
Revision history for this message
Vadim Rovachev (vrovachev) wrote :
Changed in fuel:
milestone: none → 6.1
Bartłomiej Piotrowski (bpiotrowski)
Changed in fuel:
assignee: nobody → Bartlomiej Piotrowski (bpiotrowski)
Revision history for this message
Bartłomiej Piotrowski (bpiotrowski) wrote :

It looks like HAProxy fails to start:

Fri Apr 10 09:54:03 +0000 2015 /Stage[main]/Openstack::Ha::Horizon/Openstack::Ha::Haproxy_service[horizon]/Exec[haproxy restart for horizon]/returns (notice): 2015/04/10_09:53:52 INFO: haproxy daemon running
Fri Apr 10 09:54:03 +0000 2015 /Stage[main]/Openstack::Ha::Horizon/Openstack::Ha::Haproxy_service[horizon]/Exec[haproxy restart for horizon]/returns (notice): 2015/04/10_09:53:52 INFO: Stopped haproxy daemon.
Fri Apr 10 09:54:03 +0000 2015 /Stage[main]/Openstack::Ha::Horizon/Openstack::Ha::Haproxy_service[horizon]/Exec[haproxy restart for horizon]/returns (notice): 2015/04/10_09:53:52 INFO: haproxy daemon is not running
Fri Apr 10 09:54:03 +0000 2015 /Stage[main]/Openstack::Ha::Horizon/Openstack::Ha::Haproxy_service[horizon]/Exec[haproxy restart for horizon]/returns (notice): 2015/04/10_09:53:52 INFO: [WARNING] 099/095352 (5062) : config : 'stats' statement ignored for proxy 'mysqld' as it requires HTTP mode.
Fri Apr 10 09:54:03 +0000 2015 /Stage[main]/Openstack::Ha::Horizon/Openstack::Ha::Haproxy_service[horizon]/Exec[haproxy restart for horizon]/returns (notice): 2015/04/10_09:53:52 ERROR: RTNETLINK answers: File exists
Fri Apr 10 09:54:03 +0000 2015 /Stage[main]/Openstack::Ha::Horizon/Openstack::Ha::Haproxy_service[horizon]/Exec[haproxy restart for horizon]/returns (notice): 2015/04/10_09:53:52 ERROR: Cannot find device "hapr-host"
Fri Apr 10 09:54:03 +0000 2015 /Stage[main]/Openstack::Ha::Horizon/Openstack::Ha::Haproxy_service[horizon]/Exec[haproxy restart for horizon]/returns (notice): Device "hapr-host" does not exist.
Fri Apr 10 09:54:03 +0000 2015 /Stage[main]/Openstack::Ha::Horizon/Openstack::Ha::Haproxy_service[horizon]/Exec[haproxy restart for horizon]/returns (notice): 2015/04/10_09:53:53 ERROR: Cannot find device "hapr-host"
Fri Apr 10 09:54:03 +0000 2015 /Stage[main]/Openstack::Ha::Horizon/Openstack::Ha::Haproxy_service[horizon]/Exec[haproxy restart for horizon]/returns (notice): 2015/04/10_09:54:03 INFO: Started haproxy daemon.
Fri Apr 10 09:54:03 +0000 2015 /Stage[main]/Openstack::Ha::Horizon/Openstack::Ha::Haproxy_service[horizon]/Exec[haproxy restart for horizon]/returns (notice): executed successfully
Fri Apr 10 09:54:03 +0000 2015 /Stage[main]/Openstack::Ha::Horizon/Openstack::Ha::Haproxy_service[horizon]/Exec[haproxy restart for horizon] (debug): The container Openstack::Ha::Haproxy_service[horizon] will propagate my refresh event
Fri Apr 10 09:54:03 +0000 2015 /Stage[main]/Openstack::Ha::Horizon/Openstack::Ha::Haproxy_service[horizon]/Exec[haproxy restart for horizon] (info): Evaluated in 10.31 seconds

Revision history for this message
Bartłomiej Piotrowski (bpiotrowski) wrote :

If so, it's a duplicate of https://bugs.launchpad.net/fuel/+bug/1430317, I'll check a few things yet.

Bartłomiej Piotrowski (bpiotrowski)
Changed in fuel:
assignee: Bartlomiej Piotrowski (bpiotrowski) → Fuel Library Team (fuel-library)
importance: Undecided → High
status: New → Confirmed
Oleksiy Molchanov (omolchanov)
Changed in fuel:
assignee: Fuel Library Team (fuel-library) → Oleksiy Molchanov (omolchanov)
Revision history for this message
Tomasz 'Zen' Napierala (tzn) wrote :

Does it need further investigation or are we shure it's a duplicate?

Revision history for this message
Oleksiy Molchanov (omolchanov) wrote :

It is not a duplicate, cause bug #1430317 is not harmful.

As we can see in logs, the deployment was successful and it took ~1 hour. Somehow the test didn't receive the response that deployment is over during 2 hours. I suppose it was some kind of test env problem, because next test were successful.

I will mark it as incomplete.

Changed in fuel:
status: Confirmed → Incomplete
Revision history for this message
Nastya Urlapova (aurlapova) wrote :

Moved to Invalid, staging looks well, environment with this issue was erased.

Changed in fuel:
status: Incomplete → Invalid
Revision history for this message
Ivan Kolodyazhny (e0ne) wrote :

6.0.1 Staing failed: https://old-stable-ci.infra.mirantis.net/job/6.0.1.staging.ubuntu.bvt_2/266/console

Vladimir Sharshov (vsharshov)
no longer affects: fuel/6.0.x
Changed in fuel:
status: Invalid → Confirmed
milestone: 6.1 → 6.0-mu-6
milestone: 6.0-mu-6 → 6.0-updates
tags: added: non-release
Revision history for this message
Denis Meltsaykin (dmeltsaykin) wrote :

Closing this as Invalid, as 6.0.1 is not relevant anymore.

Changed in fuel:
status: Confirmed → Invalid
assignee: Oleksiy Molchanov (omolchanov) → MOS Maintenance (mos-maintenance)
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.