[systest] Make timeout value for master node provisioning configurable

Bug #1528853 reported by Kyrylo Romanenko
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Released
High
Fuel QA Team

Bug Description

On slow lab master node provision often fails by timeout.
Current value of timeout is: timeout=10 * 60
1.5x or 2x timeout possibly could fix this issue.

13:11:58 ======================================================================
13:11:58 ERROR: Create environment and set up master node
13:11:58 ----------------------------------------------------------------------
13:11:58 Traceback (most recent call last):
13:11:58 File "/home/jenkins/venv-mos-master/local/lib/python2.7/site-packages/proboscis/case.py", line 296, in testng_method_mistake_capture_func
13:11:58 compatability.capture_type_error(s_func)
13:11:58 File "/home/jenkins/venv-mos-master/local/lib/python2.7/site-packages/proboscis/compatability/exceptions_2_6.py", line 27, in capture_type_error
13:11:58 func()
13:11:58 File "/home/jenkins/venv-mos-master/local/lib/python2.7/site-packages/proboscis/case.py", line 350, in func
13:11:58 func(test_case.state.get_state())
13:11:58 File "/home/jenkins/venv-mos-master/fuel-qa/fuelweb_test/helpers/decorators.py", line 81, in wrapper
13:11:58 result = func(*args, **kwargs)
13:11:58 File "/home/jenkins/venv-mos-master/fuel-qa/fuelweb_test/tests/base_test_case.py", line 146, in setup_master
13:11:58 self.env.setup_environment()
13:11:58 File "/home/jenkins/venv-mos-master/fuel-qa/fuelweb_test/models/environment.py", line 429, in setup_environment
13:11:58 self.wait_for_provisioning()
13:11:58 File "/home/jenkins/venv-mos-master/fuel-qa/fuelweb_test/__init__.py", line 57, in wrapped
13:11:58 result = func(*args, **kwargs)
13:11:58 File "/home/jenkins/venv-mos-master/fuel-qa/fuelweb_test/models/environment.py", line 499, in wait_for_provisioning
13:11:58 (self.d_env.admin_net), 22), timeout=timeout)
13:11:58 File "/home/jenkins/venv-mos-master/local/lib/python2.7/site-packages/devops/helpers/helpers.py", line 108, in _wait
13:11:58 return raising_predicate()
13:11:58 File "/home/jenkins/venv-mos-master/fuel-qa/fuelweb_test/models/environment.py", line 499, in <lambda>
13:11:58 (self.d_env.admin_net), 22), timeout=timeout)
13:11:58 File "/home/jenkins/venv-mos-master/local/lib/python2.7/site-packages/devops/helpers/helpers.py", line 56, in _tcp_ping
13:11:58 s.connect((str(host), int(port)))
13:11:58 File "/usr/lib/python2.7/socket.py", line 224, in meth
13:11:58 return getattr(self._sock,name)(*args)
13:11:58 error: [Errno 111] Connection refused

Igor (ipukha)
Changed in fuel:
status: New → Confirmed
Changed in fuel:
assignee: nobody → Fuel QA Team (fuel-qa)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-qa (master)

Fix proposed to branch: master
Review: https://review.openstack.org/261033

Changed in fuel:
assignee: Fuel QA Team (fuel-qa) → Vasyl Saienko (vsaienko)
status: Confirmed → In Progress
tags: added: area-qa system-tests
summary: - Increase timeout for master node provisioning
+ [systest] Make timeout value for master node provisioning configurable
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-qa (stable/8.0)

Fix proposed to branch: stable/8.0
Review: https://review.openstack.org/261287

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-qa (master)

Reviewed: https://review.openstack.org/261033
Committed: https://git.openstack.org/cgit/openstack/fuel-qa/commit/?id=3231add1b37f27b2837aa5ede07dd54987e5c0c2
Submitter: Jenkins
Branch: master

commit 3231add1b37f27b2837aa5ede07dd54987e5c0c2
Author: vsaienko <email address hidden>
Date: Wed Dec 23 17:32:26 2015 +0200

    Add WAIT_FOR_PROVISIONING_TIMEOUT timeout

    Allow to set WAIT_FOR_PROVISIONING_TIMEOUT

    Change-Id: Ic63949cbc2c500e0f532b9a7ff69b9f0f94503f5
    Closes-Bug: #1528853

Changed in fuel:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-qa (stable/8.0)

Reviewed: https://review.openstack.org/261287
Committed: https://git.openstack.org/cgit/openstack/fuel-qa/commit/?id=e2333e1992ed95541b80e3ebacb3ff0380c67295
Submitter: Jenkins
Branch: stable/8.0

commit e2333e1992ed95541b80e3ebacb3ff0380c67295
Author: vsaienko <email address hidden>
Date: Wed Dec 23 17:32:26 2015 +0200

    Add WAIT_FOR_PROVISIONING_TIMEOUT timeout

    Allow to set WAIT_FOR_PROVISIONING_TIMEOUT

    Change-Id: Ic63949cbc2c500e0f532b9a7ff69b9f0f94503f5
    Closes-Bug: #1528853

Revision history for this message
Andrey Sledzinskiy (asledzinskiy) wrote :

we have a lot of master node failures on swarm so we may need to increase default value of timeout

Changed in fuel:
assignee: Vasyl Saienko (vsaienko) → Fuel QA Team (fuel-qa)
status: Fix Committed → Confirmed
importance: Medium → High
Revision history for this message
Andrey Sledzinskiy (asledzinskiy) wrote :
Changed in fuel:
status: Confirmed → Fix Committed
Changed in fuel:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.