Nodes does not manage to get online state in 4*60 seconds

[Maksym Strukov]

Steps:
1. Revert snapshot upgrade_master_nova_cinder
2. Add controller
3. Initialize stop deployment action

Actual:
https://product-ci.infra.mirantis.net/job/7.0.system_test.ubuntu.upgrade_nova_cinder/20/testReport/(root)/add_node_after_upgrade_nova_cinder/add_node_after_upgrade_nova_cinder/

Traceback (most recent call last):
  File "/usr/lib/python2.7/unittest/case.py", line 331, in run
    testMethod()
  File "/usr/lib/python2.7/unittest/case.py", line 1043, in runTest
    self._testFunc()
  File "/home/jenkins/venv-nailgun-tests-2.9/local/lib/python2.7/site-packages/proboscis/case.py", line 296, in testng_method_mistake_capture_func
    compatability.capture_type_error(s_func)
  File "/home/jenkins/venv-nailgun-tests-2.9/local/lib/python2.7/site-packages/proboscis/compatability/exceptions_2_6.py", line 27, in capture_type_error
    func()
  File "/home/jenkins/venv-nailgun-tests-2.9/local/lib/python2.7/site-packages/proboscis/case.py", line 350, in func
    func(test_case.state.get_state())
  File "/home/jenkins/workspace/7.0.system_test.ubuntu.upgrade_nova_cinder/UPGRADE/fuelweb_test/helpers/decorators.py", line 80, in wrapper
    result = func(*args, **kwargs)
  File "/home/jenkins/workspace/7.0.system_test.ubuntu.upgrade_nova_cinder/UPGRADE/fuelweb_test/tests/tests_upgrade/test_upgrade.py", line 1137, in add_node_after_upgrade_nova_cinder
    self.env.d_env.nodes().slaves[:4])
  File "/home/jenkins/workspace/7.0.system_test.ubuntu.upgrade_nova_cinder/UPGRADE/fuelweb_test/__init__.py", line 58, in wrapped
    result = func(*args, **kwargs)
  File "/home/jenkins/workspace/7.0.system_test.ubuntu.upgrade_nova_cinder/UPGRADE/fuelweb_test/models/fuel_web_client.py", line 1756, in wait_nodes_get_online_state
    'Node {0} has not become online'.format(node.name))
  File "/home/jenkins/venv-nailgun-tests-2.9/local/lib/python2.7/site-packages/proboscis/asserts.py", line 163, in assert_true
    raise ASSERTION_ERROR(message)
AssertionError: Node slave-01 has not become online

After reverting env all nodes online:
[root@nailgun ~]# fuel --user=admin --password=newpass nodes
DEPRECATION WARNING: /etc/fuel/client/config.yaml exists and will be used as the source for settings. This behavior is deprecated. Please specify the path to your custom settings file in the FUELC
LIENT_CUSTOM_SETTINGS environment variable.
id | status | name | cluster | ip | mac | roles | pending_roles | online | group_id
---|----------|-------------------------|---------|-------------|-------------------|-------|-----------------|--------|---------
2 | discover | slave-02_compute_cinder | 1 | 10.109.10.4 | 64:d0:78:e6:c9:0d | | cinder, compute | True | 1
3 | discover | slave-03_compute_cinder | 1 | 10.109.10.5 | 64:f0:0e:0d:62:4f | | cinder, compute | True | 1
4 | discover | slave-04_controller | 1 | 10.109.10.6 | 64:d0:c0:1a:f5:33 | | controller | True | 1
1 | discover | slave-01_controller | 1 | 10.109.10.3 | 64:1c:d1:45:54:3d | | controller | True | 1

Assume 4*60 seconds just not enough. Let's increase timeout?

Env:
{"build_id": "290", "build_number": "290", "release_versions": {"2015.1.0-7.0": {"VERSION": {"build_id": "290", "build_number": "290", "api": "1.0", "fuel-library_sha": "3a8ff78bd1d6338289107370ab13d5e80491dbab", "nailgun_sha": "16a39d40120dd4257698795f12de4ae8200b1778", "feature_groups": ["mirantis"], "fuel-nailgun-agent_sha": "d7027952870a35db8dc52f185bb1158cdd3d1ebd", "openstack_version": "2015.1.0-7.0", "fuel-agent_sha": "082a47bf014002e515001be05f99040437281a2d", "production": "docker", "python-fuelclient_sha": "2864459e27b0510a0f7aedac6cdf27901ef5c481", "astute_sha": "a717657232721a7fafc67ff5e1c696c9dbeb0b95", "fuel-ostf_sha": "1f08e6e71021179b9881a824d9c999957fcc7045", "release": "7.0", "fuelmain_sha": "6b83d6a6a75bf7bca3177fcf63b2eebbf1ad0a85"}}, "2014.2.2-6.1": {"VERSION": {"build_id": "2015-06-19_13-02-31", "build_number": "525", "api": "1.0", "fuel-library_sha": "2e7a08ad9792c700ebf08ce87f4867df36aa9fab", "nailgun_sha": "dbd54158812033dd8cfd7e60c3f6650f18013a37", "feature_groups": ["mirantis"], "openstack_version": "2014.2.2-6.1", "production": "docker", "python-fuelclient_sha": "4fc55db0265bbf39c369df398b9dc7d6469ba13b", "astute_sha": "1ea8017fe8889413706d543a5b9f557f5414beae", "fuel-ostf_sha": "8fefcf7c4649370f00847cc309c24f0b62de718d", "release": "6.1", "fuelmain_sha": "a3998372183468f56019c8ce21aa8bb81fee0c2f"}}}, "auth_required": true, "api": "1.0", "fuel-library_sha": "3a8ff78bd1d6338289107370ab13d5e80491dbab", "nailgun_sha": "16a39d40120dd4257698795f12de4ae8200b1778", "feature_groups": ["mirantis"], "fuel-nailgun-agent_sha": "d7027952870a35db8dc52f185bb1158cdd3d1ebd", "openstack_version": "2015.1.0-7.0", "fuel-agent_sha": "082a47bf014002e515001be05f99040437281a2d", "production": "docker", "python-fuelclient_sha": "2864459e27b0510a0f7aedac6cdf27901ef5c481", "astute_sha": "a717657232721a7fafc67ff5e1c696c9dbeb0b95", "fuel-ostf_sha": "1f08e6e71021179b9881a824d9c999957fcc7045", "release": "7.0", "fuelmain_sha": "6b83d6a6a75bf7bca3177fcf63b2eebbf1ad0a85"}

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/222751

[OpenStack Infra (hudson-openstack)]

Change abandoned by Maksym Strukov (<email address hidden>) on branch: master
Review: https://review.openstack.org/222751
Reason: dup of https://github.com/stackforge/fuel-qa/commit/acc00885c952f992f0434b51dcfade42597c37e7