[system test] Server handshake while try to run OSTF after deploying sometimes fails with error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

[Alexander Kurenyshev]

It seems to be a floating bug.
Server handshake while try to run OSTF after deploying sometimes fails with error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

For now it was on test deploy_ha_one_controller_neutron.
https://product-ci.infra.mirantis.net/job/7.0.system_test.ubuntu.thread_2/78/console

At the tracelog:

Traceback (most recent call last):
  File "/home/jenkins/venv-nailgun-tests-2.9/local/lib/python2.7/site-packages/proboscis/case.py", line 296, in testng_method_mistake_capture_func
    compatability.capture_type_error(s_func)
  File "/home/jenkins/venv-nailgun-tests-2.9/local/lib/python2.7/site-packages/proboscis/compatability/exceptions_2_6.py", line 27, in capture_type_error
    func()
  File "/home/jenkins/venv-nailgun-tests-2.9/local/lib/python2.7/site-packages/proboscis/case.py", line 350, in func
    func(test_case.state.get_state())
  File "/home/jenkins/workspace/7.0.system_test.ubuntu.thread_2/fuelweb_test/helpers/decorators.py", line 80, in wrapper
    result = func(*args, **kwargs)
  File "/home/jenkins/workspace/7.0.system_test.ubuntu.thread_2/fuelweb_test/tests/test_ha_one_controller.py", line 110, in deploy_ha_one_controller_neutron
    snapshot_name="deploy_ha_one_controller_neutron")
  File "/home/jenkins/workspace/7.0.system_test.ubuntu.thread_2/fuelweb_test/tests/test_ha_one_controller_base.py", line 52, in deploy_ha_one_controller_neutron_base
    data['user'], data['password'], data['tenant'])
  File "/home/jenkins/workspace/7.0.system_test.ubuntu.thread_2/fuelweb_test/helpers/os_actions.py", line 33, in __init__
    tenant)
  File "/home/jenkins/workspace/7.0.system_test.ubuntu.thread_2/fuelweb_test/helpers/common.py", line 66, in __init__
    ca_cert=path_to_cert)
  File "/home/jenkins/workspace/7.0.system_test.ubuntu.thread_2/fuelweb_test/helpers/common.py", line 177, in _get_keystoneclient
    cacert=ca_cert)
  File "/home/jenkins/venv-nailgun-tests-2.9/local/lib/python2.7/site-packages/keystoneclient/v2_0/client.py", line 152, in __init__
    self.authenticate()
  File "/home/jenkins/venv-nailgun-tests-2.9/local/lib/python2.7/site-packages/keystoneclient/utils.py", line 336, in inner
    return func(*args, **kwargs)
  File "/home/jenkins/venv-nailgun-tests-2.9/local/lib/python2.7/site-packages/keystoneclient/httpclient.py", line 507, in authenticate
    resp = self.get_raw_token_from_identity_service(**kwargs)
  File "/home/jenkins/venv-nailgun-tests-2.9/local/lib/python2.7/site-packages/keystoneclient/v2_0/client.py", line 196, in get_raw_token_from_identity_service
    _("Authorization Failed: %s") % e)
AuthorizationFailure: Authorization Failed: SSL exception connecting to https://10.109.6.3:5000/v2.0/tokens: [Errno 1] _ssl.c:510: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: master
Change author: Alexander Kurenyshev <email address hidden>
Review: https://review.fuel-infra.org/11167

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/11167
Submitter: Aleksandra Fedorova <email address hidden>
Branch: master

Commit: b781dda599aace58bab893df89165cdef28d9259
Author: Alexander Kurenyshev <email address hidden>
Date: Fri Sep 4 09:24:22 2015

Export two variables for SSL certs

Add export for two variables to generate and
store certificates for SSL support
at the jenkins job workspace

Change-Id: I361d1d3023df42679c066f0d8ddd26564afdd672
Closes-Bug: 1491001

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/220723

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/220723
Committed: https://git.openstack.org/cgit/stackforge/fuel-qa/commit/?id=00204688f01b8367100c452e58befbcd3fee2014
Submitter: Jenkins
Branch: master

commit 00204688f01b8367100c452e58befbcd3fee2014
Author: NastyaUrlapova <email address hidden>
Date: Sat Sep 5 11:03:27 2015 +0300

    Changed default directory for certificates

    Change-Id: Idf1585bbd5ef010874ca354b26055a08d102ecd5
    Related-Bug: #1491001

[Tatyanka (tatyana-leontovich)]

still reproduced on some cases - example:
 https://product-ci.infra.mirantis.net/job/7.0.system_test.ubuntu.one_controller_actions/4/testReport/(root)/deploy_ha_one_controller_neutron/deploy_ha_one_controller_neutron/

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/7.0
Review: https://review.openstack.org/221246

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/221246
Committed: https://git.openstack.org/cgit/stackforge/fuel-qa/commit/?id=b8f77a4533f3fc7f99aeb303c3191bda6e50f86e
Submitter: Jenkins
Branch: stable/7.0

commit b8f77a4533f3fc7f99aeb303c3191bda6e50f86e
Author: NastyaUrlapova <email address hidden>
Date: Sat Sep 5 11:03:27 2015 +0300

    Changed default directory for certificates

    Change-Id: Idf1585bbd5ef010874ca354b26055a08d102ecd5
    Related-Bug: #1491001
    (cherry picked from commit 00204688f01b8367100c452e58befbcd3fee2014)

[Alexander Kurenyshev (akurenyshev)]

Bug was reproduced because of a missed commit with fix when stable branch 7.0 was introduced.
Since fix was cherry-picked to the stable branch I moved this bug to Fix Committed.
Lets wait for results of SWARM with this fix.