[sahara] Create big cluster failed because token expired

[Sergey Reshetnyak]

Big Sahara cluster creation is failed because of Keystone token expiration.

Steps to reproduce:
- Create big enough Sahara cluster (400 workers).
- Check that cluster creation time is bigger than Keystone token expiration time (1 hour by default)

Expected result:
- Sahara cluster is created successfully without any errors

Actual result:
- Sahara cluster creation is failed after the token expiration time due to the Keystone authentication error

This behaviour was caught on the following environment configuration (although this can be reproduced on any OpenStack environment with Sahara (no matter Heat or direct engine is used) with enough time-consuming cluster creation task):

api: '1.0'
astute_sha: 34e0493afa22999c4a07d3198ceb945116ab7932
auth_required: true
build_id: 2015-07-27_09-24-22
build_number: '98'
feature_groups:
- mirantis
fuel-agent_sha: 2a65f11c10b0aeb5184247635a19740fc3edde21
fuel-library_sha: 39c3162ee2e2ff6e3af82f703998f95ff4cc2b7a
fuel-ostf_sha: 94a483c8aba639be3b96616c1396ef290dcc00cd
fuelmain_sha: 921918a3bd3d278431f35ad917989e46b0c24100
nailgun_sha: d5c19f6afc66b5efe3c61ecb49025c1002ccbdc6
openstack_version: 2015.1.0-7.0
production: docker
python-fuelclient_sha: 58c411d87a7eaf0fd6892eae2b5cb1eff4190c98
release: '7.0'

Solution: introduce token creation from the trusts for all clusters resources operations.

[Fuel Devops McRobotson (fuel-devops-robot)]

Related fix proposed to branch: openstack-ci/fuel-7.0/2015.1.0
Change author: Sergey Reshetnyak <email address hidden>
Review: https://review.fuel-infra.org/10393

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: openstack-ci/fuel-7.0/2015.1.0
Change author: Sergey Reshetnyak <email address hidden>
Review: https://review.fuel-infra.org/10416

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/10393
Submitter: mos-infra-ci <>
Branch: openstack-ci/fuel-7.0/2015.1.0

Commit: ec585ae41c13a8d7154fe0c685354c764805cedd
Author: Sergey Reshetnyak <email address hidden>
Date: Wed Aug 19 10:16:42 2015

Improve auth token usage in Sahara

Changes:
* create trust for all clusters
* create token from trust for all resource operations
* use cache for auth tokens

Change-Id: Ibf2749a8d850ea3e40b910654856cc2a4db9a4df
Related-bug: #1484442

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/10416
Submitter: mos-infra-ci <>
Branch: openstack-ci/fuel-7.0/2015.1.0

Commit: 1b88c319a33bc9d03a5b521115ff2c9e94a87615
Author: Sergey Reshetnyak <email address hidden>
Date: Thu Aug 20 09:34:20 2015

Improve auth token usage - part 2

Changes:
* add periodic job for cleanup tokens
* rework calling OS clients

Change-Id: I36e4a56d287dd52496f4e95a4c39c333c22502a4
Closes-bug: #1484442

[Dina Belova (dbelova)]

Checked on 50 nodes. 230 workers cluster was able to be started on general HW.

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: openstack-ci/fuel-8.0/liberty
Change author: Sergey Reshetnyak <email address hidden>
Review: https://review.fuel-infra.org/13395

[Fuel Devops McRobotson (fuel-devops-robot)]

Related fix proposed to branch: openstack-ci/fuel-8.0/liberty
Change author: Sergey Reshetnyak <email address hidden>
Review: https://review.fuel-infra.org/13412

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: openstack-ci/fuel-8.0/liberty
Change author: Vitaly Gridnev <email address hidden>
Review: https://review.fuel-infra.org/14654

[Fuel Devops McRobotson (fuel-devops-robot)]

Change abandoned by Sergey Reshetnyak <email address hidden> on branch: openstack-ci/fuel-8.0/liberty
Review: https://review.fuel-infra.org/13395

[Fuel Devops McRobotson (fuel-devops-robot)]

Change abandoned by Sergey Reshetnyak <email address hidden> on branch: openstack-ci/fuel-8.0/liberty
Review: https://review.fuel-infra.org/13412

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/14654
Submitter: Pkgs Jenkins <email address hidden>
Branch: openstack-ci/fuel-8.0/liberty

Commit: 7c51d77292057f5c5fdf646e96f10bbb6cc5a33e
Author: Vitaly Gridnev <email address hidden>
Date: Mon Dec 14 13:46:17 2015

Fix problems with old tokens

This change is the combination of the following changes:

 * https://review.openstack.org/#/c/251813/
 * https://review.openstack.org/#/c/251835/
 * https://review.openstack.org/#/c/251488/

Change-Id: I8a6db6e99bc716b2e754ae4160d8cdf017b523d9
Closes-bug: 1484442

[Ivan Lozgachev (ilozgachev)]

Hard to reproduce, lack of environment. Moving to Fix Released.

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: 9.0/mitaka
Change author: Vitaly Gridnev <email address hidden>
Review: https://review.fuel-infra.org/18622

[Fuel Devops McRobotson (fuel-devops-robot)]

Change abandoned by Vitaly Gridnev <email address hidden> on branch: 9.0/mitaka
Review: https://review.fuel-infra.org/18622
Reason: Changes was merged to upstream