new tests needed for cold-restart recovery

[Andrew Woodward]

Some new tests are needed for cold-restart recovery validation. Currently they should be performed on demand until the cold-start recovery process is passable without manual intervention.

Case 1: cold start of cluster_simple
Parameters: 1 controller, 1 compute, 2 ceph-osd, ceph for image, ceph for cinder.
Steps:
1) Deploy cluster
2) Validate cluster
3) Stop all nodes
4) Start all nodes
5) Sleep X
6) Validate cluster

Case 2: cold start of cluster_ha
Parameters: 3 controller + ceph-osd, 1 compute
Same steps as case 1

[Nikolay Fedotov (nfedotov)]

Here is review for case #1: https://review.openstack.org/#/c/59665/
Case #2 covered in CephHA tests

[Andrew Woodward (xarses)]

Both tests are missing a "cold restart" the all nodes should be force power-off and then started and allowed to boot. Once online again the tests should pass. This should probably be separate from the ceph tests as they are related more directly to testing the clusters ability to start from a crashed off state, ceph is included in the roles as it's self heal should be tested as well.

[Nikolay Fedotov (nfedotov)]

Thanks for feedback. I missed "force power-off and then started and allowed to boot" .
The review has been updated. Could you review it again. There are three tests, two for simple environments (without and with ceph nodes) and another one for HA environment with Cinder and Ceph nodes.