OSTF capabilities not sufficient for "scale up" use case

[Dmitriy Novakovskiy]

User story:

As a cloud operator, I want to run tests on new compute nodes that I added to cluster with Fuel in order to verify their sanity and functionality.

The fundamental flaw that current "Health Check" implementation has is the lack of means to specify the scale of tests and the subset of nodes where they should be ran. When user adds a few new nodes to a cloud it's not possible to run VM creation/connectivity/volume attachment tests on these specific nodes.

Proposed solution:

Step 1) For OSTF tests in "Functional tests" group (most if not all of them involve doing stuff on Compute nodes) allow user to specify:
- on which nodes the test will be executed (checklist of Compute nodes that the env has)
- the "scale" of the test - create N VMs, run pings for N minutes, read/write N Gigs of data to a volume, etc

Step 2) When new nodes are added to a running cloud - display a message for user suggesting that he/she should run "these" tests for "these newly added" nodes. This will help detect possible failures earlier

[Tatyanka (tatyana-leontovich)]

Dmitry, thank you a lot for ideas. It is really good points, that can be extended in scale scope. So I beleive we should move those scenario in blueprint and extend test cases with the additional checks, include ha non-destructive tests (like verify that added node are include in rabbit cluster and messages are repricated on it too) and may be some ops actiona like migration from node - to node(in case if it was computes) and resize and other stuff

[Dima Shulyak (dshulyak)]

Those tests almost impossible to implement in a decent way using ostf, due to architecture limitation (both nailgun and ostf), if this will be considered as very important thing todo - then someone will need to rethink the way ostf runs tests