[upgrade] Single OSTF can't be used for different OpensTack releases

[Artem Panchenko]

Single OSTF engine can't be used for environments which have different OpensTack releases, but after master node upgrade new version of OSTF is available for all old environments. This can potencially cause failing of helth checks, because:

- some tests can be modified is order to support new release;
- new tests can be added, which aren't compatible with old release;
- old tests can be removed as deprecated in new release.

I think we can add tags for all OSTF tests with supported release ID or run few instances of ostf-server on master node, one for each release.

[Dmitry Pyzhov (dpyzhov)]

Updated ostf in stable branch should support previous deployments. That's why I've removed this bug from stable series.
For 5.1 we need to check behavior of ostf on environments from previous release.

[Dima Shulyak (dshulyak)]

Is there any tests right now that will be failed after upgrade due to incompatibility?
If there is no such tests i propose to convert this to blueprint and target to 6.0

[Andrey Sledzinskiy (asledzinskiy)]

No, there are no such tests at the moment

[Tatyanka (tatyana-leontovich)]

Actually we have ))) difference,
As example for 5.0 Murana is disabled - but it works fine on 5.0.1 and 5.1 . After upgrades is still disabled) I beleive we should add tags for support releass in each tests where incompabilities take palces

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/109270

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/109270
Committed: https://git.openstack.org/cgit/stackforge/fuel-ostf/commit/?id=1a427f2c4d8ae9fd431084eef59afca5316753f1
Submitter: Jenkins
Branch: master

commit 1a427f2c4d8ae9fd431084eef59afca5316753f1
Author: Dima Shulyak <email address hidden>
Date: Thu Jul 24 15:35:22 2014 +0300

    Add fuel_version to cluster_tags

    Adding fuel_version tag to cluster_tags
    we guarantee that tests will be rediscovered
    after cluster is updated

    Change-Id: Ifaa39b748322d244f387d12a56429d99e4827beb
    Closes-Bug: #1337823