[system-tests] *_actions fields in EnviromentModel class spawns a new *Actions instance (and ssh remote) per each call

[Vladimir Khlyunev]

Each time we are using self.env.admin_actions (or any *_actions property) the EnviromentModel spawns a new *Actions instance and also spawns new ssh remote

@property
def admin_actions(self):
    return AdminActions(self.d_env.get_admin_remote())

This behaviour can cause unclosed ssh connections and right now *Actions classes doesn't supports context manager.

https://github.com/stackforge/fuel-qa/blob/96fb2bfb0586680b610184880556c182d07f52a6/fuelweb_test/models/environment.py#L70

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/250782

[Dennis Dmitriev (ddmitriev)]

There are several issues regarding this bug:

1. Spawn multiple instances of the same classes: https://github.com/openstack/fuel-qa/blob/master/fuelweb_test/models/environment.py#L84-L102
2. Spawn multiple instances of SSHClient for each request.
3. Limit of newly creating connections can be quickly reached because of connections that remain in CLOSE_WAIT status for a while.

The solution should reduce the amount of different instances of classes and connections that are used for the same remote node:

1. Make singletons for classes https://github.com/openstack/fuel-qa/blob/master/fuelweb_test/models/environment.py#L84-L102 , use IP of master node to initialize them instead of SSHClient instance;

2. Make a connection manager class that will:

    - keep a list of already created connections and their parameters (IP, user/pass, key)

    - for each call of 'execute', it should check if the connection is alive, and restore current working directory. Both can be solved by executing 'cd ~' on remote before executing the user command, and re-initialize the connection in case if the command 'cd ~' failed.

    - 'execute', 'upload', 'download' and other methods should get IP as the parameter instead of SSHClient instance, so that the connection manager could pick up an existing connection and close it when unused for some time.

3. To fit this connection manager to the test case timeline, it should:

    - be initialized from https://github.com/openstack/fuel-qa/blob/master/fuelweb_test/tests/base_test_case.py#L34 , to get a single instance of the connection manager during a test case, and correctly close all connections after the test case.

    - the reference to the connection manager instance should be provided to EnvironmentModel and FuelWebClient classes so that they could use the same connection manager. It could be pass during initialize the classes: https://github.com/openstack/fuel-qa/blob/master/fuelweb_test/tests/base_test_case.py#L35 and https://github.com/openstack/fuel-qa/blob/master/fuelweb_test/models/environment.py#L71

[OpenStack Infra (hudson-openstack)]

Change abandoned by Vasily Gorin (<email address hidden>) on branch: master
Review: https://review.openstack.org/250782

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/252937

[Artem Panchenko (apanchenko-8)]

Looks like this bug #1527847 could be fixed by https://review.openstack.org/252937 as well. If so, the please mark it as duplicate of this one.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/252937
Committed: https://git.openstack.org/cgit/openstack/fuel-qa/commit/?id=3cb2275954b0855cf7caff3a8899172401ad34ce
Submitter: Jenkins
Branch: master

commit 3cb2275954b0855cf7caff3a8899172401ad34ce
Author: vgorin <email address hidden>
Date: Thu Dec 3 15:53:14 2015 +0300

    Introduction of new method of SSH connection

    - Create Singletone to use it as metaclass in ssh_manager.
      Singleton provides us posibility of having only one instance of class.
    - Create SSHManagaer, wich will manage SSH connections by itself.
      Almost SSHClient methods was duplicated in SSHManager.
      Now it used so:
        SSHManager().method(node_ip, *parameters)
      We will not get back SSHClient, we will just say 'Do IT on NODE with this PARAM'
      e.g SSHManager().execute_on_remote('127.0.0.1', 'bash_command')
    - Use ssh_manager in cli_cluster_deletion test

    Change-Id: I307d7d71e814b67d20cc0b4648cf6a7dac4a7829
    Closes-Bug: #1503210

[Tatyanka (tatyana-leontovich)]

verify on the latest test runs