Intermittent ssh connection timeout failure.

[Maria Guadalupe Perez Ibara]

Brief Description
-----------------
Running sanity some commands failed to execute due to a timeout error.

Severity
--------
Major

Steps to Reproduce
-------------------
$openstack volume create --size SIZE_IN_GB name
$openstack network create name
$ openstack server create --image <image> --flavor <flavor> --nic net-id=<net-id> <net-name>

Expected Behavior
------------------
the ssh commands should succeed

Actual Behavior
----------------
the commands fails whit the following error:
SSHClientException: Timed out in 30 seconds

notice that this is an intermittent problem, previous or latter commands may work

Reproducibility
---------------
Intermittent

System Configuration
--------------------
Duplex & 2+2 virtual

Branch/Pull Time/Commit
-----------------------
OS="centos"
SW_VERSION="19.01"
BUILD_TARGET="Host Installer"
BUILD_TYPE="Formal"
BUILD_ID="20190604T144018Z"

JOB="STX_build_master_master"
<email address hidden>"
BUILD_NUMBER="128"
BUILD_HOST="starlingx_mirror"
BUILD_DATE="2019-06-04 14:40:18 +0000"

Last Pass
---------
31 May 2019

Timestamp/Logs
--------------
we are working on getting logs

Test Activity
-------------
[Sanity]

[Erich Cordoba (ericho)]

As the behavior is that in a continuous stream of SSH connections, some of those fails with SSH, then script might help to reproduce the issue:

```
#!/usr/bin/env python

import sys, paramiko

if len(sys.argv) < 4:
    print "args missing"
    sys.exit(1)

hostname = sys.argv[1]
password = sys.argv[2]
command = sys.argv[3]

username = "admin"
port = 22

try:
    client = paramiko.SSHClient()
    client.load_system_host_keys()
    client.set_missing_host_key_policy(paramiko.WarningPolicy)

    client.connect(hostname, port=port, username=username, password=password)

    stdin, stdout, stderr = client.exec_command(command)
    print stdout.read(),

finally:
client.close()
```

Run it with:

python ssh_test.py 10.10.10.2 St8rlingX* "source /etc/platform/openrc && system host-list"

[Ghada Khalil (gkhalil)]

This will likely need to be investigated live on the system reporting the issue

[Ghada Khalil (gkhalil)]

Perhaps Erich can help narrow this down

[Hayde Martinez (haydemtz)]

An attempt was made to reproduce this bug, on virtual environment on a duplex configuration, we used the script above along with the test suite running at the same time. All tests from the test suite completed succesfully and the output of the commands in the script were all as expected.

Therefore, marking this bug as invalid.

Create Flavors for Instances :: Create flavors with or without pro... | PASS |
------------------------------------------------------------------------------
Create Images for Instances :: Create images with or without prope... | PASS |
------------------------------------------------------------------------------
Create Networks for Instances :: Create networks to be used to lau... | PASS |
------------------------------------------------------------------------------
Launch Instances :: Launch Cirros and Centos instances. | PASS |
------------------------------------------------------------------------------
Suspend Resume Instances :: Suspend and Resume Cirros and Centos i... | PASS |
------------------------------------------------------------------------------
Set Error Active Flags Instances :: Set 'Error' and 'Active' flags... | PASS |
------------------------------------------------------------------------------
Pause Unpause Instances :: Pause and Unpause Cirros and Centos ins... | PASS |
------------------------------------------------------------------------------
Stop Start Instances :: Stop and Start Cirros and Centos instances. | PASS |
------------------------------------------------------------------------------
Lock Unlock Instances :: Lock and Unlock Cirros and Centos instances. | PASS |
------------------------------------------------------------------------------
Reboot Instances :: Reboot Cirros and Centos instances. | PASS |
------------------------------------------------------------------------------
Rebuild Instances :: Rebuild Cirros and Centos instances. | PASS |
------------------------------------------------------------------------------
Resize Instances :: Resize Cirros and Centos instances. | PASS |
------------------------------------------------------------------------------
Set Unset Properties Instances :: Set Unset properties of Cirros a... | PASS |
------------------------------------------------------------------------------
Evacuate Instances From Hosts :: Evacuate all Cirros and Centos in... | PASS |
------------------------------------------------------------------------------
Sanity-Test.Sanity-OpenStack.01-Instance-From-Image :: Tests to cr... | PASS |
14 critical tests, 14 passed, 0 failed
14 tests total, 14 passed, 0 failed
==============================================================================
Sanity-Test.Sanity-OpenStack.02-Instance-From-Volume :: Tests to create ins...
==============================================================================
Create Flavors for Instances :: Create flavors with or without pro... | PASS |
------------------------------------------------------------------------------
Create Images for Instances :: Create images with or without prope... | PASS |
---------------------------...