nova.ci: Failed to establish authenticated ssh connection

[Amit Uniyal]

Description
===========

This is not a frequent fail but I have seen similar fail logs, so seems a good candidate to report a bug.

Failed to establish SSH connection.

login as 'cirros' user. default password: 'gocubsgo'. use 'sudo' for root.
guest-instance-1-domain-com login:
}}}

Traceback (most recent call last):
  File "/opt/stack/tempest/tempest/lib/common/ssh.py", line 136, in _get_ssh_connection
    ssh.connect(self.host, port=self.port, username=self.username,
  File "/opt/stack/tempest/.tox/tempest/lib/python3.10/site-packages/paramiko/client.py", line 365, in connect
    sock.connect(addr)
TimeoutError: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/stack/tempest/tempest/api/compute/servers/test_create_server.py", line 234, in test_create_server_with_fqdn_name
    hostname = linux_client.exec_command("hostname").rstrip()
  File "/opt/stack/tempest/tempest/lib/common/utils/linux/remote_client.py", line 31, in wrapper
    return function(self, *args, **kwargs)
  File "/opt/stack/tempest/tempest/lib/common/utils/linux/remote_client.py", line 115, in exec_command
    return self.ssh_client.exec_command(cmd)
  File "/opt/stack/tempest/tempest/lib/common/ssh.py", line 187, in exec_command
    ssh = self._get_ssh_connection()
  File "/opt/stack/tempest/tempest/lib/common/ssh.py", line 155, in _get_ssh_connection
    raise exceptions.SSHTimeout(host=self.host,
tempest.lib.exceptions.SSHTimeout: Connection to the 172.24.5.83 via SSH timed out.
User: cirros, Password: E0+G1Pi+8%rB8G5

Environment
===========
CI on master branch

Distro: Ubuntu 22.04
in-detail: https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_52d/881457/33/check/nova-next/52de8e3/job-output.txt

This is not a deployment issue though.

Logs & Configs
==============
https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_52d/881457/33/check/nova-next/52de8e3/testr_results.html

[Amit Uniyal (auniyal)]

https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_05d/877500/1/gate/nova-next/05ddada/testr_results.html

2023-08-26 08:05:46,889 111697 INFO [tempest.lib.common.rest_client] Request (TestEncryptedCinderVolumes:_run_cleanups): 202 DELETE https://10.209.36.25/volume/v3/a5e15aa66a9a40d8a53dc4e33a6e5d3c/types/be51605d-5eca-4fe0-be11-4acf9b352d7f 0.134s
2023-08-26 08:05:46,890 111697 DEBUG [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
        Body: None
    Response - Headers: {'date': 'Sat, 26 Aug 2023 08:05:46 GMT', 'server': 'Apache/2.4.41 (Ubuntu)', 'content-length': '0', 'content-type': 'text/html; charset=UTF-8', 'openstack-api-version': 'volume 3.0', 'vary': 'OpenStack-API-Version', 'x-openstack-request-id': 'req-5defcbe0-d586-48d0-a27f-913953d59339', 'connection': 'close', 'status': '202', 'content-location': 'https://10.209.36.25/volume/v3/a5e15aa66a9a40d8a53dc4e33a6e5d3c/types/be51605d-5eca-4fe0-be11-4acf9b352d7f'}
        Body: b''
}}}

Traceback (most recent call last):
  File "/opt/stack/tempest/tempest/common/utils/__init__.py", line 70, in wrapper
    return f(*func_args, **func_kwargs)
  File "/opt/stack/tempest/tempest/scenario/test_encrypted_cinder_volumes.py", line 98, in test_encrypted_cinder_volumes_cryptsetup
    server = self.launch_instance()
  File "/opt/stack/tempest/tempest/scenario/test_encrypted_cinder_volumes.py", line 49, in launch_instance
    return self.create_server(wait_until='SSHABLE')
  File "/opt/stack/tempest/tempest/scenario/manager.py", line 340, in create_server
    body, _ = compute.create_test_server(
  File "/opt/stack/tempest/tempest/common/compute.py", line 357, in create_test_server
    LOG.exception('Server %s failed to delete in time',
  File "/opt/stack/tempest/.tox/tempest/lib/python3.8/site-packages/oslo_utils/excutils.py", line 227, in __exit__
    self.force_reraise()
  File "/opt/stack/tempest/.tox/tempest/lib/python3.8/site-packages/oslo_utils/excutils.py", line 200, in force_reraise
    raise self.value
  File "/opt/stack/tempest/tempest/common/compute.py", line 333, in create_test_server
    wait_for_ssh_or_ping(
  File "/opt/stack/tempest/tempest/common/compute.py", line 148, in wait_for_ssh_or_ping
    waiters.wait_for_ssh(
  File "/opt/stack/tempest/tempest/common/waiters.py", line 649, in wait_for_ssh
    raise lib_exc.TimeoutException()
tempest.lib.exceptions.TimeoutException: Request timed out
Details: None