RuntimeError: Connection to the server '03907c5e-9ddc-402c-813b-4a4ee0eea8df' through port 3344 is not possible. failing on periodic standalone-full-tempest-scenario-master

[Bhagyashri Shewale]

Test:

neutron_tempest_plugin.scenario.test_port_forwardings.PortForwardingTestJSON.test_port_forwarding_editing_and_deleting_udp_rule

Error log:

Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/neutron_tempest_plugin/common/utils.py", line 80, in wait_until_true
    eventlet.sleep(sleep)
  File "/usr/lib/python3.6/site-packages/eventlet/greenthread.py", line 36, in sleep
    hub.switch()
  File "/usr/lib/python3.6/site-packages/eventlet/hubs/hub.py", line 313, in switch
    return self.greenlet.switch()
eventlet.timeout.Timeout: 60 seconds

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/neutron_tempest_plugin/scenario/test_port_forwardings.py", line 232, in test_port_forwarding_editing_and_deleting_udp_rule
    server[0]['port_forwarding_udp']['external_port'])))
  File "/usr/lib/python3.6/site-packages/neutron_tempest_plugin/common/utils.py", line 84, in wait_until_true
    raise exception
RuntimeError: Connection to the server 'ff2879b9-b1de-427f-9e67-8046aa375140' through port 3344 is not possible.

Test:

neutron_tempest_plugin.scenario.test_port_forwardings.PortForwardingTestJSON.test_port_forwarding_to_2_servers

Error log:

Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/neutron_tempest_plugin/scenario/test_port_forwardings.py", line 131, in test_port_forwarding_to_2_servers
    self._test_udp_port_forwarding(servers)
  File "/usr/lib/python3.6/site-packages/neutron_tempest_plugin/scenario/test_port_forwardings.py", line 122, in _test_udp_port_forwarding
    **wait_params)
  File "/usr/lib/python3.6/site-packages/neutron_tempest_plugin/common/utils.py", line 79, in wait_until_true
    while not predicate():
  File "/usr/lib/python3.6/site-packages/neutron_tempest_plugin/scenario/test_port_forwardings.py", line 121, in <lambda>
    lambda: _message_received(server, ssh_client, expected_msg),
  File "/usr/lib/python3.6/site-packages/neutron_tempest_plugin/scenario/test_port_forwardings.py", line 103, in _message_received
    constants.PROTO_NAME_UDP)
  File "/usr/lib/python3.6/site-packages/neutron_tempest_plugin/scenario/base.py", line 586, in nc_client
    self.assertEqual(0, result.exit_status)
  File "/usr/lib/python3.6/site-packages/testtools/testcase.py", line 411, in assertEqual
    self.assertThat(observed, matcher, message)
  File "/usr/lib/python3.6/site-packages/testtools/testcase.py", line 498, in assertThat
    raise mismatch_error
testtools.matchers._impl.MismatchError: 0 != 127

Jobs affected:

  - periodic-tripleo-ci-centos-8-standalone-full-tempest-scenario-master
  - periodic-tripleo-ci-centos-8-ovb-1ctlr_2comp-featureset020-master

#link:

1. https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-standalone-full-tempest-scenario-master/abffa5f/logs/undercloud/var/log/tempest/stestr_results.html.gz

2. https://review.rdoproject.org/zuul/builds?pipeline=openstack-periodic-integration-main&job_name=periodic-tripleo-ci-centos-8-standalone-full-tempest-scenario-master

3. https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-standalone-full-tempest-scenario-master/abffa5f/logs/undercloud/var/log/extra/errors.txt.txt.gz

4. https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-1ctlr_2comp-featureset020-master/41be1a5/logs/undercloud/var/log/tempest/stestr_results.html.gz

[Slawek Kaplonski (slaweq)]

I think that problem may be with that:

2021-03-11 03:11:30,483 223094 DEBUG [neutron_tempest_plugin.common.shell] Command 'ncat --version 2>&1' failed (exit_status=127):
stderr:

stdout:
/bin/sh: ncat: command not found

I will try to check that

[Bhagyashri Shewale (bhagyashri-shewale)]

Hi All,

Here is the fix for this issue: https://review.opendev.org/c/openstack/openstack-ansible-os_tempest/+/780588

This patch add the failing test to tempest skiplist: https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/779941

The patch [1] which removes the test from skiplist as we have fix ready for failing tempest tests.

[1]: https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/780694

Testproject patch: https://review.rdoproject.org/r/c/testproject/+/32411

Note: the jobs are failing on the testproject patch because of other tempest tests failure.

Thanks Slawek :)

[Ronelle Landy (rlandy)]

no activity since march - closing this out

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/openstack-ansible-os_tempest yoga-eom release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/openstack-ansible-os_tempest wallaby-eom release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/openstack-ansible-os_tempest xena-eom release.