[neutron-tempest-plugin] SSH timeout exceptions when executing remote commands

Bug #1844516 reported by Rodolfo Alonso on 2019-09-18
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Medium
Rodolfo Alonso

Bug Description

Those SSH timeout exceptions have been detected during the execution of the scenario QoS tests, in particular "test_qos_basic_and_update".

Those SSH timeouts happened during the execution of the following commands:
- "_kill_nc_process", killing any existing "nc" command being executed in the remote host.
- creating the "nc" server in the remote host.

Logs:
[1]https://9a0240ca9f61a595b570-86672578d4e6ceb498f2d932b0da6815.ssl.cf1.rackcdn.com/633871/20/check/neutron-tempest-plugin-scenario-openvswitch/772f7a4/testr_results.html.gz
[2]https://1fd93ff32a555bc48a73-5fe9d093373d887f2b09d5c4b981e1db.ssl.cf2.rackcdn.com/652099/34/check/neutron-tempest-plugin-scenario-openvswitch-rocky/4897a52/testr_results.html.gz
[3]https://openstack.fortnebula.com:13808/v1/AUTH_e8fd161dc34c421a979a9e6421f823e9/zuul_opendev_logs_cf0/679510/1/check/neutron-tempest-plugin-scenario-openvswitch/cf055c6/testr_results.html.gz

Fix proposed to branch: master
Review: https://review.opendev.org/682864

Changed in neutron:
assignee: nobody → Rodolfo Alonso (rodolfo-alonso-hernandez)
status: New → In Progress
Changed in neutron:
importance: Undecided → Medium

Reviewed: https://review.opendev.org/682864
Committed: https://git.openstack.org/cgit/openstack/neutron-tempest-plugin/commit/?id=aa65dfb5265536eca40dbaf9b1826f8bf5148f80
Submitter: Zuul
Branch: master

commit aa65dfb5265536eca40dbaf9b1826f8bf5148f80
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Wed Sep 18 11:30:04 2019 +0000

    Add retry decorator to SSH "execute" method

    In case of SSH timeout (TimeoutException, TimeoutError), the
    tenacity.retry decorator retries the execution of the SSH
    "execute" method up to 10 times.

    Some SSH execute calls, related to QoS scenario tests, have been
    enhanced by setting a relatively small timeout value. The commands
    executed should be quick enough to be executed in this amount of time.
    In case of timeout (due to communication problems), the retry decorator
    will send again the command to be executed.

    Change-Id: Idc0d55b776f499a4bc5d8c9d9a549f0af8f3fac0
    Closes-Bug: #1844516

Changed in neutron:
status: In Progress → Fix Released

This issue was fixed in the openstack/neutron-tempest-plugin 0.6.0 release.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers