ovsdbapp.exceptions.TimeoutException in functional tests

[Boden R]

It appears that as of recent we've been getting some OVS timeouts in various functional jobs [1].
One example is [2] that has a exception message with:

ovsdbapp.exceptions.TimeoutException: Commands [<ovsdbapp.schema.open_vswitch.commands.AddBridgeCommand object at 0x7fcea2645990>, <ovsdbapp.backend.ovs_idl.command.DbAddCommand object at 0x7fcea2645f10>, <ovsdbapp.backend.ovs_idl.command.DbSetCommand object at 0x7fcea5ab45d0>] exceeded timeout 10 seconds

Based on the logstash query [1] it appears this occasionally impacts functional jobs for neutron and networking-ovn.

[1] http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%20%5C%22ovsdbapp.exceptions.TimeoutException%3A%20Commands%5C%22
[2] http://logs.openstack.org/83/633283/2/check/neutron-functional-python27/48c9c98/job-output.txt.gz#_2019-02-07_12_36_06_518870

[Slawek Kaplonski (slaweq)]

It is very similar issue to reported some time ago https://bugs.launchpad.net/neutron/+bug/1802640
I will not mark it as duplicate for now because it is related to different job and timeout happens in different commands for both bugs but for me it looks like the same issue underneath.

[Rodolfo Alonso (rodolfo-alonso-hernandez)]

I think this could be solved (or at least mitigated) with https://bugs.launchpad.net/neutron/+bug/1817022

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/642721

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/642721
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=30e901242f71f783fe06f775b20afa01da41d72f
Submitter: Zuul
Branch: master

commit 30e901242f71f783fe06f775b20afa01da41d72f
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Tue Mar 12 10:59:18 2019 +0000

    Increase timeouts for OVSDB in functional tests

    Increased timeouts for OVSDB connection:
      - ovsdb_timeout = 30

    This patch will mitigate the intermittent timeouts the CI is
    experiencing while running the functional tests.

    Change-Id: I97a1d170926bb8a69dc6f7bb78a785bdea80936a
    Closes-Bug: #1815142

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/682021

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/682022

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.opendev.org/682023

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/682021
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=5da06d7f4270bc79d4ebfe76e591c649c3268615
Submitter: Zuul
Branch: stable/stein

commit 5da06d7f4270bc79d4ebfe76e591c649c3268615
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Tue Mar 12 10:59:18 2019 +0000

    Increase timeouts for OVSDB in functional tests

    Increased timeouts for OVSDB connection:
      - ovsdb_timeout = 30

    This patch will mitigate the intermittent timeouts the CI is
    experiencing while running the functional tests.

    Change-Id: I97a1d170926bb8a69dc6f7bb78a785bdea80936a
    Closes-Bug: #1815142
    (cherry picked from commit 30e901242f71f783fe06f775b20afa01da41d72f)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 15.0.0.0b1 development milestone.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/682022
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=42d9106ee5ba10b0cc6c5f970255688a9484ad78
Submitter: Zuul
Branch: stable/rocky

commit 42d9106ee5ba10b0cc6c5f970255688a9484ad78
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Tue Mar 12 10:59:18 2019 +0000

    Increase timeouts for OVSDB in functional tests

    Increased timeouts for OVSDB connection:
      - ovsdb_timeout = 30

    This patch will mitigate the intermittent timeouts the CI is
    experiencing while running the functional tests.

    Change-Id: I97a1d170926bb8a69dc6f7bb78a785bdea80936a
    Closes-Bug: #1815142
    (cherry picked from commit 30e901242f71f783fe06f775b20afa01da41d72f)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/682023
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=a28fc26aaa924e0c6a97a724d41a492c2cca3249
Submitter: Zuul
Branch: stable/queens

commit a28fc26aaa924e0c6a97a724d41a492c2cca3249
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Tue Mar 12 10:59:18 2019 +0000

    Increase timeouts for OVSDB in functional tests

    Increased timeouts for OVSDB connection:
      - ovsdb_timeout = 30

    This patch will mitigate the intermittent timeouts the CI is
    experiencing while running the functional tests.

    Change-Id: I97a1d170926bb8a69dc6f7bb78a785bdea80936a
    Closes-Bug: #1815142
    (cherry picked from commit 30e901242f71f783fe06f775b20afa01da41d72f)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 14.0.3 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 13.0.5 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 12.1.1 release.