SSH timeouts happens very often in the ovn based CI jobs

[Slawek Kaplonski]

I saw those errors mostly in the neutron-ovn-tempest-slow but probably it happens also in other jobs. Example of failures:

https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_0de/791365/7/check/neutron-ovn-tempest-slow/0de1c30/testr_results.html
https://2f7fb53980d59c550f7f-e09de525732b656a1c483807eeb06fc8.ssl.cf2.rackcdn.com/793369/3/check/neutron-ovn-tempest-slow/8a116c7/testr_results.html
https://f86d217b949ada82d82c-f669355b5e0e599ce4f84e6e473a124c.ssl.cf2.rackcdn.com/791365/6/check/neutron-ovn-tempest-slow/5dc8d92/testr_results.html

In all those cases, common thing is that VMs seems to get IP address from dhcp properly, cloud-init seems to be working fine but SSH to the FIP is not possible.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/791365
Committed: https://opendev.org/openstack/neutron/commit/4de350a54a3e3e59371391a7a022447b49ae508e
Submitter: "Zuul (22348)"
Branch: master

commit 4de350a54a3e3e59371391a7a022447b49ae508e
Author: Slawek Kaplonski <email address hidden>
Date: Fri May 14 08:31:32 2021 +0200

    Switch neutron-rally-task job to ML2/OVS and skip "test_models_sync"

    After switch of the default Neutron backend to the ML2/OVN
    in the Devstack we had 2 rally jobs running actually with the
    same Neutron backend.

    This patch configures explicitly ML2/OVS as a backend in the
    'neutron-rally-task' job so it is like it was before the change in
    the Devstack repo.

    In order to fix the CI, this patch also skips "test_models_sync" test
    cases. Once [1] is fixed and released, we'll be able to create SQL
    tables specifying the "_constraints_included" flag. When creating a
    table, this flag should be set to "False" in order to be able to set
    the "index" and "unique" parameters for each column in the table
    definition, as we do now.

    "test_update_router_admin_state" is excluded from
    "neutron-ovn-tempest-slow" until LP#1890445 is fixed.

    Set "test_restart_wsgi_on_sighup_multiple_workers" as unstable.

    CI job "neutron-ovn-tempest-slow" has been set to non-voting until
    LP#1930402 is fixed.

    Related-Bug: #1929518
    Related-Bug: #1890445
    Related-Bug: #1930402
    Related-Bug: #1930367

    [1]https://github.com/sqlalchemy/alembic/issues/848

    Change-Id: I38ed7931169434fecbbefa3762f51a522ef3c509

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/801598
Committed: https://opendev.org/openstack/neutron/commit/02c0b47d2267ee3c48d9991e5da7fd53dcf56556
Submitter: "Zuul (22348)"
Branch: master

commit 02c0b47d2267ee3c48d9991e5da7fd53dcf56556
Author: Slawek Kaplonski <email address hidden>
Date: Wed Jul 21 08:50:12 2021 +0200

    Promote neutron-ovn-tempest-slow job to be voting and gating

    This jobs is almost the same as tempest-slow-py3 since we switched
    OVN to be default backend in Neutron. And that tempest-slow-py3 job
    is used by many projects. So to avoid potential breaks of the gate for
    other projects (like we did recently, see related bug for details)
    let's make this job voting and gating.
    As it is really used in many different projects as voting and gating
    job already I don't think there is any issue with doing the same in
    the Neutron gate.

    Related-bug: #1936983
    Related-bug: #1930402

    Change-Id: I85d3830e9cc65162db846e4858871e1db547a04b

[yatin (yatinkarel)]

Closing it as neutron-ovn-tempest-slow is moved to periodic in master branch and is no longer seeing ssh failures, if it happen again this can be reopened or new issue can be created.