octavia healthcheck fails in scenario010

[Rabi Mishra]

validation check fails:

TASK [validate-services : Print out any failed Systemd services for tripleo_*] ***
Thursday 08 August 2019 04:11:59 +0000 (0:00:00.823) 1:04:33.618 *******
ok: [undercloud] => {
    "systemd_state.stdout_lines": [
        "tripleo_octavia_api_healthcheck.service loaded failed failed octavia_api healthcheck"
    ]
}

TASK [validate-services : Fails if we find failed systemd units] ***************
Thursday 08 August 2019 04:12:00 +0000 (0:00:00.264) 1:04:33.883 *******
fatal: [undercloud]: FAILED! => {
    "assertion": "systemd_state.stdout_lines|length == 0",
    "changed": false,
    "evaluated_to": false

journal log:

Aug 08 04:13:00 standalone.localdomain systemd[1]: tripleo_octavia_api_healthcheck.service: main process exited, code=exited, status=1/FAILURE
Aug 08 04:13:00 standalone.localdomain systemd[1]: Failed to start octavia_api healthcheck.
Aug 08 04:13:00 standalone.localdomain systemd[1]: Unit tripleo_octavia_api_healthcheck.service entered failed state.
Aug 08 04:13:00 standalone.localdomain systemd[1]: tripleo_octavia_api_healthcheck.service failed.

https://logs.opendev.org/28/675028/2/check/tripleo-ci-centos-7-scenario010-standalone/82847db/logs/undercloud/var/log/journal.txt.gz#_Aug_08_04_08_59

Octavia api does not seem to start and driver load fails:

2019-08-08 04:18:21.860 22 DEBUG octavia.api.app [-] api_settings.udp_connect_min_interval_health_monitor = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2589
2019-08-08 04:18:21.860 22 DEBUG octavia.api.app [-] ******************************************************************************** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2591
2019-08-08 04:18:22.169 21 ERROR octavia.api.drivers.driver_factory [-] Unable to load provider driver ovn due to: Unable to open the driver agent socket: /var/run/octavia/status.sock: DriverAgentNotFound: Unable to open the driver agent socket: /var/run/octavia/status.sock

I think this has started happening after octavia-lib started checking for sockets for the drivers.

https://github.com/openstack/octavia-lib/commit/d700c00a90fd62b4f6cb9eb30ebe5f619dd6bfda

[Rabi Mishra (rabi)]

Probably we should disable ovn provider driver in scenario010?

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/675263

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/675301

[Rabi Mishra (rabi)]

I've submitted 2 patches (puppet-tripleo/tht). If that's not the direction we want to go, feel free to submit new patches and abandon those.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/675352

[OpenStack Infra (hudson-openstack)]

Change abandoned by Rabi Mishra (<email address hidden>) on branch: master
Review: https://review.opendev.org/675263

[OpenStack Infra (hudson-openstack)]

Change abandoned by Brent Eagles (<email address hidden>) on branch: master
Review: https://review.opendev.org/675352

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/675301
Committed: https://git.openstack.org/cgit/openstack/puppet-tripleo/commit/?id=517eea74811fa5e61ea6010a620ffb50d207844b
Submitter: Zuul
Branch: master

commit 517eea74811fa5e61ea6010a620ffb50d207844b
Author: Rabi Mishra <email address hidden>
Date: Thu Aug 8 16:30:13 2019 +0530

    Remove ovn specific logic for octavia

    We probably should not have the ovn neutron driver specific
    logic here. This deprecates and ignores 'neutron_driver'
    parameter.

    Change-Id: I9ba293d6a245ee875bf273d116baf64e44d0a554
    Closes-Bug: #1839436

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/puppet-tripleo 11.2.0 release.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/687501

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/687501
Committed: https://git.openstack.org/cgit/openstack/puppet-tripleo/commit/?id=fe160dd5a875c0f4898797489bb2914b5f7302d0
Submitter: Zuul
Branch: stable/stein

commit fe160dd5a875c0f4898797489bb2914b5f7302d0
Author: Rabi Mishra <email address hidden>
Date: Thu Aug 8 16:30:13 2019 +0530

    Remove ovn specific logic for octavia

    We probably should not have the ovn neutron driver specific
    logic here. This deprecates and ignores 'neutron_driver'
    parameter.

    Change-Id: I9ba293d6a245ee875bf273d116baf64e44d0a554
    Closes-Bug: #1839436
    (cherry picked from commit 517eea74811fa5e61ea6010a620ffb50d207844b)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/puppet-tripleo 10.5.2 release.