"ovn_agent._run_process" ovs-appctl fails setting the service log configuration

[Rodolfo Alonso]

This happens when using OVN as backend and enabled the OVS/OVN compilation.

Error:
+lib/neutron_plugins/ovn_agent:_run_process:256 sudo ovs-appctl -t ovn-northd vlog/set console:off syslog:info file:info
2022-02-10T14:22:26Z|00001|daemon_unix|WARN|/var/run/openvswitch/ovn-northd.pid: open: No such file or directory

However, when the ctl file is used (/usr/local/var/run/openvswitch/ovn-northd.<PID>.ctl), the command works fine.

"ovs-appctl" should use the ctl file instead of the service name.

[yatin (yatinkarel)]

The path(/var/run/openvswitch/ovn-northd.pid) in ERROR looks suspicious, shouldn't that be "/usr/local/var/run/openvswitch/ovn-northd.pid" when OVN_BUILD_FROM_SOURCE=True[1]?

https://github.com/openstack/devstack/blob/master/lib/neutron_plugins/ovn_agent#L109-L111

@Rodolfo you seeing the issue in some CI job or local deploy? asking as i have not seen this issue in CI and in my local deploy's with OVN_BUILD_FROM_SOURCE="True", may be some other config needed to reproduce it?

[Rodolfo Alonso (rodolfo-alonso-hernandez)]

Hi Yatin:

I'm seeing this error in a local deploy. Actually I can't deploy a single node installation with OVS/OVN compilation with devstack out of the box. I'm manually executing all devstack steps to install both services to check what is going wrong. At some point, ovs stops working with the error
"Datapath invalid". Still investigating...

Regards.

[Rodolfo Alonso (rodolfo-alonso-hernandez)]

More info: once the ovs-vswitch and the ovsdb services are running, I have problems with them. When I try to create a new bridge, I receive this output:
stack@controller:/opt/stack$ sudo ovs-vsctl add-br br-ex4
ovs-vsctl: Error detected while setting up 'br-ex4'. See ovs-vswitchd log for details.
ovs-vsctl: The default log directory is "/var/log/openvswitch".

This is the vswitchd log: https://paste.opendev.org/show/beAtkLYwKuc8kz0jH0qO/

[Rodolfo Alonso (rodolfo-alonso-hernandez)]

Ok, the problem was related to OVN_BUILD_MODULES=True. This is needed for DPDK environments only.

In any case, I'll push a patch to properly handle:
- The "ovs-appctl" service reference
- Make the OVS and OVN prefix directories the same: /usr/local

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/devstack/+/828877

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/830563

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/devstack/+/828877
Committed: https://opendev.org/openstack/devstack/commit/8c6710326eaf8114b579720185161091a0a9f38c
Submitter: "Zuul (22348)"
Branch: master

commit 8c6710326eaf8114b579720185161091a0a9f38c
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Wed Feb 9 18:01:46 2022 +0000

    Fix installation with OVN backend and compilation

    This patch fixes several issues related to the installation with
    OVN backend with the OVS/OVN compilation enabled.

    The OVS/OVN local directories prefix, when both services are compiled,
    is now "/usr/local".

    The "ovn_agent._run_process" function is calling "ovs-appctl" to
    configure the logging settings of several services. Instead of
    using the service name, the ctl socket file is used instead. That
    is more robust and does not fail in systems with previous
    installations.

    Closes-Bug: #1960514

    Change-Id: I69de5333393957593db6e05495f0c3c758efefdf

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/830563
Committed: https://opendev.org/openstack/neutron/commit/d5ac103329a3c7410234c6d88c56a9fdc4fe43df
Submitter: "Zuul (22348)"
Branch: master

commit d5ac103329a3c7410234c6d88c56a9fdc4fe43df
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Wed Feb 9 22:18:06 2022 +0000

    Change OVS and OVN installation directories

    Change OVS and OVN installation directiories when building from
    source, in order to sync with [1].

    [1]https://review.opendev.org/c/openstack/devstack/+/828877

    Depends-On: https://review.opendev.org/c/openstack/devstack/+/828877

    Change-Id: I38775450de7d671febd07ead98f6eb4c70fc7d1e
    Related-Bug: #1960514