[OVN] neutron-tempest-plugin-scenario-ovn broken with "ovn-northd did not start"

[yatin]

Job failing consistently with below error since https://review.opendev.org/c/openstack/devstack/+/806858:-
2021-11-26 05:58:40.377912 | controller | + functions-common:test_with_retry:2339 : timeout 60 sh -c 'while ! test -e /usr/local/var/run/openvswitch/ovn-northd.pid; do sleep 1; done'
2021-11-26 05:59:40.383253 | controller | + functions-common:test_with_retry:2340 : die 2340 'ovn-northd did not start'
2021-11-26 05:59:40.386420 | controller | + functions-common:die:253 : local exitcode=0

Nov 26 05:58:40.329669 ubuntu-focal-inmotion-iad3-0027510462 bash[107881]: * Creating empty database /opt/stack/data/ovn/ovnsb_db.db
Nov 26 05:58:40.330503 ubuntu-focal-inmotion-iad3-0027510462 bash[107922]: chown: changing ownership of '/opt/stack/data/ovn': Operation not permitted
Nov 26 05:58:40.331416 ubuntu-focal-inmotion-iad3-0027510462 bash[107923]: chown: changing ownership of '/usr/local/var/run/openvswitch/ovs-vswitchd.pid': Operation not permitted
Nov 26 05:58:40.331416 ubuntu-focal-inmotion-iad3-0027510462 bash[107923]: chown: changing ownership of '/usr/local/var/run/openvswitch/ovsdb-server.106684.ctl': Operation not permitted
Nov 26 05:58:40.331647 ubuntu-focal-inmotion-iad3-0027510462 bash[107923]: chown: changing ownership of '/usr/local/var/run/openvswitch/br-ex.mgmt': Operation not permitted
Nov 26 05:58:40.331647 ubuntu-focal-inmotion-iad3-0027510462 bash[107923]: chown: changing ownership of '/usr/local/var/run/openvswitch/ovs-vswitchd.107192.ctl': Operation not permitted

Example logs:-
https://780a11778aeb29655ec5-5f07f27cfda9f7663453c94db6894b0a.ssl.cf5.rackcdn.com/818443/7/check/neutron-tempest-plugin-scenario-ovn/642b702/job-output.txt
https://780a11778aeb29655ec5-5f07f27cfda9f7663453c94db6894b0a.ssl.cf5.rackcdn.com/818443/7/check/neutron-tempest-plugin-scenario-ovn/642b702/controller/logs/screen-ovn-northd.txt

Job Builds:- https://zuul.openstack.org/builds?job_name=neutron-tempest-plugin-scenario-ovn

Other ovn jobs which uses OVN_BUILD_FROM_SOURCE=True will also be impacted. So need to fix affected cases by https://review.opendev.org/c/openstack/devstack/+/806858.

[yatin (yatinkarel)]

Fix Proposed: https://review.opendev.org/c/openstack/devstack/+/819402

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/devstack/+/819402
Committed: https://opendev.org/openstack/devstack/commit/b575af0cfe8aac71825b406394e9927a33ca51bd
Submitter: "Zuul (22348)"
Branch: master

commit b575af0cfe8aac71825b406394e9927a33ca51bd
Author: yatinkarel <email address hidden>
Date: Fri Nov 26 12:44:41 2021 +0530

    Do not use sudo with OVN_DATADIR when building from source

    Jobs with OVN_BUILD_FROMS_SOURCE=True are broken
    since [1] as ovn nortd not starting due to permission
    issues. Fix it by not using sudo for creating OVN_DATADIR
    when building from source.

    [1] https://review.opendev.org/c/openstack/devstack/+/806858

    Closes-Bug: #1952393
    Change-Id: I00f0c8c8173b4d8270fbb3e6079d0d8b332e9de5