I checked it on the test node and here is what I found.
When job failed it was ovs 2.12 running:
$ sudo ovs-vsctl show
a05a7746-2638-49c9-8f3a-4c293fb6aff2
Manager "ptcp:6640:127.0.0.1" is_connected: true
Bridge br-ctlplane
fail_mode: standalone
Port br-ctlplane Interface br-ctlplane type: internal
Bridge br-int
fail_mode: secure
Port br-int Interface br-int type: internal
Bridge br-ex
Port br-ex Interface br-ex type: internal
ovs_version: "2.12.0"
and because of that there was no patch ports between bridges created and in that it was same bug as [CIX][BZ:1989974][osp17][rhel8][RHOS17][RHOS16.2]Multinode jobs are failing to ping the router ip address in os_tempest.
But there was already installed ovs 2.13:
sudo su bash-4.4# rpm -qa | grep openvswitch rdo-openvswitch-2.13-3.el8.noarch python3-rdo-openvswitch-2.13-3.el8.noarch openvswitch-selinux-extra-policy-1.0-28.el8.noarch python3-openvswitch2.13-2.13.0-122.el8.x86_64 network-scripts-openvswitch2.13-2.13.0-122.el8.x86_64 rdo-network-scripts-openvswitch-2.13-3.el8.noarch openvswitch2.13-2.13.0-122.el8.x86_64
So I decided to restart openvswitch on the node, and then all seemed to be configured properly:
bash-4.4# systemctl restart openvswitch
bash-4.4# ovs-vsctl show
a05a7746-2638-49c9-8f3a-4c293fb6aff2
Manager "ptcp:6640:127.0.0.1" is_connected: true
Bridge br-ctlplane
fail_mode: standalone
Port patch-provnet-eb568cc3-4ac1-43a2-9ca8-080bc5a44aca-to-br-int Interface patch-provnet-eb568cc3-4ac1-43a2-9ca8-080bc5a44aca-to-br-int type: patch options: {peer=patch-br-int-to-provnet-eb568cc3-4ac1-43a2-9ca8-080bc5a44aca}
Port br-ctlplane Interface br-ctlplane type: internal
Bridge br-int
fail_mode: secure datapath_type: system
Port br-int Interface br-int type: internal
Port patch-br-int-to-provnet-eb568cc3-4ac1-43a2-9ca8-080bc5a44aca Interface patch-br-int-to-provnet-eb568cc3-4ac1-43a2-9ca8-080bc5a44aca type: patch options: {peer=patch-provnet-eb568cc3-4ac1-43a2-9ca8-080bc5a44aca-to-br-int}
Bridge br-ex
Port br-ex Interface br-ex type: internal
ovs_version: "2.13.5"
So it seems that we should install ovs 2.13 instead of 2.12 at the beginning of the job or restart it after update of that package before tempest is run.
I checked it on the test node and here is what I found.
When job failed it was ovs 2.12 running:
$ sudo ovs-vsctl show 2638-49c9- 8f3a-4c293fb6af f2 127.0.0. 1"
is_connected: true
Interface br-ctlplane
type: internal
Interface br-int
type: internal
Interface br-ex
type: internal
a05a7746-
Manager "ptcp:6640:
Bridge br-ctlplane
fail_mode: standalone
Port br-ctlplane
Bridge br-int
fail_mode: secure
Port br-int
Bridge br-ex
Port br-ex
ovs_version: "2.12.0"
and because of that there was no patch ports between bridges created and in that it was same bug as [CIX][BZ: 1989974] [osp17] [rhel8] [RHOS17] [RHOS16. 2]Multinode jobs are failing to ping the router ip address in os_tempest.
But there was already installed ovs 2.13: -2.13-3. el8.noarch python3- rdo-openvswitch -2.13-3. el8.noarch openvswitch- selinux- extra-policy- 1.0-28. el8.noarch python3- openvswitch2. 13-2.13. 0-122.el8. x86_64 network- scripts- openvswitch2. 13-2.13. 0-122.el8. x86_64 rdo-network- scripts- openvswitch- 2.13-3. el8.noarch openvswitch2. 13-2.13. 0-122.el8. x86_64
sudo su bash-4.4# rpm -qa | grep openvswitch rdo-openvswitch
So I decided to restart openvswitch on the node, and then all seemed to be configured properly:
bash-4.4# systemctl restart openvswitch 2638-49c9- 8f3a-4c293fb6af f2 127.0.0. 1"
is_connected: true eb568cc3- 4ac1-43a2- 9ca8-080bc5a44a ca-to-br- int
Interface patch-provnet- eb568cc3- 4ac1-43a2- 9ca8-080bc5a44a ca-to-br- int
type: patch
options: {peer=patch- br-int- to-provnet- eb568cc3- 4ac1-43a2- 9ca8-080bc5a44a ca}
Interface br-ctlplane
type: internal
datapath_ type: system
Interface br-int
type: internal int-to- provnet- eb568cc3- 4ac1-43a2- 9ca8-080bc5a44a ca
Interface patch-br- int-to- provnet- eb568cc3- 4ac1-43a2- 9ca8-080bc5a44a ca
type: patch
options: {peer=patch- provnet- eb568cc3- 4ac1-43a2- 9ca8-080bc5a44a ca-to-br- int}
Interface br-ex
type: internal
bash-4.4# ovs-vsctl show
a05a7746-
Manager "ptcp:6640:
Bridge br-ctlplane
fail_mode: standalone
Port patch-provnet-
Port br-ctlplane
Bridge br-int
fail_mode: secure
Port br-int
Port patch-br-
Bridge br-ex
Port br-ex
ovs_version: "2.13.5"
So it seems that we should install ovs 2.13 instead of 2.12 at the beginning of the job or restart it after update of that package before tempest is run.