neutron_ovs_agent constantly restarts on rhel8

[Michele Baldessari]

On rhel8 the ovs_neutron_agent container endlessly restarts with:

2019-03-05 15:41:57.096 534226 DEBUG oslo_concurrency.lockutils [req-a004e117-dba8-42bf-81fe-abf7a9f2faab - - - - -] Lock "conntrack" released by "neutron.agent.linux.ip_conntrack.get_conntrack" :: held 0.257s inner /usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:339
2019-03-05 15:41:57.096 534226 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-a004e117-dba8-42bf-81fe-abf7a9f2faab - - - - -] Exit code: 1; Stdin: ; Stdout: Table `raw' does not exist
; Stderr: Agent terminated!: neutron_lib.exceptions.ProcessExecutionError: Exit code: 1; Stdin: ; Stdout: Table `raw' does not exist
2019-03-05 15:41:57.098 534226 INFO oslo_rootwrap.client [-] Stopping rootwrap daemon process with pid=534237

This is due to an iptables bug listing tables that were not initialized:
https://bugzilla.redhat.com/show_bug.cgi?id=1673609

[Michele Baldessari (michele)]

https://review.openstack.org/641316

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/641316
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=b1d82e6acbc79d9db1a54d53cd178080b72d0845
Submitter: Zuul
Branch: master

commit b1d82e6acbc79d9db1a54d53cd178080b72d0845
Author: Michele Baldessari <email address hidden>
Date: Wed Mar 6 11:53:39 2019 +0100

    Workaround rhel8,0 iptables bug causing neutron_ovs_agent to restart continuously

    On rhel8 the ovs_neutron_agent container endlessly restarts with:

    2019-03-05 15:41:57.096 534226 DEBUG oslo_concurrency.lockutils [req-a004e117-dba8-42bf-81fe-abf7a9f2faab - - - - -] Lock "conntrack" released by "neutron.agent.linux.ip_conntrack.get_conntrack" :: held 0.257s inner /usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:339
    2019-03-05 15:41:57.096 534226 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-a004e117-dba8-42bf-81fe-abf7a9f2faab - - - - -] Exit code: 1; Stdin: ; Stdout: Table `raw' does not exist
    ; Stderr: Agent terminated!: neutron_lib.exceptions.ProcessExecutionError: Exit code: 1; Stdin: ; Stdout: Table `raw' does not exist
    2019-03-05 15:41:57.098 534226 INFO oslo_rootwrap.client [-] Stopping rootwrap daemon process with pid=534237

    This is due to an iptables bug listing tables that were not initialized:
    https://bugzilla.redhat.com/show_bug.cgi?id=1673609

    Let us just load the raw module by listing it so that the issue does not
    appear. Slawomir and I tested this on an environment and could confirm
    that the agent does not restart any longer:
    [root@undercloud-0 ~]# podman logs neutron_ovs_agent 2>&1|grep -i error
    [root@undercloud-0 ~]#

    We are leaving these commands running on any OS version as they are
    harmless in any case.

    Closes-Bug: #1818834

    Co-Authored-By: Slawomir Kaplonski <email address hidden>

    Change-Id: Ifddcec009ae93ad0e51abfe1425eb81c1817db55

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 10.4.0 release.

[Michele Baldessari (michele)]

So the fix for this was actually okay for deployment, but we will still have the issue during a reboot of undercloud/overcloud

[Job (jobcespedes)]

Installation fails when ipv6 is disabled (grub and systectl). Shouldn't be a check to run ip6tables only if ipv6 is enabled?

TASK [Run iptables -t raw -L to workaround a rhel8.0 bug (remove once rhbz#1673609 is
 fixed)] ***
2019-06-13 12:14:56 | fatal: [stein-director-01]: FAILED! => {"changed": true, "cmd": "iptables -t raw -nL\
n ip6tables -t raw -nL", "delta": "0:00:00.039344", "end": "2019-06-13 12:14:56.287777", "msg": "non-zero r
eturn code", "rc": 3, "start": "2019-06-13 12:14:56.248433", "stderr": "ip6tables v1.4.21: can't initialize
 ip6tables table `raw': Address family not supported by protocol\nPerhaps ip6tables or your kernel needs to
 be upgraded.", "stderr_lines": ["ip6tables v1.4.21: can't initialize ip6tables table `raw': Address family
 not supported by protocol", "Perhaps ip6tables or your kernel needs to be upgraded."], "stdout": "Chain PR
EROUTING (policy ACCEPT)\ntarget prot opt source destination \n\nChain OUTPUT (po
licy ACCEPT)\ntarget prot opt source destination ", "stdout_lines": ["Chain PRERO
UTING (policy ACCEPT)", "target prot opt source destination ", "", "Chain OUTPUT
(policy ACCEPT)", "target prot opt source destination "]}

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/train
Review: https://review.opendev.org/692090

[OpenStack Infra (hudson-openstack)]

Change abandoned by Natal Ngétal (<email address hidden>) on branch: stable/train
Review: https://review.opendev.org/692090
Reason: Already merged.

[Natal Ngétal (hobbestigrou)]

It's already cherry-pick in stein.