Comment 1 for bug 1895547

Revision history for this message
Pedro GuimarĂ£es (pguimaraes) wrote :

Hi,

We just hit this issue on a deployment and I need to say this is not easy to troubleshoot.

The issue we were seeing is that, in Focal, traffic was not getting src NAT from iptables when leaving POD > outside; whereas Bionic it was.

Looking into the iptables, we could see that Focal had no cali-* chains, while Bionic had those.

The reason was indeed the rp_filter errors:

2021-06-25 14:27:17.024 [FATAL][2023] int_dataplane.go 1032: Kernel's RPF check is set to 'loose'. This would allow endpoints to spoof their IP address. Calico requires net.ipv4.conf.all.rp_filter to be set t...

Calico/k8s issue: https://github.com/kubernetes-sigs/kind/issues/891

Indeed, it makes sense the change in behavior on Bionic vs. Focal due to:
https://github.com/systemd/systemd/commit/230450d4e4f1f5fc9fa4295ed9185eea5b6ea16e

We need to consider that iptables is also used by kube-proxy.

On the kubernetes-sigs above, seems they moved rp_filter=1 by default.

Howeer, if we keep the OS value, then we surely need a more clear warning, such as:

if OS>=focal and not ignore-loose-rpf and rp_filter != 1:
block the charm and warn the rp_filter mismatch with Calico config.

Additional reference: https://www.kernel.org/doc/Documentation/networking/ip-sysctl.txt
(search for rp_filter)