test_network_policies fails on "Reaching out to nginx.netpolicy with restrictions"

Bug #1912814 reported by Michael Skalka
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Charmed Kubernetes Testing
Invalid
Undecided
Unassigned

Bug Description

As seen during this test run: https://solutions.qa.canonical.com/testruns/testRun/74ac7049-5b1e-4ced-94e1-b01314b5dd3b
Artifacts here: https://oil-jenkins.canonical.com/artifacts/74ac7049-5b1e-4ced-94e1-b01314b5dd3b/generated/generated/kubernetes/juju-crashdump-kubernetes-2021-01-22-10.38.07.tar.gz

test_network_policies apparently fails to reach out to nginx, however there are no errors in the pod log from the test.

From the crashdump:
less kubernetes-worker_1/pod-logs/netpolicy-nginx-deployment-6bd4c896d6-vqrcn-nginx.log

/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
10-listen-on-ipv6-by-default.sh: Getting the checksum of /etc/nginx/conf.d/default.conf
10-listen-on-ipv6-by-default.sh: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf
/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
/docker-entrypoint.sh: Configuration complete; ready for start up
192.168.231.197 - - [22/Jan/2021:10:26:38 +0000] "GET / HTTP/1.1" 200 612 "-" "Wget" "-"
192.168.164.4 - - [22/Jan/2021:10:26:50 +0000] "GET / HTTP/1.1" 200 612 "-" "Wget" "-"
...
192.168.164.4 - - [22/Jan/2021:10:36:52 +0000] "GET / HTTP/1.1" 200 612 "-" "Wget" "-"
192.168.231.197 - - [22/Jan/2021:10:36:58 +0000] "GET / HTTP/1.1" 200 612 "-" "Wget" "-"
192.168.164.4 - - [22/Jan/2021:10:37:09 +0000] "GET / HTTP/1.1" 200 612 "-" "Wget" "-"
192.168.231.197 - - [22/Jan/2021:10:37:15 +0000] "GET / HTTP/1.1" 200 612 "-" "Wget" "-"
EOF

Revision history for this message
George Kraft (cynerva) wrote :

The test is failing because Calico isn't properly enforcing network policies. The expectation at this point in the test is that requests from bboxbad (192.168.231.197) should *not* reach nginx, because a NetworkPolicy exists that should deny it.

Calico isn't enforcing network policies because felix is crashing:

2021-01-22 10:39:37.801 [FATAL][35660] int_dataplane.go 1032: Kernel's RPF check is set to 'loose'. This would allow endpoints to spoof their IP address. Calico requires net.ipv4.conf.all.rp_filter to be set to 0 or 1. If you require loose RPF and you are not concerned about spoofing, this check can be disabled by setting the IgnoreLooseRPF configuration parameter to 'true'.

You need to either configure the calico charm with ignore-loose-rpf=true, or update the kubernetes-master and kubernetes-worker sysctl charm config to include setting net.ipv4.conf.all.rp_filter to 0 or 1.

Changed in charmed-kubernetes-testing:
status: New → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.