Activity log for bug #1899690

Date Who What changed Old value New value Message
2020-10-13 19:32:24 Stefan Lasiewski bug added bug
2020-10-13 19:35:48 Stefan Lasiewski description **What happened**: My Kubernetes 1.17 cluster (and Kubernetes 1.16 cluster) have 40-50 of firewall rules that say `[unsupported revision]`. I am concerned this will cause problems with ``` # iptables-save ... -A KUBE-SEP-AAAREDACTED1 -s 10.99.99.27/32 -j KUBE-MARK-MASQ -A KUBE-SEP-AAAREDACTED1 -p tcp -m tcp -j DNAT [unsupported revision] -A KUBE-SEP-AAAREDACTED2 -s 10.99.99.40/32 -j KUBE-MARK-MASQ -A KUBE-SEP-AAAREDACTED2 -p tcp -m tcp -j DNAT [unsupported revision] -A KUBE-SEP-AAAREDACTED3 -s 10.99.99.27/32 -j KUBE-MARK-MASQ -A KUBE-SEP-AAAREDACTED3 -p tcp -m tcp -j DNAT [unsupported revision] ``` **What you expected to happen**: I expect these firewall rules to be valid, like so: ``` # iptables-save ... -A KUBE-SEP-AAAREDACTED1 -s 10.99.99.27/32 -j KUBE-MARK-MASQ -A KUBE-SEP-AAAREDACTED1 -p tcp -m tcp -j DNAT --to-destination 10.99.99.27:9153 -A KUBE-SEP-AAAREDACTED2 -s 10.99.99.40/32 -j KUBE-MARK-MASQ -A KUBE-SEP-AAAREDACTED2 -p tcp -m tcp -j DNAT --to-destination 10.99.99.40:3306 -A KUBE-SEP-AAAREDACTED3 -s 10.99.99.27/32 -j KUBE-MARK-MASQ -A KUBE-SEP-AAAREDACTED3 -p tcp -m tcp -j DNAT --to-destination 10.99.99.27:53 ``` **How to reproduce it (as minimally and precisely as possible)**: 1. Allocate a worker node 1. Install Ubuntu 18.04.5 2. Install Kubernetes with Canal * I'm using Rancher & RKE, and I assume this happens with vanilla versions of Kubernetes as well. 3. Install the Ubuntu LTS Hardware Enablement (HWE) kernel via https://wiki.ubuntu.com/Kernel/LTSEnablementStack#Server 4. Reboot 5. When the system comes back online & Docker is running, look for invalid iptables rules as shown above. ** Environment ** - 18.04.5 LTS (Bionic Beaver) - Kernel - Linux cntest13 5.4.0-48-generic #52~18.04.1-Ubuntu SMP Thu Sep 10 12:50:22 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux\ - Kubernetes 1.17.11 - Installed via RKE - Canal version: rancher/calico-node:v3.13.4 (Not sure how to tell the other version numbers) - Docker version: 18.9.9 Happens with both Bare Metal and VM systems: - Bare metal nodes - AMD EPYC 7452 32-Core Processor, large memory, multiple NICs to different networks - VMs on VMware vSphere Default iptables version: ``` # iptables --version iptables v1.6.1 # ls -ld `which iptables` lrwxrwxrwx 1 root root 13 Nov 12 2017 /sbin/iptables -> xtables-multi # ``` ProblemType: Bug DistroRelease: Ubuntu 18.04 Package: linux-generic-hwe-18.04 5.4.0.48.52~18.04.42 ProcVersionSignature: Ubuntu 5.4.0-48.52~18.04.1-generic 5.4.60 Uname: Linux 5.4.0-48-generic x86_64 ApportVersion: 2.20.9-0ubuntu7.18 Architecture: amd64 Date: Tue Oct 13 19:19:20 2020 ProcEnviron: TERM=xterm-256color PATH=(custom, no user) XDG_RUNTIME_DIR=<set> LANG=en_US SHELL=/bin/bash SourcePackage: linux-meta-hwe-5.4 UpgradeStatus: No upgrade log present (probably fresh install) **What happened**: I'm running Kubernetes on my Ubuntu 18.04 systems. Kubernetes is installed via Rancher (RKE). My Kubernetes 1.17 cluster (and Kubernetes 1.16 cluster) has 40-50 firewall rules that say `[unsupported revision]`. I am concerned this will cause problems with ``` # iptables-save ... -A KUBE-SEP-AAAREDACTED1 -s 10.99.99.27/32 -j KUBE-MARK-MASQ -A KUBE-SEP-AAAREDACTED1 -p tcp -m tcp -j DNAT [unsupported revision] -A KUBE-SEP-AAAREDACTED2 -s 10.99.99.40/32 -j KUBE-MARK-MASQ -A KUBE-SEP-AAAREDACTED2 -p tcp -m tcp -j DNAT [unsupported revision] -A KUBE-SEP-AAAREDACTED3 -s 10.99.99.27/32 -j KUBE-MARK-MASQ -A KUBE-SEP-AAAREDACTED3 -p tcp -m tcp -j DNAT [unsupported revision] ``` **What you expected to happen**: I expect these firewall rules to be valid, like so: ``` # iptables-save ... -A KUBE-SEP-AAAREDACTED1 -s 10.99.99.27/32 -j KUBE-MARK-MASQ -A KUBE-SEP-AAAREDACTED1 -p tcp -m tcp -j DNAT --to-destination 10.99.99.27:9153 -A KUBE-SEP-AAAREDACTED2 -s 10.99.99.40/32 -j KUBE-MARK-MASQ -A KUBE-SEP-AAAREDACTED2 -p tcp -m tcp -j DNAT --to-destination 10.99.99.40:3306 -A KUBE-SEP-AAAREDACTED3 -s 10.99.99.27/32 -j KUBE-MARK-MASQ -A KUBE-SEP-AAAREDACTED3 -p tcp -m tcp -j DNAT --to-destination 10.99.99.27:53 ``` **How to reproduce it (as minimally and precisely as possible)**: 1. Allocate a worker node 1. Install Ubuntu 18.04.5 2. Install Kubernetes with Canal   * I'm using Rancher & RKE, and I assume this happens with vanilla versions of Kubernetes as well. 3. Install the Ubuntu LTS Hardware Enablement (HWE) kernel via https://wiki.ubuntu.com/Kernel/LTSEnablementStack#Server 4. Reboot 5. When the system comes back online & Docker is running, look for invalid iptables rules as shown above. ** Environment ** - 18.04.5 LTS (Bionic Beaver) - Kernel - Linux cntest13 5.4.0-48-generic #52~18.04.1-Ubuntu SMP Thu Sep 10 12:50:22 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux\ - Kubernetes 1.17.11 - Installed via RKE - Canal version: rancher/calico-node:v3.13.4 (Not sure how to tell the other version numbers) - Docker version: 18.9.9 Happens with both Bare Metal and VM systems: - Bare metal nodes - AMD EPYC 7452 32-Core Processor, large memory, multiple NICs to different networks - VMs on VMware vSphere Default iptables version: ``` # iptables --version iptables v1.6.1 # ls -ld `which iptables` lrwxrwxrwx 1 root root 13 Nov 12 2017 /sbin/iptables -> xtables-multi # ``` ProblemType: Bug DistroRelease: Ubuntu 18.04 Package: linux-generic-hwe-18.04 5.4.0.48.52~18.04.42 ProcVersionSignature: Ubuntu 5.4.0-48.52~18.04.1-generic 5.4.60 Uname: Linux 5.4.0-48-generic x86_64 ApportVersion: 2.20.9-0ubuntu7.18 Architecture: amd64 Date: Tue Oct 13 19:19:20 2020 ProcEnviron:  TERM=xterm-256color  PATH=(custom, no user)  XDG_RUNTIME_DIR=<set>  LANG=en_US  SHELL=/bin/bash SourcePackage: linux-meta-hwe-5.4 UpgradeStatus: No upgrade log present (probably fresh install)
2020-10-13 19:36:11 Stefan Lasiewski description **What happened**: I'm running Kubernetes on my Ubuntu 18.04 systems. Kubernetes is installed via Rancher (RKE). My Kubernetes 1.17 cluster (and Kubernetes 1.16 cluster) has 40-50 firewall rules that say `[unsupported revision]`. I am concerned this will cause problems with ``` # iptables-save ... -A KUBE-SEP-AAAREDACTED1 -s 10.99.99.27/32 -j KUBE-MARK-MASQ -A KUBE-SEP-AAAREDACTED1 -p tcp -m tcp -j DNAT [unsupported revision] -A KUBE-SEP-AAAREDACTED2 -s 10.99.99.40/32 -j KUBE-MARK-MASQ -A KUBE-SEP-AAAREDACTED2 -p tcp -m tcp -j DNAT [unsupported revision] -A KUBE-SEP-AAAREDACTED3 -s 10.99.99.27/32 -j KUBE-MARK-MASQ -A KUBE-SEP-AAAREDACTED3 -p tcp -m tcp -j DNAT [unsupported revision] ``` **What you expected to happen**: I expect these firewall rules to be valid, like so: ``` # iptables-save ... -A KUBE-SEP-AAAREDACTED1 -s 10.99.99.27/32 -j KUBE-MARK-MASQ -A KUBE-SEP-AAAREDACTED1 -p tcp -m tcp -j DNAT --to-destination 10.99.99.27:9153 -A KUBE-SEP-AAAREDACTED2 -s 10.99.99.40/32 -j KUBE-MARK-MASQ -A KUBE-SEP-AAAREDACTED2 -p tcp -m tcp -j DNAT --to-destination 10.99.99.40:3306 -A KUBE-SEP-AAAREDACTED3 -s 10.99.99.27/32 -j KUBE-MARK-MASQ -A KUBE-SEP-AAAREDACTED3 -p tcp -m tcp -j DNAT --to-destination 10.99.99.27:53 ``` **How to reproduce it (as minimally and precisely as possible)**: 1. Allocate a worker node 1. Install Ubuntu 18.04.5 2. Install Kubernetes with Canal   * I'm using Rancher & RKE, and I assume this happens with vanilla versions of Kubernetes as well. 3. Install the Ubuntu LTS Hardware Enablement (HWE) kernel via https://wiki.ubuntu.com/Kernel/LTSEnablementStack#Server 4. Reboot 5. When the system comes back online & Docker is running, look for invalid iptables rules as shown above. ** Environment ** - 18.04.5 LTS (Bionic Beaver) - Kernel - Linux cntest13 5.4.0-48-generic #52~18.04.1-Ubuntu SMP Thu Sep 10 12:50:22 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux\ - Kubernetes 1.17.11 - Installed via RKE - Canal version: rancher/calico-node:v3.13.4 (Not sure how to tell the other version numbers) - Docker version: 18.9.9 Happens with both Bare Metal and VM systems: - Bare metal nodes - AMD EPYC 7452 32-Core Processor, large memory, multiple NICs to different networks - VMs on VMware vSphere Default iptables version: ``` # iptables --version iptables v1.6.1 # ls -ld `which iptables` lrwxrwxrwx 1 root root 13 Nov 12 2017 /sbin/iptables -> xtables-multi # ``` ProblemType: Bug DistroRelease: Ubuntu 18.04 Package: linux-generic-hwe-18.04 5.4.0.48.52~18.04.42 ProcVersionSignature: Ubuntu 5.4.0-48.52~18.04.1-generic 5.4.60 Uname: Linux 5.4.0-48-generic x86_64 ApportVersion: 2.20.9-0ubuntu7.18 Architecture: amd64 Date: Tue Oct 13 19:19:20 2020 ProcEnviron:  TERM=xterm-256color  PATH=(custom, no user)  XDG_RUNTIME_DIR=<set>  LANG=en_US  SHELL=/bin/bash SourcePackage: linux-meta-hwe-5.4 UpgradeStatus: No upgrade log present (probably fresh install) This is a follow up from a bug that I filed for Kubernetes: https://github.com/kubernetes/kubernetes/issues/95409 **What happened**: I'm running Kubernetes on my Ubuntu 18.04 systems. Kubernetes is installed via Rancher (RKE). My Kubernetes 1.17 cluster (and Kubernetes 1.16 cluster) has 40-50 firewall rules that say `[unsupported revision]`. I am concerned this will cause problems with ``` # iptables-save ... -A KUBE-SEP-AAAREDACTED1 -s 10.99.99.27/32 -j KUBE-MARK-MASQ -A KUBE-SEP-AAAREDACTED1 -p tcp -m tcp -j DNAT [unsupported revision] -A KUBE-SEP-AAAREDACTED2 -s 10.99.99.40/32 -j KUBE-MARK-MASQ -A KUBE-SEP-AAAREDACTED2 -p tcp -m tcp -j DNAT [unsupported revision] -A KUBE-SEP-AAAREDACTED3 -s 10.99.99.27/32 -j KUBE-MARK-MASQ -A KUBE-SEP-AAAREDACTED3 -p tcp -m tcp -j DNAT [unsupported revision] ``` **What you expected to happen**: I expect these firewall rules to be valid, like so: ``` # iptables-save ... -A KUBE-SEP-AAAREDACTED1 -s 10.99.99.27/32 -j KUBE-MARK-MASQ -A KUBE-SEP-AAAREDACTED1 -p tcp -m tcp -j DNAT --to-destination 10.99.99.27:9153 -A KUBE-SEP-AAAREDACTED2 -s 10.99.99.40/32 -j KUBE-MARK-MASQ -A KUBE-SEP-AAAREDACTED2 -p tcp -m tcp -j DNAT --to-destination 10.99.99.40:3306 -A KUBE-SEP-AAAREDACTED3 -s 10.99.99.27/32 -j KUBE-MARK-MASQ -A KUBE-SEP-AAAREDACTED3 -p tcp -m tcp -j DNAT --to-destination 10.99.99.27:53 ``` **How to reproduce it (as minimally and precisely as possible)**: 1. Allocate a worker node 1. Install Ubuntu 18.04.5 2. Install Kubernetes with Canal   * I'm using Rancher & RKE, and I assume this happens with vanilla versions of Kubernetes as well. 3. Install the Ubuntu LTS Hardware Enablement (HWE) kernel via https://wiki.ubuntu.com/Kernel/LTSEnablementStack#Server 4. Reboot 5. When the system comes back online & Docker is running, look for invalid iptables rules as shown above. ** Environment ** - 18.04.5 LTS (Bionic Beaver) - Kernel - Linux cntest13 5.4.0-48-generic #52~18.04.1-Ubuntu SMP Thu Sep 10 12:50:22 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux\ - Kubernetes 1.17.11 - Installed via RKE - Canal version: rancher/calico-node:v3.13.4 (Not sure how to tell the other version numbers) - Docker version: 18.9.9 Happens with both Bare Metal and VM systems: - Bare metal nodes - AMD EPYC 7452 32-Core Processor, large memory, multiple NICs to different networks - VMs on VMware vSphere Default iptables version: ``` # iptables --version iptables v1.6.1 # ls -ld `which iptables` lrwxrwxrwx 1 root root 13 Nov 12 2017 /sbin/iptables -> xtables-multi # ``` ProblemType: Bug DistroRelease: Ubuntu 18.04 Package: linux-generic-hwe-18.04 5.4.0.48.52~18.04.42 ProcVersionSignature: Ubuntu 5.4.0-48.52~18.04.1-generic 5.4.60 Uname: Linux 5.4.0-48-generic x86_64 ApportVersion: 2.20.9-0ubuntu7.18 Architecture: amd64 Date: Tue Oct 13 19:19:20 2020 ProcEnviron:  TERM=xterm-256color  PATH=(custom, no user)  XDG_RUNTIME_DIR=<set>  LANG=en_US  SHELL=/bin/bash SourcePackage: linux-meta-hwe-5.4 UpgradeStatus: No upgrade log present (probably fresh install)
2021-11-22 18:09:42 Stefan Lasiewski bug watch added https://github.com/kubernetes/kubernetes/issues/95409