ping with large package size fails

[norman shen]

We are using neutron rocky, with security driver set to iptables_hybrid, the cluster is deployed on top of a kubernetes cluster. And all the networks are set to mtu 1500

The problem I am facing right now is that ping across compute nodes fails with a packet size larger than mtu.

ping -s 2000 172.20.93.171

Surprisingly, if I ping an IP address from the same node, it works without any issue.

I have done a simple tcpdump on qvb like (both on remote and local compute node)

tcpdump -i qbvxxxx host 172.20.93.171 and icmp

And I saw the traffic, but if I am listening on tap or qbr, no traffic is captured.

I try to add a log iptable rule to debug, by

iptables -t raw -I PREROUTING 1 -m physdev --physdev-in qvb373214e3-8d -p icmp -s 172.20.93.173/12 -j LOG --log-prefix='[netfilter] '

Weird enough, there are no packets counted when packet size set to 2000.

[Brian Haley (brian-haley)]

Packets larger than the MTU have typically always had issues traversing bridges, and there won't be any path MTU info returned to inform the sender. They're most likely being dropped "entering" the tap where the MTU is being enforced.

I guess my question is - why should a 2000 byte packet successfully traverse a network that has a 1500 byte MTU? I would actually expect it to fail. Does using the '-M want' flag to ping make it succeed, which would correctly fragment the packet on transmit? Sorry for seeming negative.

Not sure this is a bug in neutron, I think we'd see the same issue manually configuring a bridge connecting two L2 networks and sending a packets larger than the MTU.

[norman shen (jshen28)]

Hmm... thank you very much for the quick reply. I've tried ping with -M want option but it still does not work. (btw I think pinging will be fragmented if -M do does not set)

My worry is fragmented ping should be able to transmitted through a normal 1500 byte mtu network. I have not tried UDP but I guess it will fail too, since UDP requires fragmentation to transmit large packet, I am worrying it will cause unexpected network interruption.

[norman shen (jshen28)]

Oh, btw if I disable netfilter on bridge by setting /proc/sys/net/bridge/bridge-nf-call-iptables to 0, looks like fragmented packet could reach remote end.

[norman shen (jshen28)]

root@compute34:~# conntrack -L | grep 172.20
udp 17 28 src=172.20.93.189 dst=224.0.0.102 sport=1985 dport=1985 [UNREPLIED] src=224.0.0.102 dst=172.20.93.189 sport=1985 dport=1985 mark=0 zone=4123 use=1
udp 17 28 src=172.20.93.189 dst=224.0.0.102 sport=1985 dport=1985 [UNREPLIED] src=224.0.0.102 dst=172.20.93.189 sport=1985 dport=1985 mark=0 zone=4122 use=1
udp 17 28 src=172.20.93.189 dst=224.0.0.102 sport=1985 dport=1985 [UNREPLIED] src=224.0.0.102 dst=172.20.93.189 sport=1985 dport=1985 mark=0 zone=4121 use=1
udp 17 28 src=172.20.93.189 dst=224.0.0.102 sport=1985 dport=1985 [UNREPLIED] src=224.0.0.102 dst=172.20.93.189 sport=1985 dport=1985 mark=0 zone=4118 use=1
udp 17 28 src=172.20.93.189 dst=224.0.0.102 sport=1985 dport=1985 [UNREPLIED] src=224.0.0.102 dst=172.20.93.189 sport=1985 dport=1985 mark=0 zone=4116 use=1
udp 17 28 src=172.20.93.189 dst=224.0.0.102 sport=1985 dport=1985 [UNREPLIED] src=224.0.0.102 dst=172.20.93.189 sport=1985 dport=1985 mark=0 zone=4117 use=1
udp 17 28 src=172.20.93.189 dst=224.0.0.102 sport=1985 dport=1985 [UNREPLIED] src=224.0.0.102 dst=172.20.93.189 sport=1985 dport=1985 mark=0 zone=4101 use=1
udp 17 28 src=172.20.93.189 dst=224.0.0.102 sport=1985 dport=1985 [UNREPLIED] src=224.0.0.102 dst=172.20.93.189 sport=1985 dport=1985 mark=0 zone=4100 use=1
udp 17 15 src=172.20.3.36 dst=255.255.255.255 sport=54185 dport=1947 [UNREPLIED] src=255.255.255.255 dst=172.20.3.36 sport=1947 dport=54185 mark=0 zone=4111 use=1
tcp 6 86395 ESTABLISHED src=172.20.6.170 dst=172.18.255.21 sport=59309 dport=4305 src=172.18.255.21 dst=172.20.6.170 sport=4305 dport=59309 [ASSURED] mark=0 zone=4103 use=1
icmp 1 26 src=172.20.6.135 dst=172.20.6.150 type=8 code=0 id=21904 [UNREPLIED] src=172.20.6.150 dst=172.20.6.135 type=0 code=0 id=21904 mark=0 zone=4103 use=1
icmp 1 6 src=172.20.6.135 dst=172.20.6.140 type=8 code=0 id=34474 [UNREPLIED] src=172.20.6.140 dst=172.20.6.135 type=0 code=0 id=34474 mark=0 zone=4103 use=1
icmp 1 16 src=172.20.6.135 dst=172.20.6.150 type=8 code=0 id=50471 [UNREPLIED] src=172.20.6.150 dst=172.20.6.135 type=0 code=0 id=50471 mark=0 zone=4103 use=1
udp 17 29 src=172.20.93.188 dst=224.0.0.102 sport=1985 dport=1985 [UNREPLIED] src=224.0.0.102 dst=172.20.93.188 sport=1985 dport=1985 mark=0 zone=4123 use=1
udp 17 29 src=172.20.93.188 dst=224.0.0.102 sport=1985 dport=1985 [UNREPLIED] src=224.0.0.102 dst=172.20.93.188 sport=1985 dport=1985 mark=0 zone=4122 use=1
udp 17 29 src=172.20.93.188 dst=224.0.0.102 sport=1985 dport=1985 [UNREPLIED] src=224.0.0.102 dst=172.20.93.188 sport=1985 dport=1985 mark=0 zone=4121 use=1
udp 17 29 src=172.20.93.188 dst=224.0.0.102 sport=1985 dport=1985 [UNREPLIED] src=224.0.0.102 dst=172.20.93.188 sport=1985 dport=1985 mark=0 zone=4118 use=1
udp 17 29 src=172.20.93.188 dst=224.0.0.102 sport=1985 dport=1985 [UNREPLIED] src=224.0.0.102 dst=172.20.93.188 sport=1985 dport=1985 mark=0 zone=4116 use=1
udp 17 29 src=172.20.93.188 dst=224.0.0.102 sport=1985 dport=1985 [UNREPLIED] src=224.0.0.102 dst=172.20.93.188 sport=1985 dport=1985 mark=0 zone=4117 use=1
udp 17 29 src=172.20.93.188 dst=224.0.0.102 spor...

[Launchpad Janitor (janitor)]

[Expired for neutron because there has been no activity for 60 days.]