ssh fails between VMs during test_basic_snat_behavior_with_fip test

[Vedamurthy Joshi]

R2.1 Build 6 Ubuntu 12.04.3 Multi-node setup

python -m testtools.run scripts.neutron.test_routers.TestRouterSNAT.test_basic_snat_behavior_with_fip

In the above test, a public VM on one compute tries to ssh to another VM in private VN using its floating ip.

It was seen that ssh is failing. Praveen debugged the issue. Recent bridging changes seems to have broken MTU calculations in vrouter.

root@nodec55:~# tcpdump -ni tapb3bc7243-37 host 10.204.219.91
tcpdump: WARNING: tapb3bc7243-37: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tapb3bc7243-37, link-type EN10MB (Ethernet), capture size 65535 bytes
00:47:26.937087 ARP, Request who-has 10.204.219.91 tell 10.204.219.89, length 28
00:47:26.937199 ARP, Reply 10.204.219.91 is-at 02:b3:bc:72:43:37, length 28
 00:47:27.646226 IP 10.204.219.93.34198 > 10.204.219.91.22: Flags [F.], seq 1571559361, ack 1790301180, win 237, options [nop,nop,TS val 2042312 ecr 1998862,nop,nop,sack 1 {1409:1633}], length 0
00:47:27.646550 IP 10.204.219.91.22 > 10.204.219.93.34198: Flags [.], ack 1, win 257, options [nop,nop,TS val 2034015 ecr 2042312], length 0
^[[A00:47:27.906091 IP 10.204.219.93.34199 > 10.204.219.91.22: Flags [S], seq 1061010332, win 29200, options [mss 1420,sackOK,TS val 2042377 ecr 0,nop,wscale 7], length 0
00:47:27.906312 IP 10.204.219.91.22 > 10.204.219.93.34199: Flags [S.], seq 1634829830, ack 1061010333, win 28960, options [mss 1460,sackOK,TS val 2034080 ecr 2042377,nop,wscale 7], length 0
00:47:27.906816 IP 10.204.219.93.34199 > 10.204.219.91.22: Flags [.], ack 1, win 229, options [nop,nop,TS val 2042377 ecr 2034080], length 0
00:47:27.907275 IP 10.204.219.93.34199 > 10.204.219.91.22: Flags [P.], seq 1:42, ack 1, win 229, options [nop,nop,TS val 2042377 ecr 2034080], length 41
00:47:27.907385 IP 10.204.219.91.22 > 10.204.219.93.34199: Flags [.], ack 42, win 227, options [nop,nop,TS val 2034080 ecr 2042377], length 0
00:47:27.916060 IP 10.204.219.91.22 > 10.204.219.93.34199: Flags [P.], seq 1:42, ack 42, win 227, options [nop,nop,TS val 2034082 ecr 2042377], length 41
00:47:27.916511 IP 10.204.219.93.34199 > 10.204.219.91.22: Flags [.], ack 42, win 229, options [nop,nop,TS val 2042379 ecr 2034082], length 0
00:47:27.916832 IP 10.204.219.93.34199 > 10.204.219.91.22: Flags [P.], seq 42:2010, ack 42, win 229, options [nop,nop,TS val 2042380 ecr 2034082], length 1968
00:47:27.916969 IP 10.204.219.91.22 > 10.204.219.93.34199: Flags [.], ack 2010, win 257, options [nop,nop,TS val 2034083 ecr 2042380], length 0
00:47:27.917011 IP 10.204.219.91.22 > 10.204.219.93.34199: Flags [.], seq 42:1450, ack 2010, win 257, options [nop,nop,TS val 2034083 ecr 2042380], length 1408
00:47:27.917032 IP 10.204.219.91.22 > 10.204.219.93.34199: Flags [P.], seq 1450:1674, ack 2010, win 257, options [nop,nop,TS val 2034083 ecr 2042380], length 224
00:47:27.917285 IP 10.204.219.93.34199 > 10.204.219.91.22: Flags [.], ack 42, win 237, options [nop,nop,TS val 2042380 ecr 2034083,nop,nop,sack 1 {1450:1674}], length 0
00:47:27.917384 IP 10.204.219.89 > 10.204.219.91: ICMP 10.204.219.93 unreachable - need to frag (mtu 1454), length 136
00:47:27.917486 IP 10.204.219.91.22 > 10.204.219.93.34199: Flags [.], seq 42:1444, ack 2010, win 257, options [nop,nop,TS val 2034083 ecr 2042380], length 1402
00:47:27.917504 IP 10.204.219.91.22 > 10.204.219.93.34199: Flags [.], seq 1444:1450, ack 2010, win 257, options [nop,nop,TS val 2034083 ecr 2042380], length 6
00:47:27.917684 IP 10.204.219.89 > 10.204.219.91: ICMP 10.204.219.93 unreachable - need to frag (mtu 1454), length 136
00:47:27.917794 IP 10.204.219.93.34199 > 10.204.219.91.22: Flags [.], ack 42, win 245, options [nop,nop,TS val 2042380 ecr 2034083,nop,nop,sack 1 {1444:1674}], length 0
00:47:27.937371 ARP, Request who-has 10.204.219.91 tell 10.204.219.89, length 28
00:47:27.937754 ARP, Reply 10.204.219.91 is-at 02:b3:bc:72:43:37, length 28
00:47:28.120248 IP 10.204.219.91.22 > 10.204.219.93.34199: Flags [.], seq 42:1444, ack 2010, win 257, options [nop,nop,TS val 2034134 ecr 2042380], length 1402
00:47:28.120448 IP 10.204.219.89 > 10.204.219.91: ICMP 10.204.219.93 unreachable - need to frag (mtu 1454), length 136 <<<<<<<<
00:47:28.528257 IP 10.204.219.91.22 > 10.204.219.93.34199: Flags [.], seq 42:1444, ack 2010, win 257, options [nop,nop,TS val 2034236 ecr 2042380], length 1402
00:47:28.528423 IP 10.204.219.89 > 10.204.219.91: ICMP 10.204.219.93 unreachable - need to frag (mtu 1454), length 136
 00:47:29.344278 IP 10.204.219.91.22 > 10.204.219.93.34199: Flags [.], seq 42:1444, ack 2010, win 257, options [nop,nop,TS val 2034440 ecr 2042380], length 1402
00:47:29.344507 IP 10.204.219.89 > 10.204.219.91: ICMP 10.204.219.93 unreachable - need to frag (mtu 1454), length 136
00:47:30.980257 IP 10.204.219.91.22 > 10.204.219.93.34199: Flags [.], seq 42:1444, ack 2010, win 257, options [nop,nop,TS val 2034849 ecr 2042380], length 1402
00:47:30.980497 IP 10.204.219.89 > 10.204.219.91: ICMP 10.204.219.93 unreachable - need to frag (mtu 1454), length 136
^C
31 packets captured

[Divakar Dharanalakota (ddivakar)]

Vrouter changes are committed with the change id in R2.1 1597b3622821ff5d7ecec2c721b565b69998b4e8
-Divakar