vRouter should do DNAT on ICMP error pkts destined to VMs behind FIP

[amit surana]

consider a case where a VM, that has a FIP assigned to its port, pings a remote destination. Now, lets say that the packet gets dropped by an intermediate underlay router (for whatever reason) and an ICMP errors is sent back to the client. This packet would have source IP as intermediate underlay router, and dest-ip as FIP. Before this error is forwarded to the source VM, DNAT must be done and the FIP must be translated to the senders actual IP. This does not occur. This breaks features like PMTU discover cause the source VM will just ignore the incoming ICMP error.

secondly, consider the same case as above, but this time the VM is trying to reach an underlay server via LLS. If the packet is dropped by the local vRouter (TTL expiry, lets say), the source/dest IP of the icmp error packet seen on the client is the vhost0 IP.

Finally, consider a case where a packet sourced from a VM is dropped by vRouter (lets say it was a traceroute packet). Then the ICMP TTL-exceeded ICMP error packet generated by vRouter has the vRouter's fabric/vhost0 IP as source-ip and VM IP as destination IP. Instead, the packet should have the VN gateway IP as the source-ip (underlay IP should be hidden from the VM).

LLS flow. Local vRouter drops the packet due to TTL expiry. vhost0 ip is 172.16.180.11. ICMP error packet has vhost0 ip as source/dest.

root@soln-slave-vm:~# hping3 169.254.169.1 -p 7000 -k -S -t 1
HPING 169.254.169.1 (eth0 169.254.169.1): S set, 40 headers + 0 data bytes
16:27:56.807014 02:4c:32:2e:21:d6 > 00:00:5e:00:01:00, ethertype IPv4 (0x0800), length 54: 41.1.1.3.1822 > 169.254.169.1.7000: Flags [S], seq 269816718, win 512, length 0
16:27:56.808668 90:e2:ba:50:b3:5c > 02:4c:32:2e:21:d6, ethertype IPv4 (0x0800), length 82: 172.16.180.11 > 172.16.180.11: ICMP time exceeded in-transit, length 48
16:27:56.808680 02:4c:32:2e:21:d6 > 00:00:5e:00:01:00, ethertype IPv4 (0x0800), length 82: 172.16.180.11 > 172.16.180.11: ICMP time exceeded in-transit, length 48
16:27:57.807600 02:4c:32:2e:21:d6 > 00:00:5e:00:01:00, ethertype IPv4 (0x0800), length 54: 41.1.1.3.1822 > 169.254.169.1.7000: Flags [S], seq 939458930, win 512, length 0
16:27:57.808034 90:e2:ba:50:b3:5c > 02:4c:32:2e:21:d6, ethertype IPv4 (0x0800), length 82: 172.16.180.11 > 172.16.180.11: ICMP time exceeded in-transit, length 48

FIP flow: VM local IP is 41.1.1.3, FIP is 51.1.1.3. This time the ICMP error packet is going to the FIP IP, rather than the VMs untranslated IP.

root@soln-slave-vm:~# hping3 52.1.1.3 -p 7000 -k -S -t 1
HPING 52.1.1.3 (eth0 52.1.1.3): S set, 40 headers + 0 data bytes
16:26:53.290725 02:4c:32:2e:21:d6 > 00:00:5e:00:01:00, ethertype IPv4 (0x0800), length 54: 41.1.1.3.2975 > 52.1.1.3.7000: Flags [S], seq 1287204978, win 512, length 0
16:26:53.292209 90:e2:ba:50:b3:5c > 02:4c:32:2e:21:d6, ethertype IPv4 (0x0800), length 82: 172.16.180.11 > 51.1.1.3: ICMP time exceeded in-transit, length 48
16:26:53.292218 02:4c:32:2e:21:d6 > 00:00:5e:00:01:00, ethertype IPv4 (0x0800), length 82: 172.16.180.11 > 51.1.1.3: ICMP time exceeded in-transit, length 48
16:26:54.291282 02:4c:32:2e:21:d6 > 00:00:5e:00:01:00, ethertype IPv4 (0x0800), length 54: 41.1.1.3.2975 > 52.1.1.3.7000: Flags [S], seq 1268583658, win 512, length 0
16:26:54.291715 90:e2:ba:50:b3:5c > 02:4c:32:2e:21:d6, ethertype IPv4 (0x0800), length 82: 172.16.180.11 > 51.1.1.3: ICMP time exceeded in-transit, length 48

[Anand H. Krishnan (anandhk)]

ICMP errors, in this case, are generated by agent and directly sent to the interface.

[Hari Prasad Killi (haripk)]

TTL expired ICMP having vhost IP as source was fixed with https://bugs.launchpad.net/juniperopenstack/+bug/1616430.

[Hari Prasad Killi (haripk)]

For the first part of the bug, vrouter peeks into the ICMP error, gets the original flow and then does NAT in this case.