2020-04-20 09:20:50 |
Removed by request |
description |
Setup: Openstack-Ansible cluster(Rocky - 18.1.8) with computes nodes using DVR. OS version Ubuntu 16.04.6 LTS with kernel 4.15.0-34-generic.
Problem: We can see internal IP leaked without NAT on our physical interface. This happens in TCP communication where client stopped abruptly before the server.
Steps to reproduce:
TCP Client(192.168.100.24, 10.96.48.159)
TCP Server(192.168.100.20, 10.96.48.207)
Server sends RST packets on connection termination.
Step1: Start the server and client.
Setp2: Stop the client(KeyboardInterrupt) while the server is still in the connection.
tcpdump on the bond interface of the compute node in which the tcp client is running
07:50:35.658208 IP 10.96.48.159.36394 > 10.96.48.207.5005: Flags [S], seq 3764020836, win 64240, options [mss 1460,sackOK,TS val 2823050719 ecr 0,nop,wscale 7], length 0
07:50:35.658539 IP 10.96.48.207.5005 > 10.96.48.159.36394: Flags [S.], seq 1750463809, ack 3764020837, win 65160, options [mss 1460,sackOK,TS val 2874529221 ecr 2823050719,nop,wscale 7], length 0
07:50:35.658717 IP 10.96.48.159.36394 > 10.96.48.207.5005: Flags [.], ack 1, win 502, options [nop,nop,TS val 2823050720 ecr 2874529221], length 0
07:50:35.658746 IP 10.96.48.159.36394 > 10.96.48.207.5005: Flags [P.], seq 1:14, ack 1, win 502, options [nop,nop,TS val 2823050720 ecr 2874529221], length 13
07:50:35.658949 IP 10.96.48.207.5005 > 10.96.48.159.36394: Flags [.], ack 14, win 509, options [nop,nop,TS val 2874529221 ecr 2823050720], length 0
07:50:35.659113 IP 10.96.48.159.36394 > 10.96.48.207.5005: Flags [P.], seq 14:32, ack 1, win 502, options [nop,nop,TS val 2823050720 ecr 2874529221], length 18
07:50:35.659299 IP 10.96.48.207.5005 > 10.96.48.159.36394: Flags [.], ack 32, win 509, options [nop,nop,TS val 2874529221 ecr 2823050720], length 0
07:50:40.729542 IP 10.96.48.159.36394 > 10.96.48.207.5005: Flags [F.], seq 32, ack 1, win 502, options [nop,nop,TS val 2823055790 ecr 2874529221], length 0
07:50:40.773484 IP 10.96.48.207.5005 > 10.96.48.159.36394: Flags [.], ack 33, win 509, options [nop,nop,TS val 2874534335 ecr 2823055790], length 0
07:53:35.732815 IP 10.96.48.207.5005 > 10.96.48.159.36394: Flags [P.], seq 1:21, ack 33, win 509, options [nop,nop,TS val 2874709290 ecr 2823055790], length 20
07:53:35.732878 IP 10.96.48.207.5005 > 10.96.48.159.36394: Flags [R.], seq 21, ack 33, win 509, options [nop,nop,TS val 2874709291 ecr 2823055790], length 0
07:53:35.733668 IP 192.168.100.24.36394 > 10.96.48.207.5005: Flags [R], seq 3764020869, win 0, length 0
tcpdump on the bond interface of the compute node in which the tcp server is running
07:50:35.658302 IP 10.96.48.159.36394 > 10.96.48.207.5005: Flags [S], seq 3764020836, win 64240, options [mss 1460,sackOK,TS val 2823050719 ecr 0,nop,wscale 7], length 0
07:50:35.658589 IP 10.96.48.207.5005 > 10.96.48.159.36394: Flags [S.], seq 1750463809, ack 3764020837, win 65160, options [mss 1460,sackOK,TS val 2874529221 ecr 2823050719,nop,wscale 7], length 0
07:50:35.658811 IP 10.96.48.159.36394 > 10.96.48.207.5005: Flags [.], ack 1, win 502, options [nop,nop,TS val 2823050720 ecr 2874529221], length 0
07:50:35.658901 IP 10.96.48.159.36394 > 10.96.48.207.5005: Flags [P.], seq 1:14, ack 1, win 502, options [nop,nop,TS val 2823050720 ecr 2874529221], length 13
07:50:35.658998 IP 10.96.48.207.5005 > 10.96.48.159.36394: Flags [.], ack 14, win 509, options [nop,nop,TS val 2874529221 ecr 2823050720], length 0
07:50:35.659205 IP 10.96.48.159.36394 > 10.96.48.207.5005: Flags [P.], seq 14:32, ack 1, win 502, options [nop,nop,TS val 2823050720 ecr 2874529221], length 18
07:50:35.659350 IP 10.96.48.207.5005 > 10.96.48.159.36394: Flags [.], ack 32, win 509, options [nop,nop,TS val 2874529221 ecr 2823050720], length 0
07:50:40.729633 IP 10.96.48.159.36394 > 10.96.48.207.5005: Flags [F.], seq 32, ack 1, win 502, options [nop,nop,TS val 2823055790 ecr 2874529221], length 0
07:50:40.773533 IP 10.96.48.207.5005 > 10.96.48.159.36394: Flags [.], ack 33, win 509, options [nop,nop,TS val 2874534335 ecr 2823055790], length 0
07:53:35.732868 IP 10.96.48.207.5005 > 10.96.48.159.36394: Flags [P.], seq 1:21, ack 33, win 509, options [nop,nop,TS val 2874709290 ecr 2823055790], length 20
07:53:35.732898 IP 10.96.48.207.5005 > 10.96.48.159.36394: Flags [R.], seq 21, ack 33, win 509, options [nop,nop,TS val 2874709291 ecr 2823055790], length 0
07:53:35.733767 IP 192.168.100.24.36394 > 10.96.48.207.5005: Flags [R], seq 3764020869, win 0, length 0
07:53:35.734408 IP 192.168.100.24.36394 > 10.96.48.207.5005: Flags [R], seq 3764020869, win 0, length 0
07:53:35.734602 IP 192.168.100.24.36394 > 10.96.48.207.5005: Flags [R], seq 3764020869, win 0, length 0
07:53:35.734748 IP 192.168.100.24.36394 > 10.96.48.207.5005: Flags [R], seq 3764020869, win 0, length 0
07:53:35.734873 IP 192.168.100.24.36394 > 10.96.48.207.5005: Flags [R], seq 3764020869, win 0, length 0
07:53:35.734973 IP 192.168.100.24.36394 > 10.96.48.207.5005: Flags [R], seq 3764020869, win 0, length 0
07:53:35.735073 IP 192.168.100.24.36394 > 10.96.48.207.5005: Flags [R], seq 3764020869, win 0, length 0
07:53:35.735171 IP 192.168.100.24.36394 > 10.96.48.207.5005: Flags [R], seq 3764020869, win 0, length 0
07:53:35.735269 IP 192.168.100.24.36394 > 10.96.48.207.5005: Flags [R], seq 3764020869, win 0, length 0
07:53:35.735366 IP 192.168.100.24.36394 > 10.96.48.207.5005: Flags [R], seq 3764020869, win 0, length 0
07:53:35.735464 IP 192.168.100.24.36394 > 10.96.48.207.5005: Flags [R], seq 3764020869, win 0, length 0
07:53:35.735561 IP 192.168.100.24.36394 > 10.96.48.207.5005: Flags [R], seq 3764020869, win 0, length 0
07:53:35.735662 IP 192.168.100.24.36394 > 10.96.48.207.5005: Flags [R], seq 3764020869, win 0, length 0
07:53:35.735776 IP 192.168.100.24.36394 > 10.96.48.207.5005: Flags [R], seq 3764020869, win 0, length 0
07:53:35.735877 IP 192.168.100.24.36394 > 10.96.48.207.5005: Flags [R], seq 3764020869, win 0, length 0
07:53:35.735975 IP 192.168.100.24.36394 > 10.96.48.207.5005: Flags [R], seq 3764020869, win 0, length 0
07:53:35.736073 IP 192.168.100.24.36394 > 10.96.48.207.5005: Flags [R], seq 3764020869, win 0, length 0
07:53:35.736171 IP 192.168.100.24.36394 > 10.96.48.207.5005: Flags [R], seq 3764020869, win 0, length 0
07:53:35.736269 IP 192.168.100.24.36394 > 10.96.48.207.5005: Flags [R], seq 3764020869, win 0, length 0
07:53:35.736367 IP 192.168.100.24.36394 > 10.96.48.207.5005: Flags [R], seq 3764020869, win 0, length 0
07:53:35.736465 IP 192.168.100.24.36394 > 10.96.48.207.5005: Flags [R], seq 3764020869, win 0, length 0 |
Setup: Openstack-Ansible cluster(Rocky - 18.1.8) with computes nodes using DVR. OS version Ubuntu 16.04.6 LTS with kernel 4.15.0-34-generic.
Problem: We can see internal IP leaked without NAT on our physical interface. This happens in TCP communication where client stopped abruptly before the server. The leaked packets are always RST packets.
Steps to reproduce:
TCP Client(192.168.100.24, 10.96.48.159)
TCP Server(192.168.100.20, 10.96.48.207)
Server sends RST packets on connection termination.
Step1: Start the server and client.
Setp2: Stop the client(KeyboardInterrupt) while the server is still in the connection.
tcpdump on the bond interface of the compute node in which the tcp client is running
07:50:35.658208 IP 10.96.48.159.36394 > 10.96.48.207.5005: Flags [S], seq 3764020836, win 64240, options [mss 1460,sackOK,TS val 2823050719 ecr 0,nop,wscale 7], length 0
07:50:35.658539 IP 10.96.48.207.5005 > 10.96.48.159.36394: Flags [S.], seq 1750463809, ack 3764020837, win 65160, options [mss 1460,sackOK,TS val 2874529221 ecr 2823050719,nop,wscale 7], length 0
07:50:35.658717 IP 10.96.48.159.36394 > 10.96.48.207.5005: Flags [.], ack 1, win 502, options [nop,nop,TS val 2823050720 ecr 2874529221], length 0
07:50:35.658746 IP 10.96.48.159.36394 > 10.96.48.207.5005: Flags [P.], seq 1:14, ack 1, win 502, options [nop,nop,TS val 2823050720 ecr 2874529221], length 13
07:50:35.658949 IP 10.96.48.207.5005 > 10.96.48.159.36394: Flags [.], ack 14, win 509, options [nop,nop,TS val 2874529221 ecr 2823050720], length 0
07:50:35.659113 IP 10.96.48.159.36394 > 10.96.48.207.5005: Flags [P.], seq 14:32, ack 1, win 502, options [nop,nop,TS val 2823050720 ecr 2874529221], length 18
07:50:35.659299 IP 10.96.48.207.5005 > 10.96.48.159.36394: Flags [.], ack 32, win 509, options [nop,nop,TS val 2874529221 ecr 2823050720], length 0
07:50:40.729542 IP 10.96.48.159.36394 > 10.96.48.207.5005: Flags [F.], seq 32, ack 1, win 502, options [nop,nop,TS val 2823055790 ecr 2874529221], length 0
07:50:40.773484 IP 10.96.48.207.5005 > 10.96.48.159.36394: Flags [.], ack 33, win 509, options [nop,nop,TS val 2874534335 ecr 2823055790], length 0
07:53:35.732815 IP 10.96.48.207.5005 > 10.96.48.159.36394: Flags [P.], seq 1:21, ack 33, win 509, options [nop,nop,TS val 2874709290 ecr 2823055790], length 20
07:53:35.732878 IP 10.96.48.207.5005 > 10.96.48.159.36394: Flags [R.], seq 21, ack 33, win 509, options [nop,nop,TS val 2874709291 ecr 2823055790], length 0
07:53:35.733668 IP 192.168.100.24.36394 > 10.96.48.207.5005: Flags [R], seq 3764020869, win 0, length 0
tcpdump on the bond interface of the compute node in which the tcp server is running
07:50:35.658302 IP 10.96.48.159.36394 > 10.96.48.207.5005: Flags [S], seq 3764020836, win 64240, options [mss 1460,sackOK,TS val 2823050719 ecr 0,nop,wscale 7], length 0
07:50:35.658589 IP 10.96.48.207.5005 > 10.96.48.159.36394: Flags [S.], seq 1750463809, ack 3764020837, win 65160, options [mss 1460,sackOK,TS val 2874529221 ecr 2823050719,nop,wscale 7], length 0
07:50:35.658811 IP 10.96.48.159.36394 > 10.96.48.207.5005: Flags [.], ack 1, win 502, options [nop,nop,TS val 2823050720 ecr 2874529221], length 0
07:50:35.658901 IP 10.96.48.159.36394 > 10.96.48.207.5005: Flags [P.], seq 1:14, ack 1, win 502, options [nop,nop,TS val 2823050720 ecr 2874529221], length 13
07:50:35.658998 IP 10.96.48.207.5005 > 10.96.48.159.36394: Flags [.], ack 14, win 509, options [nop,nop,TS val 2874529221 ecr 2823050720], length 0
07:50:35.659205 IP 10.96.48.159.36394 > 10.96.48.207.5005: Flags [P.], seq 14:32, ack 1, win 502, options [nop,nop,TS val 2823050720 ecr 2874529221], length 18
07:50:35.659350 IP 10.96.48.207.5005 > 10.96.48.159.36394: Flags [.], ack 32, win 509, options [nop,nop,TS val 2874529221 ecr 2823050720], length 0
07:50:40.729633 IP 10.96.48.159.36394 > 10.96.48.207.5005: Flags [F.], seq 32, ack 1, win 502, options [nop,nop,TS val 2823055790 ecr 2874529221], length 0
07:50:40.773533 IP 10.96.48.207.5005 > 10.96.48.159.36394: Flags [.], ack 33, win 509, options [nop,nop,TS val 2874534335 ecr 2823055790], length 0
07:53:35.732868 IP 10.96.48.207.5005 > 10.96.48.159.36394: Flags [P.], seq 1:21, ack 33, win 509, options [nop,nop,TS val 2874709290 ecr 2823055790], length 20
07:53:35.732898 IP 10.96.48.207.5005 > 10.96.48.159.36394: Flags [R.], seq 21, ack 33, win 509, options [nop,nop,TS val 2874709291 ecr 2823055790], length 0
07:53:35.733767 IP 192.168.100.24.36394 > 10.96.48.207.5005: Flags [R], seq 3764020869, win 0, length 0
07:53:35.734408 IP 192.168.100.24.36394 > 10.96.48.207.5005: Flags [R], seq 3764020869, win 0, length 0
07:53:35.734602 IP 192.168.100.24.36394 > 10.96.48.207.5005: Flags [R], seq 3764020869, win 0, length 0
07:53:35.734748 IP 192.168.100.24.36394 > 10.96.48.207.5005: Flags [R], seq 3764020869, win 0, length 0
07:53:35.734873 IP 192.168.100.24.36394 > 10.96.48.207.5005: Flags [R], seq 3764020869, win 0, length 0
07:53:35.734973 IP 192.168.100.24.36394 > 10.96.48.207.5005: Flags [R], seq 3764020869, win 0, length 0
07:53:35.735073 IP 192.168.100.24.36394 > 10.96.48.207.5005: Flags [R], seq 3764020869, win 0, length 0
07:53:35.735171 IP 192.168.100.24.36394 > 10.96.48.207.5005: Flags [R], seq 3764020869, win 0, length 0
07:53:35.735269 IP 192.168.100.24.36394 > 10.96.48.207.5005: Flags [R], seq 3764020869, win 0, length 0
07:53:35.735366 IP 192.168.100.24.36394 > 10.96.48.207.5005: Flags [R], seq 3764020869, win 0, length 0
07:53:35.735464 IP 192.168.100.24.36394 > 10.96.48.207.5005: Flags [R], seq 3764020869, win 0, length 0
07:53:35.735561 IP 192.168.100.24.36394 > 10.96.48.207.5005: Flags [R], seq 3764020869, win 0, length 0
07:53:35.735662 IP 192.168.100.24.36394 > 10.96.48.207.5005: Flags [R], seq 3764020869, win 0, length 0
07:53:35.735776 IP 192.168.100.24.36394 > 10.96.48.207.5005: Flags [R], seq 3764020869, win 0, length 0
07:53:35.735877 IP 192.168.100.24.36394 > 10.96.48.207.5005: Flags [R], seq 3764020869, win 0, length 0
07:53:35.735975 IP 192.168.100.24.36394 > 10.96.48.207.5005: Flags [R], seq 3764020869, win 0, length 0
07:53:35.736073 IP 192.168.100.24.36394 > 10.96.48.207.5005: Flags [R], seq 3764020869, win 0, length 0
07:53:35.736171 IP 192.168.100.24.36394 > 10.96.48.207.5005: Flags [R], seq 3764020869, win 0, length 0
07:53:35.736269 IP 192.168.100.24.36394 > 10.96.48.207.5005: Flags [R], seq 3764020869, win 0, length 0
07:53:35.736367 IP 192.168.100.24.36394 > 10.96.48.207.5005: Flags [R], seq 3764020869, win 0, length 0
07:53:35.736465 IP 192.168.100.24.36394 > 10.96.48.207.5005: Flags [R], seq 3764020869, win 0, length 0 |
|