SNAT is not working

[Vinicius Coelho]

Centos 8.3, Openstack Ussuri.
I have 3 controllers node and 2 network nodes.
I'm using self-service network with linuxbridge.
The SNAT is not working. A tcpdump (in the destination) shows that the ip is not being masquerade. If I assing a floating IP, everything works.
Here is the router iptables:
# Generated by iptables-save v1.8.4 on Thu Apr 22 09:30:43 2021
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:neutron-filter-top - [0:0]
:neutron-l3-agent-FORWARD - [0:0]
:neutron-l3-agent-INPUT - [0:0]
:neutron-l3-agent-OUTPUT - [0:0]
:neutron-l3-agent-local - [0:0]
:neutron-l3-agent-scope - [0:0]
-A INPUT -j neutron-l3-agent-INPUT
-A FORWARD -j neutron-filter-top
-A FORWARD -j neutron-l3-agent-FORWARD
-A OUTPUT -j neutron-filter-top
-A OUTPUT -j neutron-l3-agent-OUTPUT
-A neutron-filter-top -j neutron-l3-agent-local
-A neutron-l3-agent-FORWARD -j neutron-l3-agent-scope
-A neutron-l3-agent-INPUT -m mark --mark 0x1/0xffff -j ACCEPT
-A neutron-l3-agent-INPUT -p tcp -m tcp --dport 9697 -j DROP
-A neutron-l3-agent-scope -o qr-ba03dc8a-87 -m mark ! --mark 0x4010000/0xffff0000 -j DROP
COMMIT
# Completed on Thu Apr 22 09:30:43 2021
# Generated by iptables-save v1.8.4 on Thu Apr 22 09:30:43 2021
*mangle
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:neutron-l3-agent-FORWARD - [0:0]
:neutron-l3-agent-INPUT - [0:0]
:neutron-l3-agent-OUTPUT - [0:0]
:neutron-l3-agent-POSTROUTING - [0:0]
:neutron-l3-agent-PREROUTING - [0:0]
:neutron-l3-agent-float-snat - [0:0]
:neutron-l3-agent-floatingip - [0:0]
:neutron-l3-agent-mark - [0:0]
:neutron-l3-agent-scope - [0:0]
-A PREROUTING -j neutron-l3-agent-PREROUTING
-A INPUT -j neutron-l3-agent-INPUT
-A FORWARD -j neutron-l3-agent-FORWARD
-A OUTPUT -j neutron-l3-agent-OUTPUT
-A POSTROUTING -j neutron-l3-agent-POSTROUTING
-A neutron-l3-agent-POSTROUTING -o qg-33922118-c1 -m connmark --mark 0x0/0xffff0000 -j CONNMARK --save-mark --nfmask 0xffff0000 --ctmask 0xffff0000
-A neutron-l3-agent-PREROUTING -j neutron-l3-agent-mark
-A neutron-l3-agent-PREROUTING -j neutron-l3-agent-scope
-A neutron-l3-agent-PREROUTING -m connmark ! --mark 0x0/0xffff0000 -j CONNMARK --restore-mark --nfmask 0xffff0000 --ctmask 0xffff0000
-A neutron-l3-agent-PREROUTING -j neutron-l3-agent-floatingip
-A neutron-l3-agent-PREROUTING -d 169.254.169.254/32 -i qr-+ -p tcp -m tcp --dport 80 -j MARK --set-xmark 0x1/0xffff
-A neutron-l3-agent-float-snat -m connmark --mark 0x0/0xffff0000 -j CONNMARK --save-mark --nfmask 0xffff0000 --ctmask 0xffff0000
-A neutron-l3-agent-mark -i qg-33922118-c1 -j MARK --set-xmark 0x2/0xffff
-A neutron-l3-agent-scope -i qr-ba03dc8a-87 -j MARK --set-xmark 0x4010000/0xffff0000
-A neutron-l3-agent-scope -i qg-33922118-c1 -j MARK --set-xmark 0x4010000/0xffff0000
COMMIT
# Completed on Thu Apr 22 09:30:43 2021
# Generated by iptables-save v1.8.4 on Thu Apr 22 09:30:43 2021
*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:neutron-l3-agent-OUTPUT - [0:0]
:neutron-l3-agent-POSTROUTING - [0:0]
:neutron-l3-agent-PREROUTING - [0:0]
:neutron-l3-agent-float-snat - [0:0]
:neutron-l3-agent-snat - [0:0]
:neutron-postrouting-bottom - [0:0]
-A PREROUTING -j neutron-l3-agent-PREROUTING
-A POSTROUTING -j neutron-l3-agent-POSTROUTING
-A POSTROUTING -j neutron-postrouting-bottom
-A OUTPUT -j neutron-l3-agent-OUTPUT
-A neutron-l3-agent-POSTROUTING ! -o qg-33922118-c1 -m conntrack ! --ctstate DNAT -j ACCEPT
-A neutron-l3-agent-PREROUTING -d 169.254.169.254/32 -i qr-+ -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 9697
-A neutron-l3-agent-snat -j neutron-l3-agent-float-snat
-A neutron-l3-agent-snat -o qg-33922118-c1 -m connmark --mark 0x4010000/0xffff0000 -j ACCEPT
-A neutron-l3-agent-snat -o qg-33922118-c1 -j SNAT --to-source X.X.X.X --random-fully
-A neutron-l3-agent-snat -m mark ! --mark 0x2/0xffff -m conntrack --ctstate DNAT -j SNAT --to-source X.X.X.X --random-fully
-A neutron-postrouting-bottom -m comment --comment "Perform source NAT on outgoing traffic." -j neutron-l3-agent-snat
COMMIT
# Completed on Thu Apr 22 09:30:43 2021
# Generated by iptables-save v1.8.4 on Thu Apr 22 09:30:43 2021
*raw
:PREROUTING ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:neutron-l3-agent-OUTPUT - [0:0]
:neutron-l3-agent-PREROUTING - [0:0]
-A PREROUTING -j neutron-l3-agent-PREROUTING
-A OUTPUT -j neutron-l3-agent-OUTPUT
COMMIT
# Completed on Thu Apr 22 09:30:43 2021

[Rodolfo Alonso (rodolfo-alonso-hernandez)]

Hello Vinicius:

Can you add more detail to this bug?

If I'm not wrong, what you are trying is to connect a VM to provider network using a router (a trivial case), using a router. And the router is centralized in the controller node. Correct?

Are you using HA or legacy routers?

What OS are you using?

What version of iptables do you have? I see you are using iptables v1.8.4. Is this the nftables binary or the legacy one?

Are you using iptables Firewall? Are you using security groups?

Where are you capturing the traffic? Where/what is this destination port?

Regards.

[Vinicius Coelho (viniciuscoelho21)]

Hi Rodolfo,
 Yes, you are correct. The problem is the traffic throught the router. The router is centralized.
 Im not using HA and my OS is CentOS 8.3 and iptables version is 1.8.4 and is nftables.

 Im not using iptables Firewall and I'm using security group.

 MY network configuration:
 Tentant network: 172.16.0.0/24
 Provider network: 177.190.X.X/24

 External client (outside openstack infrastructure ): 178.190.X.X

 When I try to access my external client from one server that is in the tenant network I can see the packets, but those packets are not masqueraded. So the packets are coming as 172.16.0.0 to my external client.

 If I assing an floating IP, everything works. If i try to access directly from the router, everything works too.

[Vinicius Coelho (viniciuscoelho21)]

Hello, does anyone have any idea how to get around this problem?

[LIU Yulong (dragon889)]

The "enable_snat" of router gateway is True or False? If it is False, then your SNAT function of this router will not work.

[Vinicius Coelho (viniciuscoelho21)]

Hi, The enable_snat is enabled