| 2024-01-05 07:25:15 |
Serhii Popovych |
description |
Hello,
docker behavior for name resolution different for user-defined networks (i.e. --network=mynet):
it unconditionally configures Embedded DNS server for this.
However this have side effect on netfilter conntrack. Since Linux kernel commit
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=ba3fbe663635ae7b33a2d972c5d2def036258e42
conntrack hooks registered only when explicitly requested by nft ruleset or enable_hooks modparam).
Above output shows embedded DNS server redirect rules
configured per container with user-defined network:
-----------------------------------------------------
# docker run -idt --name='test' --hostname='test' --network='nft-isolated' --rm ubuntu:jammy bash
# docker inspect test1 | grep '"Pid":'
"Pid": 3334,
# ln -sf /proc/3334/ns/net /var/run/netns/test1
# ip netns exec test1 iptables -L -nv -t nat
Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
0 0 DOCKER_OUTPUT all -- * * 0.0.0.0/0 127.0.0.11
Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
0 0 DOCKER_POSTROUTING all -- * * 0.0.0.0/0 127.0.0.11
Chain DOCKER_OUTPUT (1 references)
pkts bytes target prot opt in out source destination
0 0 DNAT tcp -- * * 0.0.0.0/0 127.0.0.11 tcp dpt:53 to:127.0.0.11:34563
0 0 DNAT udp -- * * 0.0.0.0/0 127.0.0.11 udp dpt:53 to:127.0.0.11:34392
Chain DOCKER_POSTROUTING (1 references)
pkts bytes target prot opt in out source destination
0 0 SNAT tcp -- * * 127.0.0.11 0.0.0.0/0 tcp spt:34563 to::53
0 0 SNAT udp -- * * 127.0.0.11 0.0.0.0/0 udp spt:34392 to::53
-----------------------------------------------------
This ruleset effectively enables conntrack for container:
# docker exec -it test1 cat /proc/sys/net/netfilter/nf_conntrack_count
15
# ip netns exec test1 conntrack -C
15
Counters aren't zero, thus conntrack enabled, and conntrack -L shows entries when executed
with ip-netns(8) exec command.
Since connection tracking hash table isn't per network namespace and having single container
with thousands of conntrack entries can effectively affect other containers and host performance
also causing packet loss:
# dmesg | grep 'nf_conntrack'
nf_conntrack: table full, dropping packet
...
Is there a way to disable embedded DNS server for user-defined networks to avoid conntrack table overflows?
Thanks,
Serhii |
Hello,
docker behavior for name resolution different for user-defined networks (i.e. --network=mynet):
it unconditionally configures Embedded DNS server for this.
However this have side effect on netfilter conntrack. Since Linux kernel commit
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=ba3fbe663635ae7b33a2d972c5d2def036258e42
conntrack hooks registered only when explicitly requested by nft ruleset or enable_hooks modparam).
Above output shows embedded DNS server redirect rules
configured per container with user-defined network:
-----------------------------------------------------
# docker run -idt --name='test1' --hostname='test1' --network='nft-isolated' --rm ubuntu:jammy bash
# docker inspect test1 | grep '"Pid":'
"Pid": 3334,
# ln -sf /proc/3334/ns/net /var/run/netns/test1
# ip netns exec test1 iptables -L -nv -t nat
Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
0 0 DOCKER_OUTPUT all -- * * 0.0.0.0/0 127.0.0.11
Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
0 0 DOCKER_POSTROUTING all -- * * 0.0.0.0/0 127.0.0.11
Chain DOCKER_OUTPUT (1 references)
pkts bytes target prot opt in out source destination
0 0 DNAT tcp -- * * 0.0.0.0/0 127.0.0.11 tcp dpt:53 to:127.0.0.11:34563
0 0 DNAT udp -- * * 0.0.0.0/0 127.0.0.11 udp dpt:53 to:127.0.0.11:34392
Chain DOCKER_POSTROUTING (1 references)
pkts bytes target prot opt in out source destination
0 0 SNAT tcp -- * * 127.0.0.11 0.0.0.0/0 tcp spt:34563 to::53
0 0 SNAT udp -- * * 127.0.0.11 0.0.0.0/0 udp spt:34392 to::53
-----------------------------------------------------
This ruleset effectively enables conntrack for container:
# docker exec -it test1 cat /proc/sys/net/netfilter/nf_conntrack_count
15
# ip netns exec test1 conntrack -C
15
Counters aren't zero, thus conntrack enabled, and conntrack -L shows entries when executed
with ip-netns(8) exec command.
Since connection tracking hash table isn't per network namespace and having single container
with thousands of conntrack entries can effectively affect other containers and host performance
also causing packet loss:
# dmesg | grep 'nf_conntrack'
nf_conntrack: table full, dropping packet
...
Is there a way to disable embedded DNS server for user-defined networks to avoid conntrack table overflows?
Thanks,
Serhii |
|
