dhcp client resets IP address every few minutes

[Yevhen Ivanchenko]

Hello,
Network manager resets IP address every 30 minutes (perhaps this time depends on default-lease-time)

This issue affects at least the latest package Network Manager in Ubuntu 20.04.2 - network-manager 1.22.10-1ubuntu2.2.

It seems this issue has been occuring more than a year (perhaps from release Ubuntu 20.04) and is not fixed yet.
It is possible that this issue occurs only with our dhcp server. We have not checked if the issue occurs with other dhcp servers.

----
lsb_release -rd
Description: Ubuntu 20.04.2 LTS
Release: 20.04
----
apt-cache policy network-manager
network-manager:
  Installed: 1.22.10-1ubuntu2.2
  Candidate: 1.22.10-1ubuntu2.2
  Version table:
 *** 1.22.10-1ubuntu2.2 500
        500 http://ua.archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages
        100 /var/lib/dpkg/status
     1.22.10-1ubuntu1 500
        500 http://ua.archive.ubuntu.com/ubuntu focal/main amd64 Packages
-----

STR:
Install Ubuntu 20.04.2 (update if needed) and connect to network with dhcp server.

AR:
Connection drops every few minutes, then NetworkManager reconnects again.
On screen we can see message "Network disconnect" and right away "Network connect"

ER:
Connection should be stable.

We have found the similar bug here - https://bugzilla.redhat.com/show_bug.cgi?id=1817788
It is possible that in my case the root of issue is the same and the isue is fixed in network manager 1.22.12

We need a new networkmanager package in Ubuntu 20.04 repository which has not got this bug.

There is a part of /var/log/syslog
Here we can see error "error -113 dispatching events"
----
Jun 16 16:01:40 user-System-Product-Name NetworkManager[589]: <debug> [1623848500.8092] dhcp4 (enp3s0): send REQUEST of 192.168.61.156 to 192.168.61.2
Jun 16 16:01:40 user-System-Product-Name NetworkManager[589]: <error> [1623848500.8225] dhcp4 (enp3s0): error -113 dispatching events
Jun 16 16:01:40 user-System-Product-Name NetworkManager[589]: <info> [1623848500.8226] dhcp4 (enp3s0): state changed bound -> fail
Jun 16 16:01:40 user-System-Product-Name NetworkManager[589]: <debug> [1623848500.8226] device[5565e4c243c020e9] (enp3s0): new DHCPv4 client state 6
Jun 16 16:01:40 user-System-Product-Name NetworkManager[589]: <debug> [1623848500.8227] device[5565e4c243c020e9] (enp3s0): DHCPv4 failed (ip_state done, was_active 0)
Jun 16 16:01:40 user-System-Product-Name NetworkManager[589]: <info> [1623848500.8228] device (enp3s0): DHCPv4: trying to acquire a new lease within 90 seconds
Jun 16 16:01:40 user-System-Product-Name NetworkManager[589]: <trace> [1623848500.8228] dbus-object[cffc3e670403e5f9]: unexport: "/org/freedesktop/NetworkManager/DHCP4Config/8"
Jun 16 16:01:40 user-System-Product-Name NetworkManager[589]: <trace> [1623848500.8232] dbus-object[e4f7ca28a56cb5dc]: export: "/org/freedesktop/NetworkManager/DHCP4Config/9"
Jun 16 16:02:18 user-System-Product-Name NetworkManager[589]: <trace> [1623848538.4785] device[5565e4c243c020e9] (enp3s0): connectivity: [IPv6] periodic-check: re-scheduled in 299970 milliseconds (300 se
conds interval)
Jun 16 16:02:18 user-System-Product-Name NetworkManager[589]: <trace> [1623848538.4787] device[5565e4c243c020e9] (enp3s0): connectivity: [IPv6] start check (seq:86, periodic-check)
Jun 16 16:02:18 user-System-Product-Name NetworkManager[589]: <debug> [1623848538.4790] connectivity: (enp3s0,IPv6,86) start request to 'http://connectivity-check.ubuntu.com/' (try resolving 'connectivit
y-check.ubuntu.com' using systemd-resolved)
Jun 16 16:02:18 user-System-Product-Name NetworkManager[589]: <debug> [1623848538.4833] connectivity: (enp3s0,IPv6,86) can't resolve a name via systemd-resolved: GDBus.Error:org.freedesktop.resolve1.NoSu
chRR: 'connectivity-check.ubuntu.com' does not have any RR of the requested type
Jun 16 16:02:18 user-System-Product-Name systemd-resolved[549]: Server returned error NXDOMAIN, mitigating potential DNS violation DVE-2018-0001, retrying transaction with reduced feature level UDP.
Jun 16 16:02:18 user-System-Product-Name systemd-resolved[549]: message repeated 2 times: [ Server returned error NXDOMAIN, mitigating potential DNS violation DVE-2018-0001, retrying transaction with reduced feature level UDP.]

----

We are attaching tcpdump file for investigation.
Here client sent a request and server answered with ACK. It seem that client failed on the ACK and after 90 seconds client sent a new request.

[Yevhen Ivanchenko (yevhenivanchenko)]

Adding syslog file

[Yevhen Ivanchenko (yevhenivanchenko)]

The reason why dhcp client reset connection was an ICMP message
When the client send a REQUEST, the gateway replied with ICMP code 10 (Host administratively prohibited)
Then dhcp server replied with ACK
--
It happened because iptables on the server did not allow incoming connections to port 67 of the dhcp server and it responded with an ICMP (Host administratively prohibited).
But it seems the requests came to the dhcp server anyway and dhcp server responded with ACK.

I sent only dhcp packets in the dump file but there no ICMP messages were included.