Discovered ip addresses mapped to an invalid name (ending with -)

[Joao Andre Simioni]

[Problem Description]

We are seeing some addresses being discovered and their names mapped to invalid entries (in this case, ending with a dash).

This causes named to fail to load that specific Zone file, with a check-names error:

05-Jul-2024 00:14:57.295 /var/snap/maas/35359/bind/zone.redlab.clientnet.com:105: 10-25-62-32-red-clientnet-.redlab.clientnet.com: bad owner name (check-names)
05-Jul-2024 00:14:57.295 zone redlab.clientnet.com/IN: loading from master file /var/snap/maas/35359/bind/zone.redlab.clientnet.com failed: bad owner name (check-names)
05-Jul-2024 00:14:57.295 zone redlab.clientnet.com/IN: not loaded due to errors.

This generates another error in regiond.log:

2024-07-04 18:19:53 maasserver.region_controller: [critical] Failed configuring DNS; killing and restarting
        Traceback (most recent call last):
          File "/snap/maas/35359/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 661, in callback
            self._startRunCallbacks(result)
          File "/snap/maas/35359/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 763, in _startRunCallbacks
            self._runCallbacks()
          File "/snap/maas/35359/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 857, in _runCallbacks
            current.result = callback( # type: ignore[misc]
          File "/snap/maas/35359/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1750, in gotResult
            current_context.run(_inlineCallbacks, r, gen, status)
        --- <exception caught here> ---
          File "/snap/maas/35359/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1660, in _inlineCallbacks
            result = current_context.run(gen.send, result)
          File "/snap/maas/35359/lib/python3.10/site-packages/maasserver/region_controller.py", line 360, in _checkSerial
            raise DNSReloadError(
        maasserver.region_controller.DNSReloadError: Failed to reload DNS; serial mismatch on domains redlab.clientnet.com

Trying to delete or rename these entries won't work, with the following error:

Error:
DNSResource matching query does not exist.

[Environment]

Jammy with MAAS 3.4 using SNAPs. PostgreSQL 14 in a different host.

[Workaround]

We disabled Network Discovery and used the "Clear all discoveries" button. Some invalid entries were deleted, but some were still present.

We identified that the persistent discoveries were DPU devices on some hosts. We found the hosts who had those DPUs and deleted them (the hosts) from MAAS to restore the service.

[Considerations]

The auto-discovered names should pass through a normalization function to make sure they comply to RFC naming convention (suggestion).

Also, it would be important to understand how the DPUs are being discovered even with the discovery disabled.

[Joao Andre Simioni (jasimioni)]

A sos report is available in:

https://drive.google.com/drive/folders/1zulSUU4ao2zAElgNLIECANyM8mJaOvnY

[Alexsander de Souza (alexsander-souza)]

please, open another bug for the DPU discovery issue, it's hard to track the fix of two distinct issue in the same LP issue.

[Joao Andre Simioni (jasimioni)]

Raised https://bugs.launchpad.net/maas/+bug/2072672

[Jerzy Husakowski (jhusakowski)]

https://datatracker.ietf.org/doc/html/rfc1034#section-3.5 specifies that a DNS label cannot end with a hyphen. MAAS must not generate such labels.

[Jerzy Husakowski (jhusakowski)]

Joao, would it be possible to attach the ARP table from MAAS rack controller(s)?

[Joao Andre Simioni (jasimioni)]

Jerzy,

this is a relayed VLAN so MAAS does not have ARP contents in that network.

I'll request that information from one of the machines.

[Joao Andre Simioni (jasimioni)]

ARP output is available at:

https://docs.google.com/document/d/1YfxD_TExhTPJig7lvVQvrpcG0FLro00NgsvSTStQmW8/edit