dhcpd.conf not written due to byte size of hosts value in rpc

[Cory]

I am using version 2.8.7-8611-g.f2514168f-0ubuntu1~18.04.1

MaaS stops being able to write the dhcpd.conf and never starts the dhcpd service.

Taking a look at the python module used by MaaS it appears there is a 64k byte limit.

2022-06-17 16:33:30 maasserver.dhcp: [critical] Error configuring DHCPv4 on rack controller 'maas-server (7gyesb)':
 Traceback (most recent call last):
   File "/usr/lib/python3/dist-packages/provisioningserver/prometheus/utils.py", line 126, in wrapper
     result = func(*args, **kwargs)
   File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 142, in wrapper
     return func(*args, **kwargs)
   File "/usr/lib/python3/dist-packages/provisioningserver/rpc/common.py", line 178, in __call__
     timeout, self._conn.callRemote, cmd, **kwargs
   File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 340, in deferWithTimeout
     d = maybeDeferred(func, *args, **kwargs)
 --- <exception caught here> ---
   File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 150, in maybeDeferred
     result = f(*args, **kw)
   File "/usr/lib/python3/dist-packages/twisted/protocols/amp.py", line 971, in callRemote
     return co._doCommand(self)
   File "/usr/lib/python3/dist-packages/twisted/protocols/amp.py", line 2002, in _doCommand
     self.requiresAnswer)
   File "/usr/lib/python3/dist-packages/provisioningserver/rpc/common.py", line 263, in _sendBoxCommand
     command, box, requiresAnswer=requiresAnswer
   File "/usr/lib/python3/dist-packages/twisted/protocols/amp.py", line 902, in _sendBoxCommand
     box._sendTo(self.boxSender)
   File "/usr/lib/python3/dist-packages/twisted/protocols/amp.py", line 723, in _sendTo
     proto.sendBox(self)
   File "/usr/lib/python3/dist-packages/twisted/protocols/amp.py", line 2386, in sendBox
     self.transport.write(box.serialize())
   File "/usr/lib/python3/dist-packages/twisted/protocols/amp.py", line 701, in serialize
     raise TooLong(False, True, v, k)
 twisted.protocols.amp.TooLong:

[Alexsander de Souza (alexsander-souza)]

Hi,

can you share some characteristics of your setup?

1) number of subnets
2) number of hosts
3) are you using DHCP snippets?

[Cory (user195182)]

1) 2 Subnets
2) 3300
3) Yes a global snippet which doesn't appear in the UI anymore since we started experiencing the mentioned issue.

[Alexsander de Souza (alexsander-souza)]

MAAS RPC protocol has a 64kB limitation but usually the compression we use avoids hitting this limit. We need to add pagination to this request.

[Jerzy Husakowski (jhusakowski)]

This issue appears to happen on clusters with large DHCP config, and needs to be investigated & resolved.

Note: MAAS 2.8 is no longer supported, so we encourage to upgrade to the latest MAAS version.

[Mauricio Faria de Oliveira (mfo)]

This has been reported to still affect MAAS 3.3.4.

[Dan Emmons (dan-emmons)]

It's been reported that this issue still affects version 3.3.4, and we are currently working to reproduce this and confirm that it affects the current release version as well.

[Jorge Merlino (jorge-merlino)]

I can reproduce a similar behavior in 3.3.7 using dhcp snippets with random text. I have a 92K dhcpd.conf and get that same twisted.protocols.amp.TooLong exception when I try to add another snippet. It is not the exact same traceback but it is the same once the AMP library code is reached:

2024-07-25 21:04:05 maasserver.websockets.protocol: [critical] Error on request (106) dhcpsnippet.create:
        Traceback (most recent call last):
          File "/snap/maas/35704/lib/python3.10/site-packages/provisioningserver/prometheus/utils.py", line 127, in wrapper
            result = func(*args, **kwargs)
          File "/snap/maas/35704/lib/python3.10/site-packages/provisioningserver/utils/twisted.py", line 128, in wrapper
            return func(*args, **kwargs)
          File "/snap/maas/35704/lib/python3.10/site-packages/provisioningserver/rpc/common.py", line 206, in __call__
            d = deferWithTimeout(timeout, self._conn.callRemote, cmd, **kwargs)
          File "/snap/maas/35704/lib/python3.10/site-packages/provisioningserver/utils/twisted.py", line 326, in deferWithTimeout
            d = maybeDeferred(func, *args, **kwargs)
        --- <exception caught here> ---
          File "/snap/maas/35704/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 244, in inContext
            result = inContext.theWork() # type: ignore[attr-defined]
          File "/snap/maas/35704/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 260, in <lambda>
            inContext.theWork = lambda: context.call( # type: ignore[attr-defined]
          File "/snap/maas/35704/usr/lib/python3/dist-packages/twisted/python/context.py", line 117, in callWithContext
            return self.currentContext().callWithContext(ctx, func, *args, **kw)
          File "/snap/maas/35704/usr/lib/python3/dist-packages/twisted/python/context.py", line 82, in callWithContext
            return func(*args, **kw)
          File "/snap/maas/35704/lib/python3.10/site-packages/provisioningserver/utils/twisted.py", line 857, in callInContext
            return func(*args, **kwargs)
          File "/snap/maas/35704/lib/python3.10/site-packages/provisioningserver/utils/twisted.py", line 203, in wrapper
            result = func(*args, **kwargs)
          File "/snap/maas/35704/lib/python3.10/site-packages/maasserver/utils/orm.py", line 771, in call_within_transaction
            return func_outside_txn(*args, **kwargs)
          File "/snap/maas/35704/lib/python3.10/site-packages/maasserver/utils/orm.py", line 574, in retrier
            return func(*args, **kwargs)
          File "/usr/lib/python3.10/contextlib.py", line 79, in inner
            return func(*args, **kwds)
          File "/snap/maas/35704/lib/python3.10/site-packages/maasserver/websockets/base.py", line 437, in prep_user_execute
            return self._call_method_track_queries(
          File "/snap/maas/35704/lib/python3.10/site-packages/maasserver/websockets/base.py", line 461, in _call_method_track_queries
            result = method(params)
          File "/snap/maas/35704/lib/python3.10/site-packages/maasserver/websockets/handlers/dhcpsnippet.py", line 73, in create
            if for...