We discovered that network discovery was active and there was a list of some hundreds of machines in the discovered list. Following the guidance of the team I disabled discover and CPU usage dropped and API calls were able to complete without any timeout.
Now approximately 10 days later we have started getting API calls timing out again and regiond processes that appear to be CPU bound. I have restarted the machine and which it is running but a regiond process returns at 100% CPU and has not settled down.
The MAAS instance is installed in a lxd container (not VM) running on a 22.04 host. MAAS was installed from deb packages which are currently of these versions:
ubuntu@cert-maas-tel-01:~$ dpkg -l | grep maas
ii maas 1:3.2.11-12079-g.8ca75dae5-0ubuntu1~20.04.1 all "Metal as a Service" is a physical cloud and IPAM
ii maas-cli 1:3.2.11-12079-g.8ca75dae5-0ubuntu1~20.04.1 all MAAS client and command-line interface
ii maas-common 1:3.2.11-12079-g.8ca75dae5-0ubuntu1~20.04.1 all MAAS server common files
ii maas-dhcp 1:3.2.11-12079-g.8ca75dae5-0ubuntu1~20.04.1 all MAAS DHCP server
ii maas-proxy 1:3.2.11-12079-g.8ca75dae5-0ubuntu1~20.04.1 all MAAS Caching Proxy
ii maas-rack-controller 1:3.2.11-12079-g.8ca75dae5-0ubuntu1~20.04.1 all Rack Controller for MAAS
ii maas-region-api 1:3.2.11-12079-g.8ca75dae5-0ubuntu1~20.04.1 all Region controller API service for MAAS
ii maas-region-controller 1:3.2.11-12079-g.8ca75dae5-0ubuntu1~20.04.1 all Region Controller for MAAS
ii python3-django-maas 1:3.2.11-12079-g.8ca75dae5-0ubuntu1~20.04.1 all MAAS server Django web framework (Python 3)
ii python3-maas-client 1:3.2.11-12079-g.8ca75dae5-0ubuntu1~20.04.1 all MAAS python API client (Python 3)
ii python3-maas-provisioningserver 1:3.2.11-12079-g.8ca75dae5-0ubuntu1~20.04.1 all MAAS server provisioning libraries (Python 3)
The region controller used in the Devices Certification lab has also been experiencing these symptom recently.
On the first occasion that I encountered it I raised the issue via MatterMost link this bug. https:/ /chat.canonical .com/canonical/ pl/k3j4ewxe83b4 uer66nctnr1rko
We discovered that network discovery was active and there was a list of some hundreds of machines in the discovered list. Following the guidance of the team I disabled discover and CPU usage dropped and API calls were able to complete without any timeout.
Now approximately 10 days later we have started getting API calls timing out again and regiond processes that appear to be CPU bound. I have restarted the machine and which it is running but a regiond process returns at 100% CPU and has not settled down.
The MAAS instance is installed in a lxd container (not VM) running on a 22.04 host. MAAS was installed from deb packages which are currently of these versions:
ubuntu@ cert-maas- tel-01: ~$ dpkg -l | grep maas 12079-g. 8ca75dae5- 0ubuntu1~ 20.04.1 all "Metal as a Service" is a physical cloud and IPAM 12079-g. 8ca75dae5- 0ubuntu1~ 20.04.1 all MAAS client and command-line interface 12079-g. 8ca75dae5- 0ubuntu1~ 20.04.1 all MAAS server common files 12079-g. 8ca75dae5- 0ubuntu1~ 20.04.1 all MAAS DHCP server 12079-g. 8ca75dae5- 0ubuntu1~ 20.04.1 all MAAS Caching Proxy controller 1:3.2.11- 12079-g. 8ca75dae5- 0ubuntu1~ 20.04.1 all Rack Controller for MAAS 12079-g. 8ca75dae5- 0ubuntu1~ 20.04.1 all Region controller API service for MAAS controller 1:3.2.11- 12079-g. 8ca75dae5- 0ubuntu1~ 20.04.1 all Region Controller for MAAS 12079-g. 8ca75dae5- 0ubuntu1~ 20.04.1 all MAAS server Django web framework (Python 3) 12079-g. 8ca75dae5- 0ubuntu1~ 20.04.1 all MAAS python API client (Python 3) maas-provisioni ngserver 1:3.2.11- 12079-g. 8ca75dae5- 0ubuntu1~ 20.04.1 all MAAS server provisioning libraries (Python 3)
ii maas 1:3.2.11-
ii maas-cli 1:3.2.11-
ii maas-common 1:3.2.11-
ii maas-dhcp 1:3.2.11-
ii maas-proxy 1:3.2.11-
ii maas-rack-
ii maas-region-api 1:3.2.11-
ii maas-region-
ii python3-django-maas 1:3.2.11-
ii python3-maas-client 1:3.2.11-
ii python3-