[2.2] Failed to process node status messages - twisted.internet.defer.QueueOverflow

Bug #1718044 reported by Andres Rodriguez
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
MAAS
Fix Released
Critical
Blake Rouse

Bug Description

2017-09-18 17:22:20 regiond: [info] 10.245.208.32 GET /MAAS/api/2.0/version/ HTTP/1.1 --> 200 OK (referrer: -; agent: Go-http-client/1.1)
2017-09-18 17:22:20 metadataserver.api_twisted: [critical] Failed to process node status messages.

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/twisted/internet/base.py", line 1203, in mainLoop
    self.runUntilCurrent()
  File "/usr/lib/python3/dist-packages/twisted/internet/base.py", line 798, in runUntilCurrent
    f(*a, **kw)
  File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 393, in callback
    self._startRunCallbacks(result)
  File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 501, in _startRunCallbacks
    self._runCallbacks()
--- <exception caught here> ---
  File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 588, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "/usr/lib/python3/dist-packages/metadataserver/api_twisted.py", line 224, in _processMessagesLater
    self.dbtasks.addTask(self._processMessages, node, messages)
  File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 160, in wrapper
    return func(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/maasserver/utils/dbtasks.py", line 104, in addTask
    done = self.deferTask(func, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 160, in wrapper
    return func(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/maasserver/utils/dbtasks.py", line 91, in deferTask
    self.queue.put(task)
  File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1519, in put
    raise QueueOverflow()
twisted.internet.defer.QueueOverflow:
2017-09-18 17:22:21 metadataserver.api_twisted: [critical] Failed to process node status messages.

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/twisted/internet/base.py", line 1203, in mainLoop
    self.runUntilCurrent()
  File "/usr/lib/python3/dist-packages/twisted/internet/base.py", line 798, in runUntilCurrent
    f(*a, **kw)
  File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 393, in callback
    self._startRunCallbacks(result)
  File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 501, in _startRunCallbacks
    self._runCallbacks()
--- <exception caught here> ---
  File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 588, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "/usr/lib/python3/dist-packages/metadataserver/api_twisted.py", line 224, in _processMessagesLater
    self.dbtasks.addTask(self._processMessages, node, messages)
  File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 160, in wrapper
    return func(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/maasserver/utils/dbtasks.py", line 104, in addTask
    done = self.deferTask(func, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 160, in wrapper
    return func(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/maasserver/utils/dbtasks.py", line 91, in deferTask
    self.queue.put(task)
  File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1519, in put
    raise QueueOverflow()
twisted.internet.defer.QueueOverflow:
2017-09-18 17:22:21 metadataserver.api_twisted: [critical] Failed to process node status messages.

Related branches

Changed in maas:
milestone: none → 2.3.0
importance: Undecided → Critical
tags: added: cdo-qa foundations-engine
Changed in maas:
milestone: 2.3.0 → 2.3.0beta2
Changed in maas:
status: New → Triaged
tags: added: internal
Changed in maas:
milestone: 2.3.0beta2 → 2.3.0beta3
Changed in maas:
assignee: nobody → Blake Rouse (blake-rouse)
Changed in maas:
milestone: 2.3.0beta3 → 2.3.0beta4
Changed in maas:
milestone: 2.3.0beta4 → 2.3.0rc1
Changed in maas:
status: Triaged → In Progress
Changed in maas:
status: In Progress → Fix Committed
Changed in maas:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.