[2.4] UpdateNodePowerState RPC calls causes "Must be one of the node's interfaces."

Bug #1749979 reported by Andres Rodriguez
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
MAAS
Fix Released
Critical
Blake Rouse

Bug Description

This to me seem like performance issues with the RPC communication. MAAS has 100 machines, while commissioning about 40.

Ok, so this is interesting:

1. The machine got added but seems that it doesn't have an interface attached (e.g MAAS failed to add the interface of the machine)

2. The machine is in 'commissioning' state, and cannot be aborted with the same message.

2018-02-16 15:44:56 provisioningserver.rpc.common: [critical] Unhandled failure dispatching AMP command. This is probably a bug. Please ensure that this error is handled within application code or declared in the signature of the b'UpdateNodePowerState' com
mand. [autopkgtest:pid=11690:cmd=UpdateNodePowerState:ask=4fc]
        Traceback (most recent call last):
          File "/usr/lib/python3/dist-packages/twisted/internet/asyncioreactor.py", line 267, in run
            self._asyncioEventloop.run_forever()
          File "/usr/lib/python3/dist-packages/twisted/internet/asyncioreactor.py", line 290, in run
            f(*args, **kwargs)
          File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 500, in errback
            self._startRunCallbacks(fail)
          File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 567, in _startRunCallbacks
            self._runCallbacks()
        --- <exception caught here> ---
          File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 653, in _runCallbacks
            current.result = callback(current.result, *args, **kw)
          File "/usr/lib/python3/dist-packages/twisted/protocols/amp.py", line 1171, in checkKnownErrors
            key = error.trap(*command.allErrors)
          File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 359, in trap
            self.raiseException()
          File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 385, in raiseException
            raise self.value.with_traceback(self.tb)
          File "/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 250, in inContext
            result = inContext.theWork()
          File "/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 266, in <lambda>
            inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
          File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 122, in callWithContext
            return self.currentContext().callWithContext(ctx, func, *args, **kw)
          File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 85, in callWithContext
            return func(*args,**kw)
          File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 875, in callInContext
            return func(*args, **kwargs)
          File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 232, in wrapper
            result = func(*args, **kwargs)
          File "/usr/lib/python3/dist-packages/maasserver/utils/orm.py", line 754, in call_within_transaction
            return func_outside_txn(*args, **kwargs)
          File "/usr/lib/python3/dist-packages/maasserver/utils/orm.py", line 561, in retrier
            return func(*args, **kwargs)
          File "/usr/lib/python3.6/contextlib.py", line 52, in inner
            return func(*args, **kwds)
          File "/usr/lib/python3/dist-packages/maasserver/rpc/nodes.py", line 187, in update_node_power_state
            node.update_power_state(power_state)
          File "/usr/lib/python3/dist-packages/maasserver/utils/orm.py", line 738, in call_within_transaction
            return func_within_txn(*args, **kwargs)
          File "/usr/lib/python3.6/contextlib.py", line 52, in inner
            return func(*args, **kwds)
          File "/usr/lib/python3/dist-packages/maasserver/models/node.py", line 3078, in update_power_state
            self.save()
          File "/usr/lib/python3/dist-packages/maasserver/models/node.py", line 1636, in save
            super(Node, self).save(*args, **kwargs)
          File "/usr/lib/python3/dist-packages/maasserver/models/cleansave.py", line 28, in save
            self.full_clean()
          File "/usr/lib/python3/dist-packages/django/db/models/base.py", line 1250, in full_clean
            raise ValidationError(errors)
        django.core.exceptions.ValidationError: {'boot_interface': ["Must be one of the node's interfaces."]}

Tags: performance
Changed in maas:
milestone: none → 2.4.0alpha2
importance: Undecided → Critical
description: updated
Changed in maas:
status: New → Triaged
assignee: nobody → Blake Rouse (blake-rouse)
tags: added: performance
description: updated
description: updated
Changed in maas:
milestone: 2.4.0alpha2 → 2.4.0beta1
Revision history for this message
Blake Rouse (blake-rouse) wrote :

With the change made to only update and validate the fields that have changed, this is not longer an issue.

Changed in maas:
status: Triaged → Fix Committed
Changed in maas:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.