Internal server error when attempting to perform an action when the cluster is down

[Raphaël Badin]

When trying to deploy a node in the UI with the cluster service down, I get a "Internal server error page". I was expecting an error but the error I'm getting is not really helpful: a blank page with "Internal server error" (i.e. no indication of what the problem actually is). Also, the only error I can see is in /var/log/maas/maas-django.log (it should be in /var/log/maas/maas.log).

/var/log/maas/maas-django.log:

  File "/usr/lib/python2.7/dist-packages/maasserver/clusterrpc/dhcp.py", line 183, in remove_host_maps
    for nodegroup in removal_mappings
  File "/usr/lib/python2.7/dist-packages/maasserver/clusterrpc/dhcp.py", line 183, in <dictcomp>
    for nodegroup in removal_mappings
  File "/usr/lib/python2.7/dist-packages/provisioningserver/utils/twisted.py", line 106, in wrapper
    return func_in_reactor(*args, **kwargs).wait()
  File "/usr/lib/python2.7/dist-packages/crochet/_eventloop.py", line 219, in wait
    result.raiseException()
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 577, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "/usr/lib/python2.7/dist-packages/maasserver/rpc/regionservice.py", line 514, in cancelled
    uuid)
NoConnectionsAvailable: Unable to connect to cluster 06985604-940f-43d0-8b4c-701cc2e46c4f; no connections available.

[Raphaël Badin (rvb)]

I haven't tested it, but I suspect the same kind of error happens when calling methods from the API. Here again, a 5XX is expected but the error message should contains an explanation about the failure's cause.

[Graham Binns (gmb)]

This is something that we need to handle when sending power commands to nodes; do we have a nice, general way in mind of catching and handling this error? I'm playing pretty close to this code right now and would love to not find myself duplicating stuff.

[Raphaël Badin (rvb)]

I think using a middleware is the nicest way to catch and deal with exceptions that arise from different places in the code. We already have this in place for the API: see APIErrorsMiddleware. The same pattern can be used for non-APi exceptions although in this case we could: issue a warning using Django's message framework and then reload the page.

[Raphaël Badin (rvb)]

QAed this today: works fine. Like I said on IRC, it's a bit weird to display the cluster's uuid and not the cluster's name in the message. Plus I'd prefix the error message with "Error: " but that's just me :).