Editing/creating a cluster interface can cause an internal server error if cluster disconnected

Bug #1439476 reported by Mike Pontillo on 2015-04-02
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
MAAS
Critical
Gavin Panella

Bug Description

The following exceptions were seen in the log after doing the following:

 - Installing MAAS with a fresh database
 - Importing boot images
 - Adding a chassis with 2 nodes
 - Attempting commissioning (which fails since the cluster is unmanaged)
 *** At this point, I think a network has automatically been created,
 *** and the two nodes have been added to it
 - Editing the cluster interface to provide dynamic and static ranges,
   and change it to DNS+DHCP managed

2015-04-01 19:04:17 [-] 127.0.0.1 - - [02/Apr/2015:02:04:16 +0000] "GET
  /MAAS/clusters/992334f5-f5f0-4e61-88e9-e372ae174b1e/interfaces/eth0/edit/
  HTTP/1.1" 200 2804
  "http://maas-trusty/MAAS/clusters/992334f5-f5f0-4e61-88e9-e372ae174b1e/edit/"
  "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36
  (KHTML, like Gecko) Chrome/41.0.2272.104 Safari/537.36"
2015-04-01 19:04:29 [maas] WARNING: Failed to create Network when
  adding/editing cluster interface maas-eth0 with error [{'ip':
  [u'Network with this Ip already exists.'], 'name': [u'Network with
  this Name already exists.']}]. This is OK if it already exists.
2015-04-01 19:04:29 [HTTPChannel,3,127.0.0.1] 500 Error -
  /MAAS/clusters/992334f5-f5f0-4e61-88e9-e372ae174b1e/interfaces/eth0/edit/

Traceback (most recent call last):
  File ".../twisted/web/wsgi.py", line 315, in run
    appIterator = self.application(self.environ, self.startResponse)
  File ".../django/core/handlers/wsgi.py", line 206, in __call__
    response = self.get_response(request)
  File ".../maasserver/utils/views.py", line 166, in get_response
    response = get_response(request)
  File ".../maasserver/utils/views.py", line 155, in get_response
    request, get_resolver(None), sys.exc_info())
--- <exception caught here> ---
  File ".../maasserver/utils/views.py", line 140, in get_response
    return django_get_response(request)
  File ".../maasserver/utils/orm.py", line 331, in __exit__
    self.fire()
  File ".../provisioningserver/utils/twisted.py", line 151, in wrapper
    return func(*args, **kwargs)
  File ".../maasserver/utils/async.py", line 180, in fire
    self._fire_in_reactor(hook).wait()
  File ".../crochet/_eventloop.py", line 219, in wait
    result.raiseException()
  File "<string>", line 2, in raiseException

This causes a generic "Internal server error" to be seen when saving the
cluster interface configuration.

Tags: oil Edit Tag help

Related branches

Changed in maas:
importance: Undecided → High
description: updated
Raphaël Badin (rvb) wrote :

This is a crash, it should be critical.

Changed in maas:
importance: High → Critical
Larry Michel (lmic) wrote :

I am seeing this with 1.8b4. I noticed that the change goes through since I can get to it when go back.

tags: added: oil
Larry Michel (lmic) wrote :

Correction from my previous comment.. the change appears to go through following "internal server error" but functionality is broken.

Earlier, I was changing an interface to unmanaged to DHCP and DNS, and maas-dhcp would not start.. I just retried a second a go by changing the interface from DHCP and DNS to just DHCP and the save took without the "internal server error".. and maas-dhcp started!

Changed in maas:
status: New → Incomplete
Launchpad Janitor (janitor) wrote :

[Expired for MAAS because there has been no activity for 60 days.]

Changed in maas:
status: Incomplete → Expired
Andres Rodriguez (andreserl) wrote :

==> /var/log/maas/regiond.log <==
2015-10-21 21:52:48 [HTTPChannel,8,127.0.0.1] 500 Error - /MAAS/clusters/bd8ff259-e344-40a7-a0ad-a786712b6d83/interfaces/add/
 Traceback (most recent call last):
   File "/usr/lib/python2.7/dist-packages/twisted/web/wsgi.py", line 315, in run
     appIterator = self.application(self.environ, self.startResponse)
   File "/usr/lib/django16/django/core/handlers/wsgi.py", line 206, in __call__
     response = self.get_response(request)
   File "/usr/lib/python2.7/dist-packages/maasserver/utils/views.py", line 236, in get_response
     response = get_response(request)
   File "/usr/lib/python2.7/dist-packages/maasserver/utils/views.py", line 225, in get_response
     request, get_resolver(None), sys.exc_info())
 --- <exception caught here> ---
   File "/usr/lib/python2.7/dist-packages/maasserver/utils/views.py", line 210, in get_response
     return django_get_response(request)
   File "/usr/lib/python2.7/dist-packages/maasserver/utils/orm.py", line 357, in __exit__
     self.fire()
   File "/usr/lib/python2.7/dist-packages/provisioningserver/utils/twisted.py", line 200, in wrapper
     return func(*args, **kwargs)
   File "/usr/lib/python2.7/dist-packages/maasserver/utils/async.py", line 203, in fire
     self._fire_in_reactor(hook).wait()
   File "/usr/lib/python2.7/dist-packages/crochet/_eventloop.py", line 219, in wait
     result.raiseException()
   File "<string>", line 2, in raiseException

 provisioningserver.rpc.exceptions.NoConnectionsAvailable: Unable to connect to cluster bd8ff259-e344-40a7-a0ad-a786712b6d83; no connections available.

summary: - Editing a cluster interface can cause an internal server error
+ Editing/creating a cluster interface can cause an internal server error
+ if cluster disconnected
Changed in maas:
status: Expired → New
milestone: none → 1.9.0
Gavin Panella (allenap) on 2015-10-27
Changed in maas:
status: New → Triaged
assignee: nobody → Gavin Panella (allenap)
status: Triaged → In Progress
description: updated
Mike Pontillo (mpontillo) wrote :

With Gavin's help, today we looked into possibilities regarding how this might occur.

I think it's likely that dhcp_post_change_Subnet() (maasserver/models/signals/dhcp.py) is the cause of this bug.

I'm still confused about the stack traces in this bug, though. I think we need to prioritize cleaning up the stack traces we get from Twisted so that we can track these down easier...

Gavin Panella (allenap) on 2015-11-10
Changed in maas:
status: In Progress → Fix Committed
Changed in maas:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers