Restarting maas-regiond and maas-clusterd fills the logs with stacktraces

Bug #1457788 reported by Raphaël Badin on 2015-05-22
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
MAAS
High
Mike Pontillo

Bug Description

When restarting the two main MAAS services (maas-regiond and maas-clusterd), a bunch of stacktraces are printed in the logs. Ultimately, the services start okay.

service maas-regiond restart && service maas-clusterd restart

Most of these stacktraces seems to be caused by races in how the various services are started and although it is not critical, it's really painful to see that MAAS doesn't deal more gracefully with this.

This is the combination of both regiond.log and clusterd.log: http://paste.ubuntu.com/11280700/

Raphaël Badin (rvb) wrote :
Download full text (7.6 KiB)

The errors I see in there:

Error #1
=======

2015-05-22 08:39:23+0000 [HTTPPageGetter,client] Unhandled Error
 Traceback (most recent call last):
 Failure: twisted.web.error.Error: 503 Service Unavailable

2015-05-22 08:39:25+0000 [HTTPPageGetter,client] Unhandled Error
 Traceback (most recent call last):
 Failure: twisted.web.error.Error: 503 Service Unavailable

Guessing this is the cluster trying to connect to the region before it's fully up.

Error #2
=======

2015-05-22 08:39:28+0000 [-] Downloading images failed.
 Traceback (most recent call last):
   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 423, in errback
     self._startRunCallbacks(fail)
   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 490, in _startRunCallbacks
     self._runCallbacks()
   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 577, in _runCallbacks
     current.result = callback(current.result, *args, **kw)
   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1155, in gotResult
     _inlineCallbacks(r, g, deferred)
 --- <exception caught here> ---
   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1097, in _inlineCallbacks
     result = result.throwExceptionIntoGenerator(g)
   File "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator
     return g.throw(self.type, self.value, self.tb)
   File "/usr/lib/python2.7/dist-packages/provisioningserver/pserv_services/image_download_service.py", line 135, in maybe_start_download
     yield self._start_download()
   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1097, in _inlineCallbacks
     result = result.throwExceptionIntoGenerator(g)
   File "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator
     return g.throw(self.type, self.value, self.tb)
   File "/usr/lib/python2.7/dist-packages/provisioningserver/pserv_services/image_download_service.py", line 121, in _start_download
     get_proxy_url("https"))
   File "/usr/lib/python2.7/dist-packages/twisted/python/threadpool.py", line 191, in _worker
     result = context.call(ctx, function, *args, **kwargs)
   File "/usr/lib/python2.7/dist-packages/twisted/python/context.py", line 118, in callWithContext
     return self.currentContext().callWithContext(ctx, func, *args, **kw)
   File "/usr/lib/python2.7/dist-packages/twisted/python/context.py", line 81, in callWithContext
     return func(*args,**kw)
   File "/usr/lib/python2.7/dist-packages/provisioningserver/utils/twisted.py", line 154, in wrapper
     return func(*args, **kwargs)
   File "/usr/lib/python2.7/dist-packages/provisioningserver/rpc/boot_images.py", line 87, in _run_import
     boot_resources.import_images(sources)
   File "/usr/lib/python2.7/dist-packages/provisioningserver/import_images/boot_resources.py", line 246, in import_images
     image_descriptions = download_all_image_descriptions(sources)
   File "/usr/lib/python2.7/dist-packages/provisioningserver/import_images/download_descriptions.py", line 204, in download_all_image_descriptions
     source['url'], ke...

Read more...

Changed in maas:
milestone: none → 1.8.0
assignee: nobody → Mike Pontillo (mpontillo)
Mike Pontillo (mpontillo) wrote :

Split this bug out into bug #1458894 and bug #1458895.

Error #1 will be addressed separately.

Mike Pontillo (mpontillo) wrote :

Filed bug #1459380 as well for Error #1.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers