MAAS

  1. Bug #1640300
  2. Activity log

Activity log for bug #1640300

Date Who What changed Old value New value Message
2016-11-08 20:55:46 Nathaniel W. Turner bug added bug
2016-11-08 20:56:19 Nathaniel W. Turner description With no obvious preceding event, am now seeing this error in the logs (and MAAS UI is showing Internal server error for all requests): Nov 8 07:24:56 maas1 sh[17635]: 2016-11-08 07:24:56 [maas.websocket.listener] Listening for database notifications. Nov 8 07:24:56 maas1 sh[17635]: 2016-11-08 07:24:56 [HTTPChannel,14914,::1] 500 Error - /MAAS/rpc/ Nov 8 07:24:56 maas1 sh[17635]: #011Traceback (most recent call last): Nov 8 07:24:56 maas1 sh[17635]: #011 File "/usr/lib/python3/dist-packages/twisted/web/wsgi.py", line 496, in run Nov 8 07:24:56 maas1 sh[17635]: #011 appIterator = self.application(self.environ, self.startResponse) Nov 8 07:24:56 maas1 sh[17635]: #011 File "/usr/lib/python3/dist-packages/django/core/handlers/wsgi.py", line 189, in __call__ Nov 8 07:24:56 maas1 sh[17635]: #011 response = self.get_response(request) Nov 8 07:24:56 maas1 sh[17635]: #011 File "/usr/lib/python3/dist-packages/maasserver/utils/views.py", line 227, in get_response Nov 8 07:24:56 maas1 sh[17635]: #011 response = get_response(request) Nov 8 07:24:56 maas1 sh[17635]: #011 File "/usr/lib/python3/dist-packages/maasserver/utils/views.py", line 216, in get_response Nov 8 07:24:56 maas1 sh[17635]: #011 request, get_resolver(None), sys.exc_info()) Nov 8 07:24:56 maas1 sh[17635]: #011--- <exception caught here> --- Nov 8 07:24:56 maas1 sh[17635]: #011 File "/usr/lib/python3/dist-packages/maasserver/utils/views.py", line 200, in get_response Nov 8 07:24:56 maas1 sh[17635]: #011 with transaction.atomic(): Nov 8 07:24:56 maas1 sh[17635]: #011 File "/usr/lib/python3/dist-packages/django/db/transaction.py", line 150, in __enter__ Nov 8 07:24:56 maas1 sh[17635]: #011 if not connection.get_autocommit(): Nov 8 07:24:56 maas1 sh[17635]: #011 File "/usr/lib/python3/dist-packages/django/db/backends/base/base.py", line 286, in get_autocommit Nov 8 07:24:56 maas1 sh[17635]: #011 self.ensure_connection() Nov 8 07:24:56 maas1 sh[17635]: #011 File "/usr/lib/python3/dist-packages/django/db/backends/base/base.py", line 130, in ensure_connection Nov 8 07:24:56 maas1 sh[17635]: #011 self.connect() Nov 8 07:24:56 maas1 sh[17635]: #011 File "/usr/lib/python3/dist-packages/django/db/utils.py", line 98, in __exit__ Nov 8 07:24:56 maas1 sh[17635]: #011 six.reraise(dj_exc_type, dj_exc_value, traceback) Nov 8 07:24:56 maas1 sh[17635]: #011 File "/usr/lib/python3/dist-packages/django/utils/six.py", line 685, in reraise Nov 8 07:24:56 maas1 sh[17635]: #011 raise value.with_traceback(tb) Nov 8 07:24:56 maas1 sh[17635]: #011 File "/usr/lib/python3/dist-packages/django/db/backends/base/base.py", line 130, in ensure_connection Nov 8 07:24:56 maas1 sh[17635]: #011 self.connect() Nov 8 07:24:56 maas1 sh[17635]: #011 File "/usr/lib/python3/dist-packages/django/db/backends/base/base.py", line 119, in connect Nov 8 07:24:56 maas1 sh[17635]: #011 self.connection = self.get_new_connection(conn_params) Nov 8 07:24:56 maas1 sh[17635]: #011 File "/usr/lib/python3/dist-packages/django/db/backends/postgresql_psycopg2/base.py", line 176, in get_new_connection Nov 8 07:24:56 maas1 sh[17635]: #011 connection = Database.connect(**conn_params) Nov 8 07:24:56 maas1 sh[17635]: #011 File "/usr/lib/python3/dist-packages/psycopg2/__init__.py", line 164, in connect Nov 8 07:24:56 maas1 sh[17635]: #011 conn = _connect(dsn, connection_factory=connection_factory, async=async) Nov 8 07:24:56 maas1 sh[17635]: #011django.db.utils.OperationalError: FATAL: remaining connection slots are reserved for non-replication superuser connections Nov 8 07:24:56 maas1 sh[17635]: #011 Nov 8 07:24:56 maas1 sh[17635]: #011 With no obvious preceding event, am now seeing this error in the logs (and MAAS UI is showing Internal server error for all requests): Nov 8 07:24:56 maas1 sh[17635]: 2016-11-08 07:24:56 [maas.websocket.listener] Listening for database notifications. Nov 8 07:24:56 maas1 sh[17635]: 2016-11-08 07:24:56 [HTTPChannel,14914,::1] 500 Error - /MAAS/rpc/ Nov 8 07:24:56 maas1 sh[17635]: #011Traceback (most recent call last): Nov 8 07:24:56 maas1 sh[17635]: #011 File "/usr/lib/python3/dist-packages/twisted/web/wsgi.py", line 496, in run Nov 8 07:24:56 maas1 sh[17635]: #011 appIterator = self.application(self.environ, self.startResponse) Nov 8 07:24:56 maas1 sh[17635]: #011 File "/usr/lib/python3/dist-packages/django/core/handlers/wsgi.py", line 189, in __call__ Nov 8 07:24:56 maas1 sh[17635]: #011 response = self.get_response(request) Nov 8 07:24:56 maas1 sh[17635]: #011 File "/usr/lib/python3/dist-packages/maasserver/utils/views.py", line 227, in get_response Nov 8 07:24:56 maas1 sh[17635]: #011 response = get_response(request) Nov 8 07:24:56 maas1 sh[17635]: #011 File "/usr/lib/python3/dist-packages/maasserver/utils/views.py", line 216, in get_response Nov 8 07:24:56 maas1 sh[17635]: #011 request, get_resolver(None), sys.exc_info()) Nov 8 07:24:56 maas1 sh[17635]: #011--- <exception caught here> --- Nov 8 07:24:56 maas1 sh[17635]: #011 File "/usr/lib/python3/dist-packages/maasserver/utils/views.py", line 200, in get_response Nov 8 07:24:56 maas1 sh[17635]: #011 with transaction.atomic(): Nov 8 07:24:56 maas1 sh[17635]: #011 File "/usr/lib/python3/dist-packages/django/db/transaction.py", line 150, in __enter__ Nov 8 07:24:56 maas1 sh[17635]: #011 if not connection.get_autocommit(): Nov 8 07:24:56 maas1 sh[17635]: #011 File "/usr/lib/python3/dist-packages/django/db/backends/base/base.py", line 286, in get_autocommit Nov 8 07:24:56 maas1 sh[17635]: #011 self.ensure_connection() Nov 8 07:24:56 maas1 sh[17635]: #011 File "/usr/lib/python3/dist-packages/django/db/backends/base/base.py", line 130, in ensure_connection Nov 8 07:24:56 maas1 sh[17635]: #011 self.connect() Nov 8 07:24:56 maas1 sh[17635]: #011 File "/usr/lib/python3/dist-packages/django/db/utils.py", line 98, in __exit__ Nov 8 07:24:56 maas1 sh[17635]: #011 six.reraise(dj_exc_type, dj_exc_value, traceback) Nov 8 07:24:56 maas1 sh[17635]: #011 File "/usr/lib/python3/dist-packages/django/utils/six.py", line 685, in reraise Nov 8 07:24:56 maas1 sh[17635]: #011 raise value.with_traceback(tb) Nov 8 07:24:56 maas1 sh[17635]: #011 File "/usr/lib/python3/dist-packages/django/db/backends/base/base.py", line 130, in ensure_connection Nov 8 07:24:56 maas1 sh[17635]: #011 self.connect() Nov 8 07:24:56 maas1 sh[17635]: #011 File "/usr/lib/python3/dist-packages/django/db/backends/base/base.py", line 119, in connect Nov 8 07:24:56 maas1 sh[17635]: #011 self.connection = self.get_new_connection(conn_params) Nov 8 07:24:56 maas1 sh[17635]: #011 File "/usr/lib/python3/dist-packages/django/db/backends/postgresql_psycopg2/base.py", line 176, in get_new_connection Nov 8 07:24:56 maas1 sh[17635]: #011 connection = Database.connect(**conn_params) Nov 8 07:24:56 maas1 sh[17635]: #011 File "/usr/lib/python3/dist-packages/psycopg2/__init__.py", line 164, in connect Nov 8 07:24:56 maas1 sh[17635]: #011 conn = _connect(dsn, connection_factory=connection_factory, async=async) Nov 8 07:24:56 maas1 sh[17635]: #011django.db.utils.OperationalError: FATAL: remaining connection slots are reserved for non-replication superuser connections Nov 8 07:24:56 maas1 sh[17635]: #011 Nov 8 07:24:56 maas1 sh[17635]: #011 Some kind of leak, maybe? Versions: maas 2.1.0+bzr5480-0ubuntu
2016-11-08 21:06:29 Nathaniel W. Turner attachment added contents of /var/log/postgresql/ https://bugs.launchpad.net/maas/+bug/1640300/+attachment/4774558/+files/postgresql-logs.tar.gz
2016-11-08 21:17:11 Nathaniel W. Turner attachment added /var/log/maas https://bugs.launchpad.net/maas/+bug/1640300/+attachment/4774559/+files/maas-logs.tar.gz
2016-11-09 08:07:45 Andres Rodriguez nominated for series maas/trunk
2016-11-09 08:07:45 Andres Rodriguez bug task added maas/trunk
2016-11-09 08:07:45 Andres Rodriguez nominated for series maas/2.1
2016-11-09 08:07:45 Andres Rodriguez bug task added maas/2.1
2016-11-09 08:07:54 Andres Rodriguez maas/2.1: milestone 2.1.2
2016-11-09 08:07:56 Andres Rodriguez maas/trunk: milestone 2.2.0
2016-11-10 06:47:35 Launchpad Janitor branch linked lp:~blake-rouse/maas/fix-1640300
2016-11-10 06:48:46 Blake Rouse maas/trunk: status New Triaged
2016-11-10 06:48:51 Blake Rouse maas/2.1: status New Triaged
2016-11-10 06:49:03 Blake Rouse maas/trunk: status Triaged In Progress
2016-11-10 06:49:05 Blake Rouse maas/trunk: importance Undecided Critical
2016-11-10 06:49:07 Blake Rouse maas/2.1: importance Undecided Critical
2016-11-10 06:49:10 Blake Rouse maas/trunk: assignee Blake Rouse (blake-rouse)
2016-11-10 21:02:48 MAAS Lander maas/trunk: status In Progress Fix Committed
2016-11-15 15:17:27 Launchpad Janitor branch linked lp:~blake-rouse/maas/fix-1640300-2.1
2016-11-15 15:23:19 Blake Rouse maas/2.1: status Triaged In Progress
2016-11-15 15:23:21 Blake Rouse maas/2.1: assignee Blake Rouse (blake-rouse)
2016-11-17 16:38:29 MAAS Lander maas/2.1: status In Progress Fix Committed
2016-11-23 13:51:33 Andres Rodriguez maas/2.1: status Fix Committed Fix Released
2016-11-24 03:42:08 Andres Rodriguez maas/trunk: milestone 2.2.0
2016-11-24 03:42:38 Andres Rodriguez maas/trunk: status Fix Committed Fix Released