2016-10-06 12:41:20 ERROR [7fdded724880] (base) Unexpected exception occured
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/nailgun/api/v1/handlers/base.py", line 297, in handle_errors
return func(cls, *args, **kwargs)
File "<string>", line 2, in GET
File "/usr/lib/python2.7/site-packages/nailgun/api/v1/handlers/base.py", line 355, in validate
return func(cls, *args, **kwargs)
File "<string>", line 2, in GET
File "/usr/lib/python2.7/site-packages/nailgun/api/v1/handlers/base.py", line 381, in serialize
resp = func(cls, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/nailgun/api/v1/handlers/base.py", line 462, in GET
return self.collection.to_list(q)
File "/usr/lib/python2.7/site-packages/nailgun/objects/base.py", line 424, in to_list
use_iterable
TypeError: argument 2 to map() must support iteration
[pid: 20677|app: 0|req: 8/35] 10.109.0.1 () {50 vars in 850 bytes} [Thu Oct 6 12:41:20 2016] GET /api/notifications?_=1475756327171 => generated 39 bytes in 3 msecs (HTTP/1.1 500) 5 headers in 223 bytes (2 switches on core 0)
[pid: 20680|app: 0|req: 15/36] 10.109.0.1 () {50 vars in 868 bytes} [Thu Oct 6 12:41:20 2016] GET /api/nodes/allocation/stats?_=1475756327170 => generated 30 bytes in 18 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
SIGINT/SIGQUIT received...killing workers...
...brutally killing workers...
mule 1 (pid: 20681) annihilated
worker 1 buried after 1 seconds
worker 2 buried after 1 seconds
worker 3 buried after 1 seconds
worker 4 buried after 1 seconds
goodbye to uWSGI.
*** Starting uWSGI 2.0.12 (64bit) on [Thu Oct 6 12:41:29 2016] ***
compiled with version: 4.8.3 20140911 (Red Hat 4.8.3-9) on 29 April 2016 14:51:44
os: Linux-3.10.0-327.18.2.el7.x86_64 #1 SMP Thu May 12 11:03:55 UTC 2016
nodename: nailgun.test.domain.local
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 2
current working directory: /
writing pidfile to /var/run/nailgun.pid
detected binary path: /usr/sbin/uwsgi
your processes number limit is 11302
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to TCP address :8001 fd 3
uWSGI running as root, you can use --uid/--gid/--chroot options
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
Python version: 2.7.5 (default, Nov 20 2015, 02:00:19) [GCC 4.8.5 20150623 (Red Hat 4.8.5-4)]
Set PythonHome to /usr
Python main interpreter initialized at 0xe824d0
python threads support enabled
your server socket listen backlog is limited to 4096 connections
your mercy for graceful operations on workers is 60 seconds
mapped 589260 bytes (575 KB) for 4 cores
*** Operational MODE: preforking ***
added /usr/lib/python2.7/site-packages/nailgun/ to pythonpath.
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 29547)
spawned uWSGI worker 1 (pid: 29555, cores: 1)
spawned uWSGI worker 2 (pid: 29556, cores: 1)
spawned uWSGI worker 3 (pid: 29557, cores: 1)
spawned uWSGI worker 4 (pid: 29558, cores: 1)
spawned uWSGI mule 1 (pid: 29559)
2016-10-06 12:40:59 ERROR [7fdded724880] (base) Unexpected exception occured
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/nailgun/api/v1/handlers/base.py", line 297, in handle_errors
return func(cls, *args, **kwargs)
File "<string>", line 2, in GET
File "/usr/lib/python2.7/site-packages/nailgun/api/v1/handlers/base.py", line 355, in validate
return func(cls, *args, **kwargs)
File "<string>", line 2, in GET
File "/usr/lib/python2.7/site-packages/nailgun/api/v1/handlers/base.py", line 381, in serialize
resp = func(cls, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/nailgun/api/v1/handlers/node.py", line 344, in GET
unallocated_nodes = db().query(Node).filter_by(cluster_id=None).count()
File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2955, in count
return self.from_self(col).scalar()
File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2724, in scalar
ret = self.one()
File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2693, in one
ret = list(self)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2736, in __iter__
return self._execute_and_instances(context)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2751, in _execute_and_instances
result = conn.execute(querycontext.statement, self._params)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 914, in execute
return meth(self, multiparams, params)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/sql/elements.py", line 323, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1010, in _execute_clauseelement
compiled_sql, distilled_params
File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1146, in _execute_context
context)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1341, in _handle_dbapi_exception
exc_info
File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/compat.py", line 200, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
context)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/default.py", line 450, in do_execute
cursor.execute(statement, parameters)
OperationalError: (psycopg2.OperationalError) terminating connection due to administrator command
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
[SQL: 'SELECT count(*) AS count_1 \nFROM (SELECT nodes.id AS nodes_id, nodes.uuid AS nodes_uuid, nodes.cluster_id AS nodes_cluster_id, nodes.group_id AS nodes_group_id, nodes.name AS nodes_name, nodes.status AS nodes_status, nodes.meta AS nodes_meta, nodes.mac AS nodes_mac, nodes.ip AS nodes_ip, nodes.hostname AS nodes_hostname, nodes.manufacturer AS nodes_manufacturer, nodes.platform_name AS nodes_platform_name, nodes.kernel_params AS nodes_kernel_params, nodes.progress AS nodes_progress, nodes.os_platform AS nodes_os_platform, nodes.pending_addition AS nodes_pending_addition, nodes.pending_deletion AS nodes_pending_deletion, nodes.error_type AS nodes_error_type, nodes.error_msg AS nodes_error_msg, nodes.timestamp AS nodes_timestamp, nodes.online AS nodes_online, nodes.labels AS nodes_labels, nodes.roles AS nodes_roles, nodes.pending_roles AS nodes_pending_roles, nodes.primary_roles AS nodes_primary_roles, nodes.agent_checksum AS nodes_agent_checksum, nodes.replaced_deployment_info AS nodes_replaced_deployment_info, nodes.replaced_provisioning_info AS nodes_replaced_provisioning_info, nodes.network_template AS nodes_network_template, nodes.extensions AS nodes_extensions, nodes.vms_conf AS nodes_vms_conf, nodes.attributes AS nodes_attributes \nFROM nodes \nWHERE nodes.cluster_id IS NULL) AS anon_1']
[pid: 20680|app: 0|req: 14/34] 10.109.0.1 () {50 vars in 868 bytes} [Thu Oct 6 12:40:59 2016] GET /api/nodes/allocation/stats?_=1475756327168 => generated 39 bytes in 10 msecs (HTTP/1.1 500) 5 headers in 223 bytes (2 switches on core 0)
2016-10-06 12:40:59 ERROR [7fdded724880] (base) Unexpected exception occured
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/nailgun/api/v1/handlers/base.py", line 297, in handle_errors
return func(cls, *args, **kwargs)
File "<string>", line 2, in GET
File "/usr/lib/python2.7/site-packages/nailgun/api/v1/handlers/base.py", line 355, in validate
return func(cls, *args, **kwargs)
File "<string>", line 2, in GET
File "/usr/lib/python2.7/site-packages/nailgun/api/v1/handlers/base.py", line 381, in serialize
resp = func(cls, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/nailgun/api/v1/handlers/base.py", line 462, in GET
return self.collection.to_list(q)
File "/usr/lib/python2.7/site-packages/nailgun/objects/base.py", line 424, in to_list
use_iterable
TypeError: argument 2 to map() must support iteration
[pid: 20678|app: 0|req: 6/33] 10.109.0.1 () {50 vars in 850 bytes} [Thu Oct 6 12:40:59 2016] GET /api/notifications?_=1475756327169 => generated 39 bytes in 3 msecs (HTTP/1.1 500) 5 headers in 223 bytes (2 switches on core 0)
UI logs:
2016-10-06 12:41:20 ERROR [7fdded724880] (base) Unexpected exception occured python2. 7/site- packages/ nailgun/ api/v1/ handlers/ base.py" , line 297, in handle_errors python2. 7/site- packages/ nailgun/ api/v1/ handlers/ base.py" , line 355, in validate python2. 7/site- packages/ nailgun/ api/v1/ handlers/ base.py" , line 381, in serialize python2. 7/site- packages/ nailgun/ api/v1/ handlers/ base.py" , line 462, in GET .to_list( q) python2. 7/site- packages/ nailgun/ objects/ base.py" , line 424, in to_list ons?_=147575632 7171 => generated 39 bytes in 3 msecs (HTTP/1.1 500) 5 headers in 223 bytes (2 switches on core 0) allocation/ stats?_ =1475756327170 => generated 30 bytes in 18 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0) 10.0-327. 18.2.el7. x86_64 #1 SMP Thu May 12 11:03:55 UTC 2016 test.domain. local nailgun. pid -gid/-- chroot options python2. 7/site- packages/ nailgun/ to pythonpath. python2. 7/site- packages/ nailgun/ api/v1/ handlers/ base.py" , line 297, in handle_errors python2. 7/site- packages/ nailgun/ api/v1/ handlers/ base.py" , line 355, in validate python2. 7/site- packages/ nailgun/ api/v1/ handlers/ base.py" , line 381, in serialize python2. 7/site- packages/ nailgun/ api/v1/ handlers/ node.py" , line 344, in GET _nodes = db().query( Node).filter_ by(cluster_ id=None) .count( ) python2. 7/site- packages/ sqlalchemy/ orm/query. py", line 2955, in count self(col) .scalar( ) python2. 7/site- packages/ sqlalchemy/ orm/query. py", line 2724, in scalar python2. 7/site- packages/ sqlalchemy/ orm/query. py", line 2693, in one python2. 7/site- packages/ sqlalchemy/ orm/query. py", line 2736, in __iter__ and_instances( context) python2. 7/site- packages/ sqlalchemy/ orm/query. py", line 2751, in _execute_ and_instances querycontext. statement, self._params) python2. 7/site- packages/ sqlalchemy/ engine/ base.py" , line 914, in execute python2. 7/site- packages/ sqlalchemy/ sql/elements. py", line 323, in _execute_ on_connection _execute_ clauseelement( self, multiparams, params) python2. 7/site- packages/ sqlalchemy/ engine/ base.py" , line 1010, in _execute_ clauseelement python2. 7/site- packages/ sqlalchemy/ engine/ base.py" , line 1146, in _execute_context python2. 7/site- packages/ sqlalchemy/ engine/ base.py" , line 1341, in _handle_ dbapi_exception python2. 7/site- packages/ sqlalchemy/ util/compat. py", line 200, in raise_from_cause type(exception) , exception, tb=exc_tb, cause=cause) python2. 7/site- packages/ sqlalchemy/ engine/ base.py" , line 1139, in _execute_context python2. 7/site- packages/ sqlalchemy/ engine/ default. py", line 450, in do_execute execute( statement, parameters) OperationalErro r) terminating connection due to administrator command name, nodes.kernel_params AS nodes_kernel_ params, nodes.progress AS nodes_progress, nodes.os_platform AS nodes_os_platform, nodes.pending_ addition AS nodes_pending_ addition, nodes.pending_ deletion AS nodes_pending_ deletion, nodes.error_type AS nodes_error_type, nodes.error_msg AS nodes_error_msg, nodes.timestamp AS nodes_timestamp, nodes.online AS nodes_online, nodes.labels AS nodes_labels, nodes.roles AS nodes_roles, nodes.pending_roles AS nodes_pending_ roles, nodes.primary_roles AS nodes_primary_ roles, nodes.agent_ checksum AS nodes_agent_ checksum, nodes.replaced_ deployment_ info AS nodes_replaced_ deployment_ info, nodes.replaced_ provisioning_ info AS nodes_replaced_ provisioning_ info, nodes.network_ template AS nodes_network_ template, nodes.extensions AS nodes_extensions, nodes.vms_conf AS nodes_vms_conf, nodes.attributes AS nodes_attributes \nFROM nodes \nWHERE nodes.cluster_id IS NULL) AS anon_1'] allocation/ stats?_ =1475756327168 => generated 39 bytes in 10 msecs (HTTP/1.1 500) 5 headers in 223 bytes (2 switches on core 0) python2. 7/site- packages/ nailgun/ api/v1/ handlers/ base.py" , line 297, in handle_errors python2. 7/site- packages/ nailgun/ api/v1/ handlers/ base.py" , line 355, in validate python2. 7/site- packages/ nailgun/ api/v1/ handlers/ base.py" , line 381, in serialize python2. 7/site- packages/ nailgun/ api/v1/ handlers/ base.py" , line 462, in GET .to_list( q) python2. 7/site- packages/ nailgun/ objects/ base.py" , line 424, in to_list ons?_=147575632 7169 => generated 39 bytes in 3 msecs (HTTP/1.1 500) 5 headers in 223 bytes (2 switches on core 0)
Traceback (most recent call last):
File "/usr/lib/
return func(cls, *args, **kwargs)
File "<string>", line 2, in GET
File "/usr/lib/
return func(cls, *args, **kwargs)
File "<string>", line 2, in GET
File "/usr/lib/
resp = func(cls, *args, **kwargs)
File "/usr/lib/
return self.collection
File "/usr/lib/
use_iterable
TypeError: argument 2 to map() must support iteration
[pid: 20677|app: 0|req: 8/35] 10.109.0.1 () {50 vars in 850 bytes} [Thu Oct 6 12:41:20 2016] GET /api/notificati
[pid: 20680|app: 0|req: 15/36] 10.109.0.1 () {50 vars in 868 bytes} [Thu Oct 6 12:41:20 2016] GET /api/nodes/
SIGINT/SIGQUIT received...killing workers...
...brutally killing workers...
mule 1 (pid: 20681) annihilated
worker 1 buried after 1 seconds
worker 2 buried after 1 seconds
worker 3 buried after 1 seconds
worker 4 buried after 1 seconds
goodbye to uWSGI.
*** Starting uWSGI 2.0.12 (64bit) on [Thu Oct 6 12:41:29 2016] ***
compiled with version: 4.8.3 20140911 (Red Hat 4.8.3-9) on 29 April 2016 14:51:44
os: Linux-3.
nodename: nailgun.
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 2
current working directory: /
writing pidfile to /var/run/
detected binary path: /usr/sbin/uwsgi
your processes number limit is 11302
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to TCP address :8001 fd 3
uWSGI running as root, you can use --uid/-
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
Python version: 2.7.5 (default, Nov 20 2015, 02:00:19) [GCC 4.8.5 20150623 (Red Hat 4.8.5-4)]
Set PythonHome to /usr
Python main interpreter initialized at 0xe824d0
python threads support enabled
your server socket listen backlog is limited to 4096 connections
your mercy for graceful operations on workers is 60 seconds
mapped 589260 bytes (575 KB) for 4 cores
*** Operational MODE: preforking ***
added /usr/lib/
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 29547)
spawned uWSGI worker 1 (pid: 29555, cores: 1)
spawned uWSGI worker 2 (pid: 29556, cores: 1)
spawned uWSGI worker 3 (pid: 29557, cores: 1)
spawned uWSGI worker 4 (pid: 29558, cores: 1)
spawned uWSGI mule 1 (pid: 29559)
2016-10-06 12:40:59 ERROR [7fdded724880] (base) Unexpected exception occured
Traceback (most recent call last):
File "/usr/lib/
return func(cls, *args, **kwargs)
File "<string>", line 2, in GET
File "/usr/lib/
return func(cls, *args, **kwargs)
File "<string>", line 2, in GET
File "/usr/lib/
resp = func(cls, *args, **kwargs)
File "/usr/lib/
unallocated
File "/usr/lib64/
return self.from_
File "/usr/lib64/
ret = self.one()
File "/usr/lib64/
ret = list(self)
File "/usr/lib64/
return self._execute_
File "/usr/lib64/
result = conn.execute(
File "/usr/lib64/
return meth(self, multiparams, params)
File "/usr/lib64/
return connection.
File "/usr/lib64/
compiled_sql, distilled_params
File "/usr/lib64/
context)
File "/usr/lib64/
exc_info
File "/usr/lib64/
reraise(
File "/usr/lib64/
context)
File "/usr/lib64/
cursor.
OperationalError: (psycopg2.
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
[SQL: 'SELECT count(*) AS count_1 \nFROM (SELECT nodes.id AS nodes_id, nodes.uuid AS nodes_uuid, nodes.cluster_id AS nodes_cluster_id, nodes.group_id AS nodes_group_id, nodes.name AS nodes_name, nodes.status AS nodes_status, nodes.meta AS nodes_meta, nodes.mac AS nodes_mac, nodes.ip AS nodes_ip, nodes.hostname AS nodes_hostname, nodes.manufacturer AS nodes_manufacturer, nodes.platform_name AS nodes_platform_
[pid: 20680|app: 0|req: 14/34] 10.109.0.1 () {50 vars in 868 bytes} [Thu Oct 6 12:40:59 2016] GET /api/nodes/
2016-10-06 12:40:59 ERROR [7fdded724880] (base) Unexpected exception occured
Traceback (most recent call last):
File "/usr/lib/
return func(cls, *args, **kwargs)
File "<string>", line 2, in GET
File "/usr/lib/
return func(cls, *args, **kwargs)
File "<string>", line 2, in GET
File "/usr/lib/
resp = func(cls, *args, **kwargs)
File "/usr/lib/
return self.collection
File "/usr/lib/
use_iterable
TypeError: argument 2 to map() must support iteration
[pid: 20678|app: 0|req: 6/33] 10.109.0.1 () {50 vars in 850 bytes} [Thu Oct 6 12:40:59 2016] GET /api/notificati