Log from ironic conductor:
2016-03-19 07:16:36	ERR	ironic.conductor.manager [-] During sync_power_state, max retries exceeded for node e8617c57-a9b0-42ec-a339-7afdf004916f, node state None does not match expected state 'power on'. Updating DB state to 'None' Switching node to maintenance mode.
2016-03-19 07:16:36	WARNING	ironic.drivers.modules.ipmitool [-] IPMI power status failed for node e8617c57-a9b0-42ec-a339-7afdf004916f with error: Unexpected error while running command.
Command: ipmitool -I lanplus -H 10.14.252.119 -L ADMINISTRATOR -U root -R 12 -N 5 -f /tmp/tmpkvWdAb power status
Exit code: 1
Stdout: u''
Stderr: u'Error: Unable to establish IPMI v2 / RMCP+ session\n'.
2016-03-19 07:16:36	ERR	ironic.drivers.modules.ipmitool [-] IPMI Error while attempting "ipmitool -I lanplus -H 10.14.252.119 -L ADMINISTRATOR -U root -R 12 -N 5 -f /tmp/tmpkvWdAb power status"for node e8617c57-a9b0-42ec-a339-7afdf004916f. Error: Unexpected error while running command.
Command: ipmitool -I lanplus -H 10.14.252.119 -L ADMINISTRATOR -U root -R 12 -N 5 -f /tmp/tmpkvWdAb power status
Exit code: 1
Stdout: u''
Stderr: u'Error: Unable to establish IPMI v2 / RMCP+ session\n'
2016-03-19 07:11:24	WARNING	ironic.conductor.manager [-] During sync_power_state, could not get power state for node e8617c57-a9b0-42ec-a339-7afdf004916f, attempt 3 of 3. Error: IPMI call failed: power status..
2016-03-19 07:11:24	WARNING	ironic.drivers.modules.ipmitool [-] IPMI power status failed for node e8617c57-a9b0-42ec-a339-7afdf004916f with error: Unexpected error while running command.
Command: ipmitool -I lanplus -H 10.14.252.119 -L ADMINISTRATOR -U root -R 12 -N 5 -f /tmp/tmpXiUKnI power status
Exit code: 1
Stdout: u''
Stderr: u'Error: Unable to establish IPMI v2 / RMCP+ session\n'.
2016-03-19 07:11:24	ERR	ironic.drivers.modules.ipmitool [-] IPMI Error while attempting "ipmitool -I lanplus -H 10.14.252.119 -L ADMINISTRATOR -U root -R 12 -N 5 -f /tmp/tmpXiUKnI power status"for node e8617c57-a9b0-42ec-a339-7afdf004916f. Error: Unexpected error while running command.
Command: ipmitool -I lanplus -H 10.14.252.119 -L ADMINISTRATOR -U root -R 12 -N 5 -f /tmp/tmpXiUKnI power status
Exit code: 1
Stdout: u''
Stderr: u'Error: Unable to establish IPMI v2 / RMCP+ session\n'
2016-03-19 07:06:57	WARNING	ironic.conductor.manager [-] During sync_power_state, could not get power state for node e8617c57-a9b0-42ec-a339-7afdf004916f, attempt 2 of 3. Error: IPMI call failed: power status..
2016-03-19 07:06:57	WARNING	ironic.drivers.modules.ipmitool [-] IPMI power status failed for node e8617c57-a9b0-42ec-a339-7afdf004916f with error: Unexpected error while running command.
Command: ipmitool -I lanplus -H 10.14.252.119 -L ADMINISTRATOR -U root -R 12 -N 5 -f /tmp/tmp2USotP power status
Exit code: 1
Stdout: u''
Stderr: u'Error: Unable to establish IPMI v2 / RMCP+ session\n'.
2016-03-19 07:06:57	ERR	ironic.drivers.modules.ipmitool [-] IPMI Error while attempting "ipmitool -I lanplus -H 10.14.252.119 -L ADMINISTRATOR -U root -R 12 -N 5 -f /tmp/tmp2USotP power status"for node e8617c57-a9b0-42ec-a339-7afdf004916f. Error: Unexpected error while running command.
Command: ipmitool -I lanplus -H 10.14.252.119 -L ADMINISTRATOR -U root -R 12 -N 5 -f /tmp/tmp2USotP power status
Exit code: 1
Stdout: u''
Stderr: u'Error: Unable to establish IPMI v2 / RMCP+ session\n'
2016-03-19 07:02:16	ERR	ironic.conductor.manager [-] Error in tear_down of node e8617c57-a9b0-42ec-a339-7afdf004916f: Failed to set node power state to power off.
2016-03-19 07:02:16.337 3184 ERROR ironic.conductor.manager Traceback (most recent call last):
2016-03-19 07:02:16.337 3184 ERROR ironic.conductor.manager   File "/usr/lib/python2.7/dist-packages/ironic/conductor/manager.py", line 799, in _do_node_tear_down
2016-03-19 07:02:16.337 3184 ERROR ironic.conductor.manager     task.driver.deploy.tear_down(task)
2016-03-19 07:02:16.337 3184 ERROR ironic.conductor.manager   File "/usr/lib/python2.7/dist-packages/ironic/conductor/task_manager.py", line 134, in wrapper
2016-03-19 07:02:16.337 3184 ERROR ironic.conductor.manager     return f(*args, **kwargs)
2016-03-19 07:02:16.337 3184 ERROR ironic.conductor.manager   File "/usr/lib/python2.7/dist-packages/ironic_fa_deploy/modules/fuel_agent.py", line 421, in tear_down
2016-03-19 07:02:16.337 3184 ERROR ironic.conductor.manager     manager_utils.node_power_action(task, states.POWER_OFF)
2016-03-19 07:02:16.337 3184 ERROR ironic.conductor.manager   File "/usr/lib/python2.7/dist-packages/ironic/conductor/task_manager.py", line 134, in wrapper
2016-03-19 07:02:16.337 3184 ERROR ironic.conductor.manager     return f(*args, **kwargs)
2016-03-19 07:02:16.337 3184 ERROR ironic.conductor.manager   File "/usr/lib/python2.7/dist-packages/ironic/conductor/utils.py", line 120, in node_power_action
2016-03-19 07:02:16.337 3184 ERROR ironic.conductor.manager     "Error: %(error)s") % {'target': target_state, 'error': e}
2016-03-19 07:02:16.337 3184 ERROR ironic.conductor.manager   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 195, in __exit__
2016-03-19 07:02:16.337 3184 ERROR ironic.conductor.manager     six.reraise(self.type_, self.value, self.tb)
2016-03-19 07:02:16.337 3184 ERROR ironic.conductor.manager   File "/usr/lib/python2.7/dist-packages/ironic/conductor/utils.py", line 113, in node_power_action
2016-03-19 07:02:16.337 3184 ERROR ironic.conductor.manager     task.driver.power.set_power_state(task, new_state)
2016-03-19 07:02:16.337 3184 ERROR ironic.conductor.manager   File "/usr/lib/python2.7/dist-packages/ironic/conductor/task_manager.py", line 134, in wrapper
2016-03-19 07:02:16.337 3184 ERROR ironic.conductor.manager     return f(*args, **kwargs)
2016-03-19 07:02:16.337 3184 ERROR ironic.conductor.manager   File "/usr/lib/python2.7/dist-packages/ironic/drivers/modules/ipmitool.py", line 754, in set_power_state
2016-03-19 07:02:16.337 3184 ERROR ironic.conductor.manager     raise exception.PowerStateFailure(pstate=pstate)
2016-03-19 07:02:16.337 3184 ERROR ironic.conductor.manager PowerStateFailure: Failed to set node power state to power off.
2016-03-19 07:02:16.337 3184 ERROR ironic.conductor.manager 
2016-03-19 07:02:16	ERR	ironic.drivers.modules.ipmitool [-] IPMI power off timed out after 6 retries on node e8617c57-a9b0-42ec-a339-7afdf004916f.
2016-03-19 07:02:16	WARNING	ironic.drivers.modules.ipmitool [-] IPMI power off failed for node e8617c57-a9b0-42ec-a339-7afdf004916f.
2016-03-19 07:02:16	WARNING	ironic.drivers.modules.ipmitool [-] IPMI power status failed for node e8617c57-a9b0-42ec-a339-7afdf004916f with error: Unexpected error while running command.
Command: ipmitool -I lanplus -H 10.14.252.119 -L ADMINISTRATOR -U root -R 12 -N 5 -f /tmp/tmpXXTBnP power status
Exit code: 1
Stdout: u''
Stderr: u'Error: Unable to establish IPMI v2 / RMCP+ session\n'.
2016-03-19 07:02:16	ERR	ironic.drivers.modules.ipmitool [-] IPMI Error while attempting "ipmitool -I lanplus -H 10.14.252.119 -L ADMINISTRATOR -U root -R 12 -N 5 -f /tmp/tmpXXTBnP power status"for node e8617c57-a9b0-42ec-a339-7afdf004916f. Error: Unexpected error while running command.
Command: ipmitool -I lanplus -H 10.14.252.119 -L ADMINISTRATOR -U root -R 12 -N 5 -f /tmp/tmpXXTBnP power status
Exit code: 1
Stdout: u''
Stderr: u'Error: Unable to establish IPMI v2 / RMCP+ session\n'
2016-03-19 06:57:38	WARNING	ironic.drivers.modules.ipmitool [-] IPMI power off failed for node e8617c57-a9b0-42ec-a339-7afdf004916f.
2016-03-19 06:57:38	WARNING	ironic.drivers.modules.ipmitool [-] IPMI power status failed for node e8617c57-a9b0-42ec-a339-7afdf004916f with error: Unexpected error while running command.
Command: ipmitool -I lanplus -H 10.14.252.119 -L ADMINISTRATOR -U root -R 12 -N 5 -f /tmp/tmp2ogQN8 power status
Exit code: 1
Stdout: u''
Stderr: u'Error: Unable to establish IPMI v2 / RMCP+ session\n'.
2016-03-19 06:57:38	ERR	ironic.drivers.modules.ipmitool [-] IPMI Error while attempting "ipmitool -I lanplus -H 10.14.252.119 -L ADMINISTRATOR -U root -R 12 -N 5 -f /tmp/tmp2ogQN8 power status"for node e8617c57-a9b0-42ec-a339-7afdf004916f. Error: Unexpected error while running command.
Command: ipmitool -I lanplus -H 10.14.252.119 -L ADMINISTRATOR -U root -R 12 -N 5 -f /tmp/tmp2ogQN8 power status
Exit code: 1
Stdout: u''
Stderr: u'Error: Unable to establish IPMI v2 / RMCP+ session\n'
2016-03-19 06:53:10	WARNING	ironic.drivers.modules.ipmitool [-] IPMI power off failed for node e8617c57-a9b0-42ec-a339-7afdf004916f.
2016-03-19 06:53:10	WARNING	ironic.drivers.modules.ipmitool [-] IPMI power status failed for node e8617c57-a9b0-42ec-a339-7afdf004916f with error: Unexpected error while running command.
Command: ipmitool -I lanplus -H 10.14.252.119 -L ADMINISTRATOR -U root -R 12 -N 5 -f /tmp/tmpmcO4mu power status
Exit code: 1
Stdout: u''
Stderr: u'Error: Unable to establish IPMI v2 / RMCP+ session\n'.
2016-03-19 06:53:10	ERR	ironic.drivers.modules.ipmitool [-] IPMI Error while attempting "ipmitool -I lanplus -H 10.14.252.119 -L ADMINISTRATOR -U root -R 12 -N 5 -f /tmp/tmpmcO4mu power status"for node e8617c57-a9b0-42ec-a339-7afdf004916f. Error: Unexpected error while running command.
Command: ipmitool -I lanplus -H 10.14.252.119 -L ADMINISTRATOR -U root -R 12 -N 5 -f /tmp/tmpmcO4mu power status
Exit code: 1
Stdout: u''
Stderr: u'Error: Unable to establish IPMI v2 / RMCP+ session\n'
2016-03-19 06:48:49	WARNING	ironic.drivers.modules.ipmitool [-] IPMI power off failed for node e8617c57-a9b0-42ec-a339-7afdf004916f.
2016-03-19 06:48:49	WARNING	ironic.drivers.modules.ipmitool [-] IPMI power status failed for node e8617c57-a9b0-42ec-a339-7afdf004916f with error: Unexpected error while running command.
Command: ipmitool -I lanplus -H 10.14.252.119 -L ADMINISTRATOR -U root -R 12 -N 5 -f /tmp/tmpFgQn7e power status
Exit code: 1
Stdout: u''
Stderr: u'Error: Unable to establish IPMI v2 / RMCP+ session\n'.
2016-03-19 06:48:49	ERR	ironic.drivers.modules.ipmitool [-] IPMI Error while attempting "ipmitool -I lanplus -H 10.14.252.119 -L ADMINISTRATOR -U root -R 12 -N 5 -f /tmp/tmpFgQn7e power status"for node e8617c57-a9b0-42ec-a339-7afdf004916f. Error: Unexpected error while running command.
Command: ipmitool -I lanplus -H 10.14.252.119 -L ADMINISTRATOR -U root -R 12 -N 5 -f /tmp/tmpFgQn7e power status
Exit code: 1
Stdout: u''
Stderr: u'Error: Unable to establish IPMI v2 / RMCP+ session\n'
2016-03-19 06:44:31	WARNING	ironic.drivers.modules.ipmitool [-] IPMI power off failed for node e8617c57-a9b0-42ec-a339-7afdf004916f.
2016-03-19 06:44:31	WARNING	ironic.drivers.modules.ipmitool [-] IPMI power status failed for node e8617c57-a9b0-42ec-a339-7afdf004916f with error: Unexpected error while running command.
Command: ipmitool -I lanplus -H 10.14.252.119 -L ADMINISTRATOR -U root -R 12 -N 5 -f /tmp/tmptRNYgL power status
Exit code: 1
Stdout: u''
Stderr: u'Error: Unable to establish IPMI v2 / RMCP+ session\n'.
2016-03-19 06:44:31	ERR	ironic.drivers.modules.ipmitool [-] IPMI Error while attempting "ipmitool -I lanplus -H 10.14.252.119 -L ADMINISTRATOR -U root -R 12 -N 5 -f /tmp/tmptRNYgL power status"for node e8617c57-a9b0-42ec-a339-7afdf004916f. Error: Unexpected error while running command.
Command: ipmitool -I lanplus -H 10.14.252.119 -L ADMINISTRATOR -U root -R 12 -N 5 -f /tmp/tmptRNYgL power status
Exit code: 1
Stdout: u''
Stderr: u'Error: Unable to establish IPMI v2 / RMCP+ session\n'
2016-03-19 06:40:14	WARNING	ironic.drivers.modules.ipmitool [-] IPMI power off failed for node e8617c57-a9b0-42ec-a339-7afdf004916f.
2016-03-19 06:40:14	WARNING	ironic.drivers.modules.ipmitool [-] IPMI power status failed for node e8617c57-a9b0-42ec-a339-7afdf004916f with error: Unexpected error while running command.
Command: ipmitool -I lanplus -H 10.14.252.119 -L ADMINISTRATOR -U root -R 12 -N 5 -f /tmp/tmpCC_yZw power status
Exit code: 1
Stdout: u''
Stderr: u'Error: Unable to establish IPMI v2 / RMCP+ session\n'.
2016-03-19 06:40:14	ERR	ironic.drivers.modules.ipmitool [-] IPMI Error while attempting "ipmitool -I lanplus -H 10.14.252.119 -L ADMINISTRATOR -U root -R 12 -N 5 -f /tmp/tmpCC_yZw power status"for node e8617c57-a9b0-42ec-a339-7afdf004916f. Error: Unexpected error while running command.
Command: ipmitool -I lanplus -H 10.14.252.119 -L ADMINISTRATOR -U root -R 12 -N 5 -f /tmp/tmpCC_yZw power status
Exit code: 1
Stdout: u''
Stderr: u'Error: Unable to establish IPMI v2 / RMCP+ session\n'
2016-03-19 06:05:52	WARNING	ironic.conductor.manager [-] During sync_power_state, could not get power state for node e8617c57-a9b0-42ec-a339-7afdf004916f, attempt 1 of 3. Error: IPMI call failed: power status..
2016-03-19 06:05:52	WARNING	ironic.drivers.modules.ipmitool [-] IPMI power status failed for node e8617c57-a9b0-42ec-a339-7afdf004916f with error: Unexpected error while running command.
Command: ipmitool -I lanplus -H 10.14.252.119 -L ADMINISTRATOR -U root -R 12 -N 5 -f /tmp/tmpio3QwZ power status
Exit code: 1
Stdout: u''
Stderr: u'Error: Unable to establish IPMI v2 / RMCP+ session\n'.
2016-03-19 06:05:52	ERR	ironic.drivers.modules.ipmitool [-] IPMI Error while attempting "ipmitool -I lanplus -H 10.14.252.119 -L ADMINISTRATOR -U root -R 12 -N 5 -f /tmp/tmpio3QwZ power status"for node e8617c57-a9b0-42ec-a339-7afdf004916f. Error: Unexpected error while running command.
Command: ipmitool -I lanplus -H 10.14.252.119 -L ADMINISTRATOR -U root -R 12 -N 5 -f /tmp/tmpio3QwZ power status
Exit code: 1
Stdout: u''
Stderr: u'Error: Unable to establish IPMI v2 / RMCP+ session\n'
2016-03-19 03:48:46	WARNING	ironic.conductor.manager [req-7ff44e5e-e316-4cad-9c7c-8b449655ebf3 admin - - - -] A conductor with hostname node-3.ly.com was previously registered. Updating registration
2016-03-19 03:48:45	WARNING	oslo_config.cfg [-] Option "amqp_durable_queues" from group "DEFAULT" is deprecated. Use option "amqp_durable_queues" from group "oslo_messaging_rabbit".
2016-03-19 03:48:39	WARNING	oslo_messaging.server [-] start/stop/wait must be called in the same thread
2016-03-19 03:48:39	WARNING	oslo_messaging.server [-] start/stop/wait must be called in the same thread
2016-03-19 03:32:21	ERR	ironic.conductor.utils [-] Timeout reached while waiting for callback for node e8617c57-a9b0-42ec-a339-7afdf004916f
2016-03-18 08:17:19	WARNING	ironic.conductor.manager [-] Conductor could not connect to database while heartbeating.
2016-03-18 08:17:11	ERR	oslo_service.periodic_task [-] Error during ConductorManager._check_cleanwait_timeouts
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task Traceback (most recent call last):
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/oslo_service/periodic_task.py", line 218, in run_periodic_tasks
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task     task(self, context)
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/ironic/conductor/manager.py", line 1392, in _check_cleanwait_timeouts
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task     last_error=last_error)
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/ironic/conductor/manager.py", line 2054, in _fail_if_in_state
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task     for node_uuid, driver in node_iter:
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/ironic/conductor/manager.py", line 1471, in iter_nodes
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task     node_list = self.dbapi.get_nodeinfo_list(columns=columns, **kwargs)
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/ironic/db/sqlalchemy/api.py", line 204, in get_nodeinfo_list
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task     sort_key, sort_dir, query)
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/ironic/db/sqlalchemy/api.py", line 143, in _paginate_query
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task     return query.all()
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2399, in all
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task     return list(self)
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2516, in __iter__
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task     return self._execute_and_instances(context)
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2529, in _execute_and_instances
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task     close_with_result=True)
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2520, in _connection_from_session
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task     **kw)
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 882, in connection
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task     execution_options=execution_options)
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 889, in _connection_for_bind
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task     conn = engine.contextual_connect(**kw)
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2039, in contextual_connect
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task     self._wrap_pool_connect(self.pool.connect, None),
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2078, in _wrap_pool_connect
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task     e, dialect, self)
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1401, in _handle_dbapi_exception_noconnection
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task     util.raise_from_cause(newraise, exc_info)
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task     reraise(type(exception), exception, tb=exc_tb)
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2074, in _wrap_pool_connect
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task     return fn()
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 376, in connect
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task     return _ConnectionFairy._checkout(self)
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 713, in _checkout
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task     fairy = _ConnectionRecord.checkout(pool)
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 485, in checkout
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task     rec.checkin()
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 60, in __exit__
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task     compat.reraise(exc_type, exc_value, exc_tb)
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 482, in checkout
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task     dbapi_connection = rec.get_connection()
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 563, in get_connection
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task     self.connection = self.__connect()
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 607, in __connect
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task     connection = self.__pool._invoke_creator(self)
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/strategies.py", line 97, in connect
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task     return dialect.connect(*cargs, **cparams)
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 385, in connect
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task     return self.dbapi.connect(*cargs, **cparams)
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/MySQLdb/__init__.py", line 81, in Connect
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task     return Connection(*args, **kwargs)
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 206, in __init__
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task     super(Connection, self).__init__(*args, **kwargs2)
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task DBConnectionError: (_mysql_exceptions.OperationalError) (2013, "Lost connection to MySQL server at 'reading initial communication packet', system error: 0")
2016-03-18 08:17:11.832 5203 ERROR oslo_service.periodic_task 
2016-03-18 08:17:06	WARNING	ironic.conductor.manager [-] Conductor could not connect to database while heartbeating.
2016-03-18 08:17:06	ERR	oslo_service.periodic_task [-] Error during ConductorManager._check_inspect_timeouts
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task Traceback (most recent call last):
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/oslo_service/periodic_task.py", line 218, in run_periodic_tasks
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task     task(self, context)
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/ironic/conductor/manager.py", line 2018, in _check_inspect_timeouts
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task     sort_key, last_error=last_error)
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/ironic/conductor/manager.py", line 2054, in _fail_if_in_state
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task     for node_uuid, driver in node_iter:
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/ironic/conductor/manager.py", line 1471, in iter_nodes
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task     node_list = self.dbapi.get_nodeinfo_list(columns=columns, **kwargs)
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/ironic/db/sqlalchemy/api.py", line 204, in get_nodeinfo_list
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task     sort_key, sort_dir, query)
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/ironic/db/sqlalchemy/api.py", line 143, in _paginate_query
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task     return query.all()
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2399, in all
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task     return list(self)
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2516, in __iter__
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task     return self._execute_and_instances(context)
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2529, in _execute_and_instances
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task     close_with_result=True)
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2520, in _connection_from_session
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task     **kw)
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 882, in connection
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task     execution_options=execution_options)
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 889, in _connection_for_bind
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task     conn = engine.contextual_connect(**kw)
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2041, in contextual_connect
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task     **kwargs)
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 92, in __init__
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task     self.dispatch.engine_connect(self, self.__branch)
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/event/attr.py", line 256, in __call__
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task     fn(*args, **kw)
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/engines.py", line 80, in _connect_ping_listener
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task     connection.scalar(select([1]))
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 844, in scalar
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task     return self.execute(object, *multiparams, **params).scalar()
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 914, in execute
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task     return meth(self, multiparams, params)
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py", line 323, in _execute_on_connection
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task     return connection._execute_clauseelement(self, multiparams, params)
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1010, in _execute_clauseelement
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task     compiled_sql, distilled_params
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1078, in _execute_context
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task     None, None)
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1337, in _handle_dbapi_exception
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task     util.raise_from_cause(newraise, exc_info)
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task     reraise(type(exception), exception, tb=exc_tb)
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1071, in _execute_context
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task     conn = self._revalidate_connection()
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 393, in _revalidate_connection
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task     self.__connection = self.engine.raw_connection(_connection=self)
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2104, in raw_connection
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task     self.pool.unique_connection, _connection)
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2080, in _wrap_pool_connect
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task     util.reraise(*sys.exc_info())
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2074, in _wrap_pool_connect
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task     return fn()
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 318, in unique_connection
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task     return _ConnectionFairy._checkout(self)
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 713, in _checkout
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task     fairy = _ConnectionRecord.checkout(pool)
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 485, in checkout
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task     rec.checkin()
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 60, in __exit__
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task     compat.reraise(exc_type, exc_value, exc_tb)
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 482, in checkout
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task     dbapi_connection = rec.get_connection()
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 594, in get_connection
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task     self.connection = self.__connect()
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 607, in __connect
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task     connection = self.__pool._invoke_creator(self)
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/strategies.py", line 97, in connect
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task     return dialect.connect(*cargs, **cparams)
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 385, in connect
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task     return self.dbapi.connect(*cargs, **cparams)
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/MySQLdb/__init__.py", line 81, in Connect
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task     return Connection(*args, **kwargs)
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 206, in __init__
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task     super(Connection, self).__init__(*args, **kwargs2)
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task DBConnectionError: (_mysql_exceptions.OperationalError) (2013, "Lost connection to MySQL server at 'reading initial communication packet', system error: 0") [SQL: u'SELECT 1']
2016-03-18 08:17:06.811 5203 ERROR oslo_service.periodic_task 
2016-03-18 05:19:15	WARNING	ironic.conductor.manager [-] Conductor could not connect to database while heartbeating.
2016-03-17 19:21:26	WARNING	oslo_config.cfg [-] Option "amqp_durable_queues" from group "DEFAULT" is deprecated. Use option "amqp_durable_queues" from group "oslo_messaging_rabbit".