Comment 0 for bug 1368152

l3 agent dies after MessagingTimeout error, follows the trace:

neutron-l3-agent[9269]: 2014-09-10 13:16:30.633 9269 CRITICAL neutron [req-76b176bb-158e-49ca-b53a-95961b9ffaef None] MessagingTimeout: Timed out waiting for a reply to message ID ae9233a190354e73ac133523342f3daf
neutron-l3-agent[9269]: 2014-09-10 13:16:30.633 9269 TRACE neutron Traceback (most recent call last):
neutron-l3-agent[9269]: 2014-09-10 13:16:30.633 9269 TRACE neutron File "/opt/stack/venvs/openstack/bin/neutron-l3-agent", line 10, in <module>
neutron-l3-agent[9269]: 2014-09-10 13:16:30.633 9269 TRACE neutron sys.exit(main())
neutron-l3-agent[9269]: 2014-09-10 13:16:30.633 9269 TRACE neutron File "/opt/stack/venvs/openstack/lib/python2.7/site-packages/neutron/agent/l3_agent.py", line 1928, in main
neutron-l3-agent[9269]: 2014-09-10 13:16:30.633 9269 TRACE neutron manager=manager)
neutron-l3-agent[9269]: 2014-09-10 13:16:30.633 9269 TRACE neutron File "/opt/stack/venvs/openstack/lib/python2.7/site-packages/neutron/service.py", line 264, in create
neutron-l3-agent[9269]: 2014-09-10 13:16:30.633 9269 TRACE neutron periodic_fuzzy_delay=periodic_fuzzy_delay)
neutron-l3-agent[9269]: 2014-09-10 13:16:30.633 9269 TRACE neutron File "/opt/stack/venvs/openstack/lib/python2.7/site-packages/neutron/service.py", line 197, in __init__
neutron-l3-agent[9269]: 2014-09-10 13:16:30.633 9269 TRACE neutron self.manager = manager_class(host=host, *args, **kwargs)
neutron-l3-agent[9269]: 2014-09-10 13:16:30.633 9269 TRACE neutron File "/opt/stack/venvs/openstack/lib/python2.7/site-packages/neutron/agent/l3_agent.py", line 1844, in __init__
neutron-l3-agent[9269]: 2014-09-10 13:16:30.633 9269 TRACE neutron super(L3NATAgentWithStateReport, self).__init__(host=host, conf=conf)
neutron-l3-agent[9269]: 2014-09-10 13:16:30.633 9269 TRACE neutron File "/opt/stack/venvs/openstack/lib/python2.7/site-packages/neutron/agent/l3_agent.py", line 525, in __init__
neutron-l3-agent[9269]: 2014-09-10 13:16:30.633 9269 TRACE neutron self.plugin_rpc.get_service_plugin_list(self.context))
neutron-l3-agent[9269]: 2014-09-10 13:16:30.633 9269 TRACE neutron File "/opt/stack/venvs/openstack/lib/python2.7/site-packages/neutron/agent/l3_agent.py", line 144, in get_service_plugin_list
neutron-l3-agent[9269]: 2014-09-10 13:16:30.633 9269 TRACE neutron version='1.3')
neutron-l3-agent[9269]: 2014-09-10 13:16:30.633 9269 TRACE neutron File "/opt/stack/venvs/openstack/lib/python2.7/site-packages/neutron/common/log.py", line 36, in wrapper
neutron-l3-agent[9269]: 2014-09-10 13:16:30.633 9269 TRACE neutron return method(*args, **kwargs)
neutron-l3-agent[9269]: 2014-09-10 13:16:30.633 9269 TRACE neutron File "/opt/stack/venvs/openstack/lib/python2.7/site-packages/neutron/common/rpc.py", line 175, in call
neutron-l3-agent[9269]: 2014-09-10 13:16:30.633 9269 TRACE neutron context, msg, rpc_method='call', **kwargs)
neutron-l3-agent[9269]: 2014-09-10 13:16:30.633 9269 TRACE neutron File "/opt/stack/venvs/openstack/lib/python2.7/site-packages/neutron/common/rpc.py", line 201, in __call_rpc_method
neutron-l3-agent[9269]: 2014-09-10 13:16:30.633 9269 TRACE neutron return func(context, msg['method'], **msg['args'])
neutron-l3-agent[9269]: 2014-09-10 13:16:30.633 9269 TRACE neutron File "/opt/stack/venvs/openstack/lib/python2.7/site-packages/oslo/messaging/rpc/client.py", line 152, in call
neutron-l3-agent[9269]: 2014-09-10 13:16:30.633 9269 TRACE neutron retry=self.retry)
neutron-l3-agent[9269]: 2014-09-10 13:16:30.633 9269 TRACE neutron File "/opt/stack/venvs/openstack/lib/python2.7/site-packages/oslo/messaging/transport.py", line 90, in _send
neutron-l3-agent[9269]: 2014-09-10 13:16:30.633 9269 TRACE neutron timeout=timeout, retry=retry)
neutron-l3-agent[9269]: 2014-09-10 13:16:30.633 9269 TRACE neutron File "/opt/stack/venvs/openstack/lib/python2.7/site-packages/oslo/messaging/_drivers/amqpdriver.py", line 408, in send
neutron-l3-agent[9269]: 2014-09-10 13:16:30.633 9269 TRACE neutron retry=retry)
neutron-l3-agent[9269]: 2014-09-10 13:16:30.633 9269 TRACE neutron File "/opt/stack/venvs/openstack/lib/python2.7/site-packages/oslo/messaging/_drivers/amqpdriver.py", line 397, in _send
neutron-l3-agent[9269]: 2014-09-10 13:16:30.633 9269 TRACE neutron result = self._waiter.wait(msg_id, timeout)
neutron-l3-agent[9269]: 2014-09-10 13:16:30.633 9269 TRACE neutron File "/opt/stack/venvs/openstack/lib/python2.7/site-packages/oslo/messaging/_drivers/amqpdriver.py", line 285, in wait
neutron-l3-agent[9269]: 2014-09-10 13:16:30.633 9269 TRACE neutron reply, ending = self._poll_connection(msg_id, timeout)
neutron-l3-agent[9269]: 2014-09-10 13:16:30.633 9269 TRACE neutron File "/opt/stack/venvs/openstack/lib/python2.7/site-packages/oslo/messaging/_drivers/amqpdriver.py", line 235, in _poll_connection
neutron-l3-agent[9269]: 2014-09-10 13:16:30.633 9269 TRACE neutron % msg_id)
neutron-l3-agent[9269]: 2014-09-10 13:16:30.633 9269 TRACE neutron MessagingTimeout: Timed out waiting for a reply to message ID ae9233a190354e73ac133523342f3daf
neutron-l3-agent[9269]: 2014-09-10 13:16:30.633 9269 TRACE neutron
systemd[1]: neutron-l3-agent.service: main process exited, code=exited, status=1/FAILURE
systemd[1]: Unit neutron-l3-agent.service entered failed state.

I am not sure about the root cause of the MessagingTimeout either but that seems a different problem