Maybe related to https://review.openstack.org/#/c/347708/2 looks similar with bug: https://bugs.launchpad.net/neutron/+bug/1606827
I think Neutron team should look at it
node-2/var/log/neutron/neutron-openvswitch-agent.log
2016-08-08 08:16:09.394 25878 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [-] Failed reporting state! 2016-08-08 08:16:09.394 25878 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent Traceback (most recent call last): 2016-08-08 08:16:09.394 25878 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py", line 322, in _report_state 2016-08-08 08:16:09.394 25878 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent True) 2016-08-08 08:16:09.394 25878 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File "/usr/lib/python2.7/dist-packages/neutron/agent/rpc.py", line 86, in report_state 2016-08-08 08:16:09.394 25878 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent return method(context, 'report_state', **kwargs) 2016-08-08 08:16:09.394 25878 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File "/usr/lib/python2.7/dist-packages/neutron/common/rpc.py", line 155, in call 2016-08-08 08:16:09.394 25878 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent time.sleep(wait) 2016-08-08 08:16:09.394 25878 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ 2016-08-08 08:16:09.394 25878 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent self.force_reraise() 2016-08-08 08:16:09.394 25878 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise 2016-08-08 08:16:09.394 25878 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent six.reraise(self.type_, self.value, self.tb) 2016-08-08 08:16:09.394 25878 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File "/usr/lib/python2.7/dist-packages/neutron/common/rpc.py", line 136, in call 2016-08-08 08:16:09.394 25878 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent return self._original_context.call(ctxt, method, **kwargs) 2016-08-08 08:16:09.394 25878 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 158, in call 2016-08-08 08:16:09.394 25878 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent retry=self.retry) 2016-08-08 08:16:09.394 25878 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 91, in _send 2016-08-08 08:16:09.394 25878 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent timeout=timeout, retry=retry) 2016-08-08 08:16:09.394 25878 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 512, in send 2016-08-08 08:16:09.394 25878 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent retry=retry) 2016-08-08 08:16:09.394 25878 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 501, in _send 2016-08-08 08:16:09.394 25878 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent result = self._waiter.wait(msg_id, timeout) 2016-08-08 08:16:09.394 25878 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 379, in wait 2016-08-08 08:16:09.394 25878 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent message = self.waiters.get(msg_id, timeout=timeout) 2016-08-08 08:16:09.394 25878 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 277, in get 2016-08-08 08:16:09.394 25878 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent 'to message ID %s' % msg_id) 2016-08-08 08:16:09.394 25878 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent MessagingTimeout: Timed out waiting for a reply to message ID 0c5d4e5a6ec447b98c882169b7ef6aac 2016-08-08 08:16:09.394 25878 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
Maybe related to https:/ /review. openstack. org/#/c/ 347708/ 2 /bugs.launchpad .net/neutron/ +bug/1606827
looks similar with bug:
https:/
I think Neutron team should look at it
node-2/ var/log/ neutron/ neutron- openvswitch- agent.log
2016-08-08 08:16:09.394 25878 ERROR neutron. plugins. ml2.drivers. openvswitch. agent.ovs_ neutron_ agent [-] Failed reporting state! plugins. ml2.drivers. openvswitch. agent.ovs_ neutron_ agent Traceback (most recent call last): plugins. ml2.drivers. openvswitch. agent.ovs_ neutron_ agent File "/usr/lib/ python2. 7/dist- packages/ neutron/ plugins/ ml2/drivers/ openvswitch/ agent/ovs_ neutron_ agent.py" , line 322, in _report_state plugins. ml2.drivers. openvswitch. agent.ovs_ neutron_ agent True) plugins. ml2.drivers. openvswitch. agent.ovs_ neutron_ agent File "/usr/lib/ python2. 7/dist- packages/ neutron/ agent/rpc. py", line 86, in report_state plugins. ml2.drivers. openvswitch. agent.ovs_ neutron_ agent return method(context, 'report_state', **kwargs) plugins. ml2.drivers. openvswitch. agent.ovs_ neutron_ agent File "/usr/lib/ python2. 7/dist- packages/ neutron/ common/ rpc.py" , line 155, in call plugins. ml2.drivers. openvswitch. agent.ovs_ neutron_ agent time.sleep(wait) plugins. ml2.drivers. openvswitch. agent.ovs_ neutron_ agent File "/usr/lib/ python2. 7/dist- packages/ oslo_utils/ excutils. py", line 220, in __exit__ plugins. ml2.drivers. openvswitch. agent.ovs_ neutron_ agent self.force_ reraise( ) plugins. ml2.drivers. openvswitch. agent.ovs_ neutron_ agent File "/usr/lib/ python2. 7/dist- packages/ oslo_utils/ excutils. py", line 196, in force_reraise plugins. ml2.drivers. openvswitch. agent.ovs_ neutron_ agent six.reraise( self.type_ , self.value, self.tb) plugins. ml2.drivers. openvswitch. agent.ovs_ neutron_ agent File "/usr/lib/ python2. 7/dist- packages/ neutron/ common/ rpc.py" , line 136, in call plugins. ml2.drivers. openvswitch. agent.ovs_ neutron_ agent return self._original_ context. call(ctxt, method, **kwargs) plugins. ml2.drivers. openvswitch. agent.ovs_ neutron_ agent File "/usr/lib/ python2. 7/dist- packages/ oslo_messaging/ rpc/client. py", line 158, in call plugins. ml2.drivers. openvswitch. agent.ovs_ neutron_ agent retry=self.retry) plugins. ml2.drivers. openvswitch. agent.ovs_ neutron_ agent File "/usr/lib/ python2. 7/dist- packages/ oslo_messaging/ transport. py", line 91, in _send plugins. ml2.drivers. openvswitch. agent.ovs_ neutron_ agent timeout=timeout, retry=retry) plugins. ml2.drivers. openvswitch. agent.ovs_ neutron_ agent File "/usr/lib/ python2. 7/dist- packages/ oslo_messaging/ _drivers/ amqpdriver. py", line 512, in send plugins. ml2.drivers. openvswitch. agent.ovs_ neutron_ agent retry=retry) plugins. ml2.drivers. openvswitch. agent.ovs_ neutron_ agent File "/usr/lib/ python2. 7/dist- packages/ oslo_messaging/ _drivers/ amqpdriver. py", line 501, in _send plugins. ml2.drivers. openvswitch. agent.ovs_ neutron_ agent result = self._waiter. wait(msg_ id, timeout) plugins. ml2.drivers. openvswitch. agent.ovs_ neutron_ agent File "/usr/lib/ python2. 7/dist- packages/ oslo_messaging/ _drivers/ amqpdriver. py", line 379, in wait plugins. ml2.drivers. openvswitch. agent.ovs_ neutron_ agent message = self.waiters. get(msg_ id, timeout=timeout) plugins. ml2.drivers. openvswitch. agent.ovs_ neutron_ agent File "/usr/lib/ python2. 7/dist- packages/ oslo_messaging/ _drivers/ amqpdriver. py", line 277, in get plugins. ml2.drivers. openvswitch. agent.ovs_ neutron_ agent 'to message ID %s' % msg_id) plugins. ml2.drivers. openvswitch. agent.ovs_ neutron_ agent MessagingTimeout: Timed out waiting for a reply to message ID 0c5d4e5a6ec447b 98c882169b7ef6a ac plugins. ml2.drivers. openvswitch. agent.ovs_ neutron_ agent
2016-08-08 08:16:09.394 25878 ERROR neutron.
2016-08-08 08:16:09.394 25878 ERROR neutron.
2016-08-08 08:16:09.394 25878 ERROR neutron.
2016-08-08 08:16:09.394 25878 ERROR neutron.
2016-08-08 08:16:09.394 25878 ERROR neutron.
2016-08-08 08:16:09.394 25878 ERROR neutron.
2016-08-08 08:16:09.394 25878 ERROR neutron.
2016-08-08 08:16:09.394 25878 ERROR neutron.
2016-08-08 08:16:09.394 25878 ERROR neutron.
2016-08-08 08:16:09.394 25878 ERROR neutron.
2016-08-08 08:16:09.394 25878 ERROR neutron.
2016-08-08 08:16:09.394 25878 ERROR neutron.
2016-08-08 08:16:09.394 25878 ERROR neutron.
2016-08-08 08:16:09.394 25878 ERROR neutron.
2016-08-08 08:16:09.394 25878 ERROR neutron.
2016-08-08 08:16:09.394 25878 ERROR neutron.
2016-08-08 08:16:09.394 25878 ERROR neutron.
2016-08-08 08:16:09.394 25878 ERROR neutron.
2016-08-08 08:16:09.394 25878 ERROR neutron.
2016-08-08 08:16:09.394 25878 ERROR neutron.
2016-08-08 08:16:09.394 25878 ERROR neutron.
2016-08-08 08:16:09.394 25878 ERROR neutron.
2016-08-08 08:16:09.394 25878 ERROR neutron.
2016-08-08 08:16:09.394 25878 ERROR neutron.
2016-08-08 08:16:09.394 25878 ERROR neutron.
2016-08-08 08:16:09.394 25878 ERROR neutron.
2016-08-08 08:16:09.394 25878 ERROR neutron.