Fuel for OpenStack

  1. Bug #1373569
  2. Comment #8

Comment 8 for bug 1373569

Revision history for this message
Tyler Wilson (loth) wrote :

@Bogdan Looks like 00:06:05 was the first hints at issues, possibly when the flapping started (check ceilometer-agent-notification.log)

2014-09-24T00:06:05.493953+01:00 err: 2014-09-23 23:06:05.480 21070 ERROR ceilometer.openstack.common.rpc.common [-] Failed to consume message from queue: (0, 0): (320) CONNECTION_FORCED - broker forced conn
ection closure with reason 'shutdown'
2014-09-23 23:06:05.480 21070 TRACE ceilometer.openstack.common.rpc.common Traceback (most recent call last):
2014-09-23 23:06:05.480 21070 TRACE ceilometer.openstack.common.rpc.common File "/usr/lib/python2.7/dist-packages/ceilometer/openstack/common/rpc/impl_kombu.py", line 591, in ensure
2014-09-23 23:06:05.480 21070 TRACE ceilometer.openstack.common.rpc.common return method(*args, **kwargs)
2014-09-23 23:06:05.480 21070 TRACE ceilometer.openstack.common.rpc.common File "/usr/lib/python2.7/dist-packages/ceilometer/openstack/common/rpc/impl_kombu.py", line 671, in _consume
2014-09-23 23:06:05.480 21070 TRACE ceilometer.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout)
2014-09-23 23:06:05.480 21070 TRACE ceilometer.openstack.common.rpc.common File "/usr/lib/python2.7/dist-packages/kombu/connection.py", line 279, in drain_events
2014-09-23 23:06:05.480 21070 TRACE ceilometer.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs)
2014-09-23 23:06:05.480 21070 TRACE ceilometer.openstack.common.rpc.common File "/usr/lib/python2.7/dist-packages/kombu/transport/pyamqp.py", line 91, in drain_events
2014-09-23 23:06:05.480 21070 TRACE ceilometer.openstack.common.rpc.common return connection.drain_events(**kwargs)
2014-09-23 23:06:05.480 21070 TRACE ceilometer.openstack.common.rpc.common File "/usr/lib/python2.7/dist-packages/amqp/connection.py", line 320, in drain_events
2014-09-23 23:06:05.480 21070 TRACE ceilometer.openstack.common.rpc.common return amqp_method(channel, args)
2014-09-23 23:06:05.480 21070 TRACE ceilometer.openstack.common.rpc.common File "/usr/lib/python2.7/dist-packages/amqp/connection.py", line 526, in _close
2014-09-23 23:06:05.480 21070 TRACE ceilometer.openstack.common.rpc.common (class_id, method_id), ConnectionError)
2014-09-23 23:06:05.480 21070 TRACE ceilometer.openstack.common.rpc.common ConnectionForced: (0, 0): (320) CONNECTION_FORCED - broker forced connection closure with reason 'shutdown'
2014-09-24T00:06:05.493953+01:00 info: 2014-09-23 23:06:05.484 21070 INFO ceilometer.openstack.common.rpc.common [-] Reconnecting to AMQP server on 127.0.0.1:5673
2014-09-24T00:06:05.495063+01:00 info: 2014-09-23 23:06:05.484 21070 INFO ceilometer.openstack.common.rpc.common [-] Delaying reconnect for 5.0 seconds...
2014-09-24T00:06:10.511682+01:00 err: 2014-09-23 23:06:10.501 21070 ERROR ceilometer.openstack.common.rpc.common [-] AMQP server on 127.0.0.1:5673 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 1 seconds.