2017-09-01 17:59:48.574 11936 INFO oslo.messaging._drivers.impl_rabbit [-] [1b4582b9-73bc-42aa-bc6f-786b3ff4424e] Reconnected to AMQP server on 172.28.3.1:5672 via [amqp] client with port 37268.
2017-09-01 18:08:03.182 11936 WARNING oslo.service.loopingcall [req-737147d3-7ad6-4848-898a-f0ae49f3a083 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 45.14 sec
2017-09-01 18:08:03.188 11936 WARNING oslo.service.loopingcall [req-43d296d2-092c-414e-b002-50a90f7ee8e6 - - - - -] Function 'cinder.service.Service.report_state' run outlasted interval by 20.83 sec
2017-09-01 18:08:33.892 11936 WARNING oslo.service.loopingcall [req-43d296d2-092c-414e-b002-50a90f7ee8e6 - - - - -] Function 'cinder.service.Service.report_state' run outlasted interval by 20.70 sec
2017-09-01 18:11:58.716 11936 WARNING oslo.service.loopingcall [req-43d296d2-092c-414e-b002-50a90f7ee8e6 - - - - -] Function 'cinder.service.Service.report_state' run outlasted interval by 45.00 sec
2017-09-01 18:24:33.752 11936 WARNING oslo.service.loopingcall [req-43d296d2-092c-414e-b002-50a90f7ee8e6 - - - - -] Function 'cinder.service.Service.report_state' run outlasted interval by 20.28 sec
2017-09-01 18:27:33.859 11936 WARNING oslo.service.loopingcall [req-43d296d2-092c-414e-b002-50a90f7ee8e6 - - - - -] Function 'cinder.service.Service.report_state' run outlasted interval by 20.32 sec
2017-09-01 18:36:44.280 11936 INFO cinder.volume.flows.manager.create_volume [req-278ffe8f-8196-44e5-9d7a-45dfc65eb310 0ec2bce00d144aa1b8da00bddc606007 c9fe9505fdde4de680679748c7cfee7e - default default] Volume 0a02a919-8809-4be5-9c0d-9d7fe4bd1b86: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-0a02a919-8809-4be5-9c0d-9d7fe4bd1b86'}
2017-09-01 18:36:44.476 11936 INFO cinder.volume.flows.manager.create_volume [req-278ffe8f-8196-44e5-9d7a-45dfc65eb310 0ec2bce00d144aa1b8da00bddc606007 c9fe9505fdde4de680679748c7cfee7e - default default] Volume volume-0a02a919-8809-4be5-9c0d-9d7fe4bd1b86 (0a02a919-8809-4be5-9c0d-9d7fe4bd1b86): created successfully
2017-09-01 18:36:44.483 11936 INFO cinder.volume.manager [req-278ffe8f-8196-44e5-9d7a-45dfc65eb310 0ec2bce00d144aa1b8da00bddc606007 c9fe9505fdde4de680679748c7cfee7e - default default] Created volume successfully.
2017-09-01 18:38:06.992 11936 INFO cinder.volume.manager [req-e63616e2-8628-46cc-8a3d-f3e733e57e2c 0ec2bce00d144aa1b8da00bddc606007 c9fe9505fdde4de680679748c7cfee7e - default default] Deleted volume successfully.
2017-09-01 18:39:56.235 11936 ERROR oslo.messaging._drivers.impl_rabbit [-] [1b4582b9-73bc-42aa-bc6f-786b3ff4424e] AMQP server 172.28.3.1:5672 closed the connection. Check login credentials: Socket closed
2017-09-01 18:39:56.237 11936 ERROR oslo.messaging._drivers.impl_rabbit [-] [5ad98e61-df77-4f06-a800-a736c546f744] AMQP server 172.28.3.1:5672 closed the connection. Check login credentials: Socket closed
2017-09-01 18:39:56.245 11936 WARNING oslo.service.loopingcall [req-737147d3-7ad6-4848-898a-f0ae49f3a083 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 49.26 sec
2017-09-01 18:45:30.422 11936 ERROR cinder.volume.drivers.rbd [req-737147d3-7ad6-4848-898a-f0ae49f3a083 - - - - -] Error connecting to ceph cluster.
2017-09-01 18:45:30.422 11936 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
2017-09-01 18:45:30.422 11936 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 312, in _connect_to_rados
2017-09-01 18:45:30.422 11936 ERROR cinder.volume.drivers.rbd     client.connect()
2017-09-01 18:45:30.422 11936 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 875, in rados.Rados.connect (/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.1.4/rpm/el7/BUILD/ceph-12.1.4/build/src/pybind/rados/pyrex/rados.c:9764)
2017-09-01 18:45:30.422 11936 ERROR cinder.volume.drivers.rbd TimedOut: [errno 110] error connecting to the cluster
2017-09-01 18:45:30.422 11936 ERROR cinder.volume.drivers.rbd 
2017-09-01 18:45:30.496 11936 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Socket closed
2017-09-01 18:45:30.515 11936 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Socket closed
2017-09-01 18:45:30.531 11936 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Socket closed
2017-09-01 18:45:30.789 11936 INFO oslo.messaging._drivers.impl_rabbit [-] [1b4582b9-73bc-42aa-bc6f-786b3ff4424e] Reconnected to AMQP server on 172.28.3.1:5672 via [amqp] client with port 60012.
2017-09-01 18:45:30.794 11936 INFO oslo.messaging._drivers.impl_rabbit [-] [5ad98e61-df77-4f06-a800-a736c546f744] Reconnected to AMQP server on 172.28.3.1:5672 via [amqp] client with port 60016.
2017-09-01 18:45:30.797 11936 ERROR oslo.messaging._drivers.impl_rabbit [-] Failed to process message ... skipping it.
2017-09-01 18:45:30.797 11936 ERROR oslo.messaging._drivers.impl_rabbit Traceback (most recent call last):
2017-09-01 18:45:30.797 11936 ERROR oslo.messaging._drivers.impl_rabbit   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/impl_rabbit.py", line 365, in _callback
2017-09-01 18:45:30.797 11936 ERROR oslo.messaging._drivers.impl_rabbit     self.callback(RabbitMessage(message))
2017-09-01 18:45:30.797 11936 ERROR oslo.messaging._drivers.impl_rabbit   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 190, in __call__
2017-09-01 18:45:30.797 11936 ERROR oslo.messaging._drivers.impl_rabbit     unique_id = self.msg_id_cache.check_duplicate_message(message)
2017-09-01 18:45:30.797 11936 ERROR oslo.messaging._drivers.impl_rabbit   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqp.py", line 121, in check_duplicate_message
2017-09-01 18:45:30.797 11936 ERROR oslo.messaging._drivers.impl_rabbit     raise rpc_common.DuplicateMessageError(msg_id=msg_id)
2017-09-01 18:45:30.797 11936 ERROR oslo.messaging._drivers.impl_rabbit DuplicateMessageError: Found duplicate message(9d257a17c989463eb77dd918b3f9c6f3). Skipping it.
2017-09-01 18:45:30.797 11936 ERROR oslo.messaging._drivers.impl_rabbit 
2017-09-01 18:45:30.806 11936 INFO cinder.volume.flows.manager.create_volume [req-aa5c9c27-5895-474d-8f99-753cd6391432 0ec2bce00d144aa1b8da00bddc606007 c9fe9505fdde4de680679748c7cfee7e - default default] Volume c17b3bf0-b56b-4792-8b7d-404c2ea0fb8c: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-c17b3bf0-b56b-4792-8b7d-404c2ea0fb8c'}
2017-09-01 18:50:30.825 11936 ERROR cinder.volume.drivers.rbd [req-aa5c9c27-5895-474d-8f99-753cd6391432 0ec2bce00d144aa1b8da00bddc606007 c9fe9505fdde4de680679748c7cfee7e - default default] Error connecting to ceph cluster.