Job URL: https://ci.fuel-infra.org/job/11.0-community.main.ubuntu.bvt_2/644
Snapshot URL: https://ci.fuel-infra.org/job/11.0-community.main.ubuntu.bvt_2/644/artifact/logs/fail_error_ceph_rados_gw-fuel-snapshot-2017-04-08_01-02-03.tar
from fuel-node
astute.log contains
-----
2017-04-08 00:57:20 DEBUG [18293] Node 3, task openstack-haproxy-glance, manifest /etc/puppet/modules/osnailyfacter/modular/openstack-haproxy/openstack-haproxy-glance.pp, status: running
2017-04-08 00:57:20 ERROR [18293] Puppet agent took too long to run puppet task. Mark task as failed. Node 3, task openstack-haproxy-glance, manifest /etc/puppet/modules/osnailyfacter/modular/openstack-haproxy/openstack-haproxy-glance.pp
2017-04-08 00:57:20 INFO [18293] Node[3]: Finished task Task[openstack-haproxy-glance/3] with status: failed
-----
from node-3
puppet.log contents
-----
2017-04-08 00:57:30 +0000 /Stage[main]/Openstack::Ha::Haproxy_restart/Exec[haproxy-restart] (err): Failed to call refresh: Command exceeded timeout
2017-04-08 00:57:30 +0000 /Stage[main]/Openstack::Ha::Haproxy_restart/Exec[haproxy-restart] (err): Command exceeded timeout
-----
from node-3
the full haproxy.log
-----
2017-04-08T00:51:20.011666+00:00 node-3 haproxy[25679]: Proxy mysqld started.
2017-04-08T00:51:20.682351+00:00 node-3 haproxy[25683]: Backup Server mysqld/node-3 is DOWN, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms. 1 active and 1 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:51:30.016488+00:00 node-3 haproxy[25683]: Server mysqld/node-1 is DOWN, reason: Layer4 timeout, check duration: 10002ms. 0 active and 1 backup servers left. Running on backup. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:51:30.270137+00:00 node-3 haproxy[26570]: Proxy mysqld started.
2017-04-08T00:51:30.270752+00:00 node-3 haproxy[26570]: Proxy object-storage started.
2017-04-08T00:51:30.271043+00:00 node-3 haproxy[25683]: Stopping proxy mysqld in 0 ms.
2017-04-08T00:51:30.271179+00:00 node-3 haproxy[25683]: Proxy mysqld stopped (FE: 0 conns, BE: 0 conns).
2017-04-08T00:51:30.603081+00:00 node-3 haproxy[26574]: Backup Server mysqld/node-3 is DOWN, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms. 1 active and 1 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:51:31.602813+00:00 node-3 haproxy[26574]: Server object-storage/node-3 is DOWN, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms. 2 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:51:33.271050+00:00 node-3 haproxy[26574]: Server object-storage/node-1 is DOWN, reason: Layer4 timeout, check duration: 2000ms. 1 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:51:33.938121+00:00 node-3 haproxy[26574]: Server object-storage/node-4 is DOWN, reason: Layer4 timeout, check duration: 2001ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:51:33.938372+00:00 node-3 haproxy[26574]: proxy object-storage has no server available!
2017-04-08T00:51:40.273588+00:00 node-3 haproxy[26574]: Server mysqld/node-1 is DOWN, reason: Layer4 timeout, check duration: 10001ms. 0 active and 1 backup servers left. Running on backup. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:51:40.937601+00:00 node-3 haproxy[26574]: Backup Server mysqld/node-4 is DOWN, reason: Layer4 timeout, check duration: 10001ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:51:40.937948+00:00 node-3 haproxy[26574]: proxy mysqld has no server available!
2017-04-08T00:51:44.697907+00:00 node-3 haproxy[28144]: Proxy neutron started.
2017-04-08T00:51:44.701685+00:00 node-3 haproxy[28144]: Proxy mysqld started.
2017-04-08T00:51:44.702005+00:00 node-3 haproxy[28144]: Proxy object-storage started.
2017-04-08T00:51:44.705406+00:00 node-3 haproxy[26574]: Stopping proxy mysqld in 0 ms.
2017-04-08T00:51:44.705667+00:00 node-3 haproxy[26574]: Stopping proxy object-storage in 0 ms.
2017-04-08T00:51:44.705919+00:00 node-3 haproxy[26574]: Proxy mysqld stopped (FE: 0 conns, BE: 0 conns).
2017-04-08T00:51:44.706129+00:00 node-3 haproxy[26574]: Proxy object-storage stopped (FE: 0 conns, BE: 0 conns).
2017-04-08T00:51:44.922399+00:00 node-3 haproxy[28148]: Server neutron/node-3 is DOWN, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms. 2 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:51:45.585572+00:00 node-3 haproxy[28148]: Backup Server mysqld/node-3 is DOWN, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms. 1 active and 1 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:51:46.252129+00:00 node-3 haproxy[28148]: Server object-storage/node-3 is DOWN, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms. 2 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:51:48.032258+00:00 node-3 haproxy[28148]: Server object-storage/node-1 is DOWN, reason: Layer4 timeout, check duration: 2001ms. 1 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:51:48.476160+00:00 node-3 haproxy[28148]: Server object-storage/node-4 is DOWN, reason: Layer4 timeout, check duration: 2001ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:51:48.476536+00:00 node-3 haproxy[28148]: proxy object-storage has no server available!
2017-04-08T00:51:54.292258+00:00 node-3 haproxy[29032]: Proxy keystone-1 started.
2017-04-08T00:51:54.292967+00:00 node-3 haproxy[29032]: Proxy keystone-2 started.
2017-04-08T00:51:54.293441+00:00 node-3 haproxy[29032]: Proxy neutron started.
2017-04-08T00:51:54.293705+00:00 node-3 haproxy[29032]: Proxy mysqld started.
2017-04-08T00:51:54.294544+00:00 node-3 haproxy[29032]: Proxy object-storage started.
2017-04-08T00:51:54.295096+00:00 node-3 haproxy[28148]: Stopping proxy neutron in 0 ms.
2017-04-08T00:51:54.295368+00:00 node-3 haproxy[28148]: Stopping proxy mysqld in 0 ms.
2017-04-08T00:51:54.295611+00:00 node-3 haproxy[28148]: Stopping proxy object-storage in 0 ms.
2017-04-08T00:51:54.296018+00:00 node-3 haproxy[28148]: Proxy neutron stopped (FE: 0 conns, BE: 0 conns).
2017-04-08T00:51:54.296289+00:00 node-3 haproxy[28148]: Proxy mysqld stopped (FE: 0 conns, BE: 0 conns).
2017-04-08T00:51:54.296497+00:00 node-3 haproxy[28148]: Proxy object-storage stopped (FE: 0 conns, BE: 0 conns).
2017-04-08T00:51:54.424692+00:00 node-3 haproxy[29036]: Server keystone-1/node-3 is DOWN, reason: Layer7 wrong status, code: 404, info: "Not Found", check duration: 0ms. 2 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:51:54.825083+00:00 node-3 haproxy[29036]: Server keystone-2/node-3 is DOWN, reason: Layer7 wrong status, code: 404, info: "Not Found", check duration: 0ms. 2 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:51:55.225369+00:00 node-3 haproxy[29036]: Server neutron/node-3 is DOWN, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms. 2 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:51:55.625493+00:00 node-3 haproxy[29036]: Backup Server mysqld/node-3 is DOWN, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms. 1 active and 1 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:51:56.024621+00:00 node-3 haproxy[29036]: Server object-storage/node-3 is DOWN, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms. 2 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:51:57.893380+00:00 node-3 haproxy[29036]: Server object-storage/node-1 is DOWN, reason: Layer4 timeout, check duration: 2001ms. 1 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:51:58.159158+00:00 node-3 haproxy[29036]: Server object-storage/node-4 is DOWN, reason: Layer4 timeout, check duration: 2001ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:51:58.159431+00:00 node-3 haproxy[29036]: proxy object-storage has no server available!
2017-04-08T00:52:04.299168+00:00 node-3 haproxy[29036]: Server keystone-1/node-1 is DOWN, reason: Layer4 timeout, check duration: 10001ms. 1 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:52:04.557746+00:00 node-3 haproxy[29036]: Server keystone-1/node-4 is DOWN, reason: Layer4 timeout, check duration: 10000ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:52:04.558024+00:00 node-3 haproxy[29036]: proxy keystone-1 has no server available!
2017-04-08T00:52:04.692857+00:00 node-3 haproxy[29036]: Server keystone-2/node-1 is DOWN, reason: Layer4 timeout, check duration: 10001ms. 1 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:52:04.960481+00:00 node-3 haproxy[29036]: Server keystone-2/node-4 is DOWN, reason: Layer4 timeout, check duration: 10001ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:52:04.960705+00:00 node-3 haproxy[29036]: proxy keystone-2 has no server available!
2017-04-08T00:52:05.092612+00:00 node-3 haproxy[29036]: Server neutron/node-1 is DOWN, reason: Layer4 timeout, check duration: 10000ms. 1 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:52:05.359059+00:00 node-3 haproxy[29036]: Server neutron/node-4 is DOWN, reason: Layer4 timeout, check duration: 10001ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:52:05.359428+00:00 node-3 haproxy[29036]: proxy neutron has no server available!
2017-04-08T00:52:05.492590+00:00 node-3 haproxy[29036]: Server mysqld/node-1 is DOWN, reason: Layer4 timeout, check duration: 10000ms. 0 active and 1 backup servers left. Running on backup. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:52:05.758957+00:00 node-3 haproxy[29036]: Backup Server mysqld/node-4 is DOWN, reason: Layer4 timeout, check duration: 10001ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:52:05.759194+00:00 node-3 haproxy[29036]: proxy mysqld has no server available!
2017-04-08T00:52:10.071900+00:00 node-3 haproxy[30655]: Proxy stats started.
2017-04-08T00:52:10.073240+00:00 node-3 haproxy[30655]: Proxy keystone-1 started.
2017-04-08T00:52:10.073631+00:00 node-3 haproxy[30655]: Proxy keystone-2 started.
2017-04-08T00:52:10.074656+00:00 node-3 haproxy[30655]: Proxy neutron started.
2017-04-08T00:52:10.074916+00:00 node-3 haproxy[30655]: Proxy mysqld started.
2017-04-08T00:52:10.075579+00:00 node-3 haproxy[30655]: Proxy object-storage started.
2017-04-08T00:52:10.075980+00:00 node-3 haproxy[29036]: Stopping proxy keystone-1 in 0 ms.
2017-04-08T00:52:10.076594+00:00 node-3 haproxy[29036]: Stopping proxy keystone-2 in 0 ms.
2017-04-08T00:52:10.078555+00:00 node-3 haproxy[29036]: Stopping proxy neutron in 0 ms.
2017-04-08T00:52:10.079707+00:00 node-3 haproxy[29036]: Stopping proxy mysqld in 0 ms.
2017-04-08T00:52:10.080841+00:00 node-3 haproxy[29036]: Stopping proxy object-storage in 0 ms.
2017-04-08T00:52:10.081993+00:00 node-3 haproxy[29036]: Proxy keystone-1 stopped (FE: 0 conns, BE: 0 conns).
2017-04-08T00:52:10.082677+00:00 node-3 haproxy[29036]: Proxy keystone-2 stopped (FE: 0 conns, BE: 0 conns).
2017-04-08T00:52:10.083499+00:00 node-3 haproxy[29036]: Proxy neutron stopped (FE: 0 conns, BE: 0 conns).
2017-04-08T00:52:10.084326+00:00 node-3 haproxy[29036]: Proxy mysqld stopped (FE: 0 conns, BE: 0 conns).
2017-04-08T00:52:10.085141+00:00 node-3 haproxy[29036]: Proxy object-storage stopped (FE: 0 conns, BE: 0 conns).
2017-04-08T00:52:10.204828+00:00 node-3 haproxy[30659]: Server keystone-1/node-3 is DOWN, reason: Layer7 wrong status, code: 404, info: "Not Found", check duration: 0ms. 2 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:52:10.337238+00:00 node-3 haproxy[30659]: Server keystone-1/node-4 is DOWN, reason: Layer4 connection problem, info: "General socket error (Network is unreachable)", check duration: 0ms. 1 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:52:10.604355+00:00 node-3 haproxy[30659]: Server keystone-2/node-3 is DOWN, reason: Layer7 wrong status, code: 404, info: "Not Found", check duration: 0ms. 2 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:52:11.004577+00:00 node-3 haproxy[30659]: Server neutron/node-3 is DOWN, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms. 2 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:52:11.403691+00:00 node-3 haproxy[30659]: Backup Server mysqld/node-3 is DOWN, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms. 1 active and 1 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:52:11.803826+00:00 node-3 haproxy[30659]: Server object-storage/node-3 is DOWN, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms. 2 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:52:13.673039+00:00 node-3 haproxy[30659]: Server object-storage/node-1 is DOWN, reason: Layer4 timeout, check duration: 2001ms. 1 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:52:13.940295+00:00 node-3 haproxy[30659]: Server object-storage/node-4 is DOWN, reason: Layer4 timeout, check duration: 2001ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:52:13.940754+00:00 node-3 haproxy[30659]: proxy object-storage has no server available!
2017-04-08T00:52:19.097078+00:00 node-3 haproxy[31516]: Proxy stats started.
2017-04-08T00:52:19.099158+00:00 node-3 haproxy[31516]: Proxy horizon started.
2017-04-08T00:52:19.103402+00:00 node-3 haproxy[31516]: Proxy horizon-ssl started.
2017-04-08T00:52:19.103513+00:00 node-3 haproxy[31516]: Proxy keystone-1 started.
2017-04-08T00:52:19.103613+00:00 node-3 haproxy[31516]: Proxy keystone-2 started.
2017-04-08T00:52:19.103698+00:00 node-3 haproxy[31516]: Proxy neutron started.
2017-04-08T00:52:19.103793+00:00 node-3 haproxy[31516]: Proxy mysqld started.
2017-04-08T00:52:19.103881+00:00 node-3 haproxy[31516]: Proxy object-storage started.
2017-04-08T00:52:19.103968+00:00 node-3 haproxy[30659]: Stopping proxy stats in 0 ms.
2017-04-08T00:52:19.104257+00:00 node-3 haproxy[30659]: Stopping proxy keystone-1 in 0 ms.
2017-04-08T00:52:19.104470+00:00 node-3 haproxy[30659]: Stopping proxy keystone-2 in 0 ms.
2017-04-08T00:52:19.104675+00:00 node-3 haproxy[30659]: Stopping proxy neutron in 0 ms.
2017-04-08T00:52:19.104877+00:00 node-3 haproxy[30659]: Stopping proxy mysqld in 0 ms.
2017-04-08T00:52:19.105091+00:00 node-3 haproxy[30659]: Stopping proxy object-storage in 0 ms.
2017-04-08T00:52:19.105285+00:00 node-3 haproxy[30659]: Proxy stats stopped (FE: 0 conns, BE: 0 conns).
2017-04-08T00:52:19.105495+00:00 node-3 haproxy[30659]: Proxy keystone-1 stopped (FE: 0 conns, BE: 0 conns).
2017-04-08T00:52:19.105703+00:00 node-3 haproxy[30659]: Proxy keystone-2 stopped (FE: 0 conns, BE: 0 conns).
2017-04-08T00:52:19.105935+00:00 node-3 haproxy[30659]: Proxy neutron stopped (FE: 0 conns, BE: 0 conns).
2017-04-08T00:52:19.106138+00:00 node-3 haproxy[30659]: Proxy mysqld stopped (FE: 0 conns, BE: 0 conns).
2017-04-08T00:52:19.106339+00:00 node-3 haproxy[30659]: Proxy object-storage stopped (FE: 0 conns, BE: 0 conns).
2017-04-08T00:52:19.539552+00:00 node-3 haproxy[31520]: Server keystone-1/node-3 is DOWN, reason: Layer4 connection problem, info: "General socket error (Network is unreachable)", check duration: 0ms. 2 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:52:19.872233+00:00 node-3 haproxy[31520]: Server keystone-2/node-3 is DOWN, reason: Layer7 wrong status, code: 404, info: "Not Found", check duration: 0ms. 2 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:52:20.205763+00:00 node-3 haproxy[31520]: Server neutron/node-3 is DOWN, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms. 2 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:52:20.538881+00:00 node-3 haproxy[31520]: Backup Server mysqld/node-3 is DOWN, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms. 1 active and 1 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:52:20.872326+00:00 node-3 haproxy[31520]: Server object-storage/node-3 is DOWN, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms. 2 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:52:21.101952+00:00 node-3 haproxy[31520]: Server horizon-ssl/node-1 is DOWN, reason: Layer4 timeout, check duration: 2001ms. 2 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:52:21.321511+00:00 node-3 haproxy[31520]: Server horizon-ssl/node-4 is DOWN, reason: Layer4 timeout, check duration: 2001ms. 1 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:52:22.762035+00:00 node-3 haproxy[31520]: Server object-storage/node-1 is DOWN, reason: Layer4 timeout, check duration: 2001ms. 1 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:52:22.983724+00:00 node-3 haproxy[31520]: Server object-storage/node-4 is DOWN, reason: Layer4 timeout, check duration: 2001ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:52:22.984011+00:00 node-3 haproxy[31520]: proxy object-storage has no server available!
2017-04-08T00:52:29.429096+00:00 node-3 haproxy[31520]: Server keystone-1/node-1 is DOWN, reason: Layer4 timeout, check duration: 10001ms. 1 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:52:29.651247+00:00 node-3 haproxy[31520]: Server keystone-1/node-4 is DOWN, reason: Layer4 timeout, check duration: 10001ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:52:29.651704+00:00 node-3 haproxy[31520]: proxy keystone-1 has no server available!
2017-04-08T00:52:29.762486+00:00 node-3 haproxy[31520]: Server keystone-2/node-1 is DOWN, reason: Layer4 timeout, check duration: 10001ms. 1 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:52:29.984432+00:00 node-3 haproxy[31520]: Server keystone-2/node-4 is DOWN, reason: Layer4 timeout, check duration: 10000ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:52:29.984758+00:00 node-3 haproxy[31520]: proxy keystone-2 has no server available!
2017-04-08T00:52:30.098122+00:00 node-3 haproxy[31520]: Server neutron/node-1 is DOWN, reason: Layer4 timeout, check duration: 10002ms. 1 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:52:30.318437+00:00 node-3 haproxy[31520]: Server neutron/node-4 is DOWN, reason: Layer4 timeout, check duration: 10001ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:52:30.318774+00:00 node-3 haproxy[31520]: proxy neutron has no server available!
2017-04-08T00:52:30.429685+00:00 node-3 haproxy[31520]: Server mysqld/node-1 is DOWN, reason: Layer4 timeout, check duration: 10001ms. 0 active and 1 backup servers left. Running on backup. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:52:30.652324+00:00 node-3 haproxy[31520]: Backup Server mysqld/node-4 is DOWN, reason: Layer4 timeout, check duration: 10000ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2017-04-08T00:52:30.652673+00:00 node-3 haproxy[31520]: proxy mysqld has no server available!
-----
Looks like issues with network connection
