part of the log:
2016-05-20 09:27:44.163 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:27:59.164 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:28:14.165 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:28:17.235 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Auditing locally available compute resources for node vdi11
2016-05-20 09:28:17.637 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Total usable vcpus: 24, total allocated vcpus: 0
2016-05-20 09:28:17.638 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Final resource view: name=vdi11 phys_ram=64389MB used_ram=12800MB phys_disk=199GB used_disk=70GB total_vcpus=24 used_vcpus=0 pci_stats=None
2016-05-20 09:28:19.290 5576 INFO nova.compute.cloudos_resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Compute_service record updated for vdi11:vdi11
2016-05-20 09:28:29.166 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:28:32.564 5576 WARNING nova.compute.manager [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] While synchronizing instance power states, found 7 instances in the database and 2 instances on the hypervisor.
2016-05-20 09:28:44.167 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:28:59.168 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:29:14.169 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:29:18.236 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Auditing locally available compute resources for node vdi11
2016-05-20 09:29:18.627 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Total usable vcpus: 24, total allocated vcpus: 0
2016-05-20 09:29:18.628 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Final resource view: name=vdi11 phys_ram=64389MB used_ram=12800MB phys_disk=199GB used_disk=70GB total_vcpus=24 used_vcpus=0 pci_stats=None
2016-05-20 09:29:20.310 5576 INFO nova.compute.cloudos_resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Compute_service record updated for vdi11:vdi11
2016-05-20 09:29:29.170 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:29:32.563 5576 WARNING nova.compute.manager [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] While synchronizing instance power states, found 7 instances in the database and 2 instances on the hypervisor.
2016-05-20 09:29:44.171 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:29:59.172 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:30:14.173 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:30:19.236 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Auditing locally available compute resources for node vdi11
2016-05-20 09:30:19.642 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Total usable vcpus: 24, total allocated vcpus: 0
2016-05-20 09:30:19.643 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Final resource view: name=vdi11 phys_ram=64389MB used_ram=12800MB phys_disk=199GB used_disk=70GB total_vcpus=24 used_vcpus=0 pci_stats=None
2016-05-20 09:30:21.282 5576 INFO nova.compute.cloudos_resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Compute_service record updated for vdi11:vdi11
2016-05-20 09:30:29.174 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:30:34.507 5576 WARNING nova.compute.manager [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] While synchronizing instance power states, found 7 instances in the database and 2 instances on the hypervisor.
2016-05-20 09:30:44.175 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:30:59.176 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:31:14.178 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:31:21.234 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Auditing locally available compute resources for node vdi11
2016-05-20 09:31:21.622 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Total usable vcpus: 24, total allocated vcpus: 0
2016-05-20 09:31:21.623 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Final resource view: name=vdi11 phys_ram=64389MB used_ram=12800MB phys_disk=199GB used_disk=70GB total_vcpus=24 used_vcpus=0 pci_stats=None
2016-05-20 09:31:23.311 5576 INFO nova.compute.cloudos_resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Compute_service record updated for vdi11:vdi11
2016-05-20 09:31:29.179 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:31:35.509 5576 WARNING nova.compute.manager [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] While synchronizing instance power states, found 7 instances in the database and 2 instances on the hypervisor.
2016-05-20 09:31:44.180 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:31:59.181 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:32:14.182 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:32:23.230 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Auditing locally available compute resources for node vdi11
2016-05-20 09:32:23.619 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Total usable vcpus: 24, total allocated vcpus: 0
2016-05-20 09:32:23.620 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Final resource view: name=vdi11 phys_ram=64389MB used_ram=12800MB phys_disk=199GB used_disk=70GB total_vcpus=24 used_vcpus=0 pci_stats=None
2016-05-20 09:32:25.233 5576 INFO nova.compute.cloudos_resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Compute_service record updated for vdi11:vdi11
2016-05-20 09:32:29.183 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:32:37.524 5576 WARNING nova.compute.manager [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] While synchronizing instance power states, found 7 instances in the database and 2 instances on the hypervisor.
2016-05-20 09:32:44.184 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:32:59.185 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:33:14.186 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:33:24.231 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Auditing locally available compute resources for node vdi11
2016-05-20 09:33:24.642 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Total usable vcpus: 24, total allocated vcpus: 0
2016-05-20 09:33:24.643 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Final resource view: name=vdi11 phys_ram=64389MB used_ram=12800MB phys_disk=199GB used_disk=70GB total_vcpus=24 used_vcpus=0 pci_stats=None
2016-05-20 09:33:26.480 5576 INFO nova.compute.cloudos_resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Compute_service record updated for vdi11:vdi11
2016-05-20 09:33:29.187 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:33:38.579 5576 WARNING nova.compute.manager [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] While synchronizing instance power states, found 7 instances in the database and 2 instances on the hypervisor.
2016-05-20 09:33:44.188 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:33:59.190 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:34:14.191 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:34:26.230 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Auditing locally available compute resources for node vdi11
2016-05-20 09:34:26.633 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Total usable vcpus: 24, total allocated vcpus: 0
2016-05-20 09:34:26.633 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Final resource view: name=vdi11 phys_ram=64389MB used_ram=12800MB phys_disk=199GB used_disk=70GB total_vcpus=24 used_vcpus=0 pci_stats=None
2016-05-20 09:34:28.101 5576 INFO nova.compute.cloudos_resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Compute_service record updated for vdi11:vdi11
2016-05-20 09:34:29.192 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:34:38.512 5576 WARNING nova.compute.manager [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] While synchronizing instance power states, found 7 instances in the database and 2 instances on the hypervisor.
2016-05-20 09:34:44.193 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:34:59.194 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:35:14.195 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:35:27.236 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Auditing locally available compute resources for node vdi11
2016-05-20 09:35:27.629 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Total usable vcpus: 24, total allocated vcpus: 0
2016-05-20 09:35:27.629 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Final resource view: name=vdi11 phys_ram=64389MB used_ram=12800MB phys_disk=199GB used_disk=70GB total_vcpus=24 used_vcpus=0 pci_stats=None
2016-05-20 09:35:29.161 5576 INFO nova.compute.cloudos_resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Compute_service record updated for vdi11:vdi11
2016-05-20 09:35:29.195 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:35:38.526 5576 WARNING nova.compute.manager [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] While synchronizing instance power states, found 7 instances in the database and 2 instances on the hypervisor.
2016-05-20 09:35:44.196 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:35:59.197 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:36:14.199 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:36:27.233 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Auditing locally available compute resources for node vdi11
2016-05-20 09:36:27.628 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Total usable vcpus: 24, total allocated vcpus: 0
2016-05-20 09:36:27.629 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Final resource view: name=vdi11 phys_ram=64389MB used_ram=12800MB phys_disk=199GB used_disk=70GB total_vcpus=24 used_vcpus=0 pci_stats=None
2016-05-20 09:36:29.074 5576 INFO nova.compute.cloudos_resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Compute_service record updated for vdi11:vdi11
2016-05-20 09:36:29.200 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:36:39.517 5576 WARNING nova.compute.manager [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] While synchronizing instance power states, found 7 instances in the database and 2 instances on the hypervisor.
2016-05-20 09:36:44.200 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:36:59.201 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:37:14.202 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:37:27.241 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Auditing locally available compute resources for node vdi11
2016-05-20 09:37:27.637 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Total usable vcpus: 24, total allocated vcpus: 0
2016-05-20 09:37:27.638 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Final resource view: name=vdi11 phys_ram=64389MB used_ram=12800MB phys_disk=199GB used_disk=70GB total_vcpus=24 used_vcpus=0 pci_stats=None
2016-05-20 09:37:29.203 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:37:29.293 5576 INFO nova.compute.cloudos_resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Compute_service record updated for vdi11:vdi11
2016-05-20 09:37:41.511 5576 WARNING nova.compute.manager [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] While synchronizing instance power states, found 7 instances in the database and 2 instances on the hypervisor.
2016-05-20 09:37:44.204 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:37:59.205 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:38:14.206 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:38:28.235 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Auditing locally available compute resources for node vdi11
2016-05-20 09:38:28.625 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Total usable vcpus: 24, total allocated vcpus: 0
2016-05-20 09:38:28.626 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Final resource view: name=vdi11 phys_ram=64389MB used_ram=12800MB phys_disk=199GB used_disk=70GB total_vcpus=24 used_vcpus=0 pci_stats=None
2016-05-20 09:38:29.207 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:38:30.051 5576 INFO nova.compute.cloudos_resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Compute_service record updated for vdi11:vdi11
2016-05-20 09:38:42.510 5576 WARNING nova.compute.manager [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] While synchronizing instance power states, found 7 instances in the database and 2 instances on the hypervisor.
2016-05-20 09:38:44.208 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:38:59.209 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:39:14.210 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:39:29.211 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:39:30.236 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Auditing locally available compute resources for node vdi11
2016-05-20 09:39:30.642 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Total usable vcpus: 24, total allocated vcpus: 0
2016-05-20 09:39:30.642 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Final resource view: name=vdi11 phys_ram=64389MB used_ram=12800MB phys_disk=199GB used_disk=70GB total_vcpus=24 used_vcpus=0 pci_stats=None
2016-05-20 09:39:32.312 5576 INFO nova.compute.cloudos_resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Compute_service record updated for vdi11:vdi11
2016-05-20 09:39:43.516 5576 WARNING nova.compute.manager [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] While synchronizing instance power states, found 7 instances in the database and 2 instances on the hypervisor.
2016-05-20 09:39:44.212 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:39:59.213 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:40:14.214 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:40:29.215 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:40:31.229 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Auditing locally available compute resources for node vdi11
2016-05-20 09:40:31.630 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Total usable vcpus: 24, total allocated vcpus: 0
2016-05-20 09:40:31.630 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Final resource view: name=vdi11 phys_ram=64389MB used_ram=12800MB phys_disk=199GB used_disk=70GB total_vcpus=24 used_vcpus=0 pci_stats=None
2016-05-20 09:40:33.317 5576 INFO nova.compute.cloudos_resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Compute_service record updated for vdi11:vdi11
2016-05-20 09:40:44.216 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:40:45.508 5576 WARNING nova.compute.manager [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] While synchronizing instance power states, found 7 instances in the database and 2 instances on the hypervisor.
2016-05-20 09:40:59.217 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:41:14.218 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:41:29.219 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:41:31.231 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Auditing locally available compute resources for node vdi11
2016-05-20 09:41:31.595 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Total usable vcpus: 24, total allocated vcpus: 0
2016-05-20 09:41:31.596 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Final resource view: name=vdi11 phys_ram=64389MB used_ram=12800MB phys_disk=199GB used_disk=70GB total_vcpus=24 used_vcpus=0 pci_stats=None
2016-05-20 09:41:32.983 5576 INFO nova.compute.cloudos_resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Compute_service record updated for vdi11:vdi11
2016-05-20 09:41:44.220 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:41:45.513 5576 WARNING nova.compute.manager [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] While synchronizing instance power states, found 7 instances in the database and 2 instances on the hypervisor.
2016-05-20 09:41:59.221 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:42:14.222 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:42:29.223 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:42:32.230 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Auditing locally available compute resources for node vdi11
2016-05-20 09:42:32.624 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Total usable vcpus: 24, total allocated vcpus: 0
2016-05-20 09:42:32.625 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Final resource view: name=vdi11 phys_ram=64389MB used_ram=12800MB phys_disk=199GB used_disk=70GB total_vcpus=24 used_vcpus=0 pci_stats=None
2016-05-20 09:42:34.409 5576 INFO nova.compute.cloudos_resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Compute_service record updated for vdi11:vdi11
2016-05-20 09:42:44.224 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:42:46.516 5576 WARNING nova.compute.manager [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] While synchronizing instance power states, found 7 instances in the database and 2 instances on the hypervisor.
2016-05-20 09:42:59.225 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:43:14.226 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:43:29.227 5576 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 32] Broken pipe
2016-05-20 09:43:34.229 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Auditing locally available compute resources for node vdi11
2016-05-20 09:43:34.695 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Total usable vcpus: 24, total allocated vcpus: 0
2016-05-20 09:43:34.696 5576 INFO nova.compute.resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Final resource view: name=vdi11 phys_ram=64389MB used_ram=12800MB phys_disk=199GB used_disk=70GB total_vcpus=24 used_vcpus=0 pci_stats=None
2016-05-20 09:43:36.428 5576 INFO nova.compute.cloudos_resource_tracker [req-73b5a98e-226a-4592-890a-e6e6f76f0090 - - - - -] Compute_service record updated for vdi11:vdi11

the code that the error occur:
   def _heartbeat_thread_job(self):
        """Thread that maintains inactive connections
        """
        while not self._heartbeat_exit_event.is_set():
            with self._connection_lock.for_heartbeat():

                recoverable_errors = (
                    self.connection.recoverable_channel_errors +
                    self.connection.recoverable_connection_errors)

                try:
                    try:
                        self._heartbeat_check()
                        # NOTE(sileht): We need to drain event to receive
                        # heartbeat from the broker but don't hold the
                        # connection too much times. In amqpdriver a connection
                        # is used exclusively for read or for write, so we have
                        # to do this for connection used for write drain_events
                        # already do that for other connection
                        try:
                            self.connection.drain_events(timeout=0.001)
                        except socket.timeout:
                            pass
                    except recoverable_errors as exc:
                        LOG.info(_LI("A recoverable connection/channel error "
                                     "occurred, trying to reconnect: %s"), exc)
                        self.ensure_connection()
                except Exception:
                    LOG.warning(_LW("Unexpected error during heartbeart "
                                    "thread processing, retrying..."))
                    LOG.debug('Exception', exc_info=True)

            self._heartbeat_exit_event.wait(
                timeout=self._heartbeat_wait_timeout)
        self._heartbeat_exit_event.clear()

hello,
I meet one issue;this is strange.
The connecttion is ok, but the error occur when the rpc client sent the heartbeat. So it will reconnect.But the result is the same. The error do not occur at reconnection, and occur at checking the heartbeat when have ensured the connection.
The issue is frequent before we update the version, and now the issue still occur but not frequent
I do not the reason, Can I get some help from you?