VMs are failing to schedule with the following error
2013-11-15 20:50:21.405 ERROR nova.scheduler.filter_scheduler [req-d2c26348-53e6-448a-8975-4f22f4e89782 demo demo] [instance: c8069c13-593f-48fb-aae9-198961097eb2] Error from last host: devstack-precise-hpcloud-az3-662002 (node devstack-precise-hpcloud-az3-662002): [u'Traceback (most recent call last):\n', u' File "/opt/stack/new/nova/nova/compute/manager.py", line 1030, in _build_instance\n set_access_ip=set_access_ip)\n', u' File "/opt/stack/new/nova/nova/compute/manager.py", line 1439, in _spawn\n LOG.exception(_(\'Instance failed to spawn\'), instance=instance)\n', u' File "/opt/stack/new/nova/nova/compute/manager.py", line 1436, in _spawn\n block_device_info)\n', u' File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 2100, in spawn\n admin_pass=admin_password)\n', u' File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 2451, in _create_image\n content=files, extra_md=extra_md, network_info=network_info)\n', u' File "/opt/stack/new/nova/nova/api/metadata/base.py", line 165, in __init__\n ec2utils.get_ip_info_for_instance_from_nw_info(network_info)\n', u' File "/opt/stack/new/nova/nova/api/ec2/ec2utils.py", line 149, in get_ip_info_for_instance_from_nw_info\n fixed_ips = nw_info.fixed_ips()\n', u' File "/opt/stack/new/nova/nova/network/model.py", line 368, in _sync_wrapper\n self.wait()\n', u' File "/opt/stack/new/nova/nova/network/model.py", line 400, in wait\n self[:] = self._gt.wait()\n', u' File "/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py", line 168, in wait\n return self._exit_event.wait()\n', u' File "/usr/local/lib/python2.7/dist-packages/eventlet/event.py", line 120, in wait\n current.throw(*self._exc)\n', u' File "/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py", line 194, in main\n result = function(*args, **kwargs)\n', u' File "/opt/stack/new/nova/nova/compute/manager.py", line 1220, in _allocate_network_async\n dhcp_options=dhcp_options)\n', u' File "/opt/stack/new/nova/nova/network/neutronv2/api.py", line 359, in allocate_for_instance\n nw_info = self._get_instance_nw_info(context, instance, networks=nets)\n', u' File "/opt/stack/new/nova/nova/network/api.py", line 49, in wrapper\n res = f(self, context, *args, **kwargs)\n', u' File "/opt/stack/new/nova/nova/network/neutronv2/api.py", line 458, in _get_instance_nw_info\n nw_info = self._build_network_info_model(context, instance, networks)\n', u' File "/opt/stack/new/nova/nova/network/neutronv2/api.py", line 1022, in _build_network_info_model\n subnets = self._nw_info_get_subnets(context, port, network_IPs)\n', u' File "/opt/stack/new/nova/nova/network/neutronv2/api.py", line 924, in _nw_info_get_subnets\n subnets = self._get_subnets_from_port(context, port)\n', u' File "/opt/stack/new/nova/nova/network/neutronv2/api.py", line 1066, in _get_subnets_from_port\n data = neutronv2.get_client(context).list_ports(**search_opts)\n', u' File "/opt/stack/new/python-neutronclient/neutronclient/v2_0/client.py", line 111, in with_params\n ret = self.function(instance, *args, **kwargs)\n', u' File "/opt/stack/new/python-neutronclient/neutronclient/v2_0/client.py", line 306, in list_ports\n **_params)\n', u' File "/opt/stack/new/python-neutronclient/neutronclient/v2_0/client.py", line 1250, in list\n for r in self._pagination(collection, path, **params):\n', u' File "/opt/stack/new/python-neutronclient/neutronclient/v2_0/client.py", line 1263, in _pagination\n res = self.get(path, params=params)\n', u' File "/opt/stack/new/python-neutronclient/neutronclient/v2_0/client.py", line 1236, in get\n headers=headers, params=params)\n', u' File "/opt/stack/new/python-neutronclient/neutronclient/v2_0/client.py", line 1228, in retry_request\n raise exceptions.ConnectionFailed(reason=_("Maximum attempts reached"))\n', u'ConnectionFailed: Connection to neutron failed: Maximum attempts reached\n']
Connection to neutron failed: Maximum attempts reached
http://logs.openstack.org/96/56496/1/gate/gate-tempest-devstack-vm-neutron-isolated/8df6c6c/logs/screen-n-sch.txt.gz#_2013-11-15_20_50_21_405
logstash query: "Connection to neutron failed: Maximum attempts reached" AND filename:"logs/screen-n-sch.txt"
http:// logstash. openstack. org/#eyJzZWFyY2 giOiJcIkNvbm5lY 3Rpb24gdG8gbmV1 dHJvbiBmYWlsZWQ 6IE1heGltdW0gYX R0ZW1wdHMgcmVhY 2hlZFwiICBBTkQg ZmlsZW5hbWU6XCJ sb2dzL3NjcmVlbi 1uLXNjaC50eHRcI iIsImZpZWxkcyI6 W10sIm9mZnNldCI 6MCwidGltZWZyYW 1lIjoiNjA0ODAwI iwiZ3JhcGhtb2Rl IjoiY291bnQiLCJ 0aW1lIjp7InVzZX JfaW50ZXJ2YWwiO jB9LCJzdGFtcCI6 MTM4NDU1NjE2NTY 2OCwibW9kZSI6Ii IsImFuYWx5emVfZ mllbGQiOiIifQ= =