Activity log for bug #1419452

Date Who What changed Old value New value Message
2015-02-08 14:29:06 Itzik Brown bug added bug
2015-02-08 15:39:09 Davanum Srinivas (DIMS) marked as duplicate 1324041
2015-02-08 23:33:03 Itzik Brown description Restarting nova-compute fails when there is an instance with a port where binding:vif_type=binding_failed nova-compute Version ====== How to reproduce =============== 1. Launch an instance. 2. Attach another interface to the instance using a second network # nova interface-attach --net-id <net-id> <server> 3. Restart nova-compute Version ====== python-nova-2014.2.1-14.el7ost.noarch python-novaclient-2.20.0-1.el7ost.noarch openstack-nova-common-2014.2.1-14.el7ost.noarch openstack-nova-compute-2014.2.1-14.el7ost.noarch nova-compute Log =============== 2015-02-08 16:00:27.695 5794 DEBUG nova.virt.libvirt.vif [-] vif_type=binding_failed instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone=None,cell_name=None,cleaned=False,config_drive='',created_at=2015-02-08T13:56:26Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,disable_terminate=False,display_description='vm1',display_name='vm1',ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,host='puma48.scl.lab.tlv.redhat.com',hostname='vm1',id=190,image_ref='c6e549fd-4ba2-4723-a7c9-c1fa13cd24f8',info_cache=InstanceInfoCache,instance_type_id=5,kernel_id='',key_data=None,key_name=None,launch_index=0,launched_at=2015-02-08T13:56:35Z,launched_on='puma48.scl.lab.tlv.redhat.com',locked=False,locked_by=None,memory_mb=2048,metadata=<?>,node='puma48.scl.lab.tlv.redhat.com',numa_topology=<?>,os_type=None,pci_devices=<?>,power_state=1,progress=0,project_id='a10d7b579ee546ada9a9e5b70cfb9a25',ramdisk_id='',reservation_id='r-bhwfpof7',root_device_name='/dev/vda',root_gb=20,scheduled_at=None,security_groups=<?>,shutdown_terminate=False,system_metadata=<?>,task_state=None,terminated_at=None,updated_at=2015-02-08T13:56:35Z,user_data=None,user_id='4a5d45f0d5cf402b99a6819e28a12466',uuid=a7106220-1d42-4c47-9c91-e2bc8ce0a2d3,vcpus=1,vm_mode=None,vm_state='active') vif=VIF({'profile': {}, 'ovs_interfaceid': None, 'network': Network({'bridge': None, 'subnets': [Subnet({'ips': [FixedIP({'meta': {}, 'version': 4, 'type': u'fixed', 'floating_ips': [], 'address': u'192.168.250.10'})], 'version': 4, 'meta': {u'dhcp_server': u'192.168.250.3'}, 'dns': [], 'routes': [], 'cidr': u'192.168.250.0/24', 'gateway': IP({'meta': {}, 'version': 4, 'type': u'gateway', 'address': u'192.168.250.1'})})], 'meta': {u'injected': False, u'tenant_id': u'a10d7b579ee546ada9a9e5b70cfb9a25'}, 'id': u'70f730d8-aced-48e8-a08e-0395fa446e28', 'label': u'net3'}), 'devname': u'tap1e546c5b-f5', 'vnic_type': u'normal', 'qbh_params': None, 'meta': {}, 'details': {}, 'address': u'fa:16:3e:d2:bd:fb', 'active': False, 'type': u'binding_failed', 'id': u'1e546c5b-f5b5-4221-bfb2-7be46a323ea9', 'qbg_params': None}) plug /usr/lib/python2.7/site-packages/nova/virt/libvirt/vif.py:531 2015-02-08 16:00:27.698 5794 ERROR nova.openstack.common.threadgroup [-] Unexpected vif_type=binding_failed 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup Traceback (most recent call last): 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/openstack/common/threadgroup.py", line 125, in wait 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup x.wait() 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/openstack/common/threadgroup.py", line 47, in wait 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup return self.thread.wait() 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 173, in wait 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup return self._exit_event.wait() 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 121, in wait 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup return hubs.get_hub().switch() 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 293, in switch 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup return self.greenlet.switch() 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 212, in main 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup result = function(*args, **kwargs) 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/openstack/common/service.py", line 492, in run_service 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup service.start() 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/service.py", line 164, in start 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup self.manager.init_host() 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1139, in init_host 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup self._init_instance(context, instance) 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 982, in _init_instance 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup self.driver.plug_vifs(instance, net_info) 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 948, in plug_vifs 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup self.vif_driver.plug(instance, vif) 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/vif.py", line 541, in plug 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup _("Unexpected vif_type=%s") % vif_type) 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup NovaException: Unexpected vif_type=binding_failed 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup Restarting nova-compute fails when there is an instance with a port where binding:vif_type=binding_failed nova-compute Version ====== How to reproduce =============== 1. Launch an instance. 2. Stop openvswitch agent on the compute node 3. Attach another interface to the instance using a second network     # nova interface-attach --net-id <net-id> <server> 4. Restart nova-compute Version ====== python-nova-2014.2.1-14.el7ost.noarch python-novaclient-2.20.0-1.el7ost.noarch openstack-nova-common-2014.2.1-14.el7ost.noarch openstack-nova-compute-2014.2.1-14.el7ost.noarch nova-compute Log =============== 2015-02-08 16:00:27.695 5794 DEBUG nova.virt.libvirt.vif [-] vif_type=binding_failed instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone=None,cell_name=None,cleaned=False,config_drive='',created_at=2015-02-08T13:56:26Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,disable_terminate=False,display_description='vm1',display_name='vm1',ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,host='puma48.scl.lab.tlv.redhat.com',hostname='vm1',id=190,image_ref='c6e549fd-4ba2-4723-a7c9-c1fa13cd24f8',info_cache=InstanceInfoCache,instance_type_id=5,kernel_id='',key_data=None,key_name=None,launch_index=0,launched_at=2015-02-08T13:56:35Z,launched_on='puma48.scl.lab.tlv.redhat.com',locked=False,locked_by=None,memory_mb=2048,metadata=<?>,node='puma48.scl.lab.tlv.redhat.com',numa_topology=<?>,os_type=None,pci_devices=<?>,power_state=1,progress=0,project_id='a10d7b579ee546ada9a9e5b70cfb9a25',ramdisk_id='',reservation_id='r-bhwfpof7',root_device_name='/dev/vda',root_gb=20,scheduled_at=None,security_groups=<?>,shutdown_terminate=False,system_metadata=<?>,task_state=None,terminated_at=None,updated_at=2015-02-08T13:56:35Z,user_data=None,user_id='4a5d45f0d5cf402b99a6819e28a12466',uuid=a7106220-1d42-4c47-9c91-e2bc8ce0a2d3,vcpus=1,vm_mode=None,vm_state='active') vif=VIF({'profile': {}, 'ovs_interfaceid': None, 'network': Network({'bridge': None, 'subnets': [Subnet({'ips': [FixedIP({'meta': {}, 'version': 4, 'type': u'fixed', 'floating_ips': [], 'address': u'192.168.250.10'})], 'version': 4, 'meta': {u'dhcp_server': u'192.168.250.3'}, 'dns': [], 'routes': [], 'cidr': u'192.168.250.0/24', 'gateway': IP({'meta': {}, 'version': 4, 'type': u'gateway', 'address': u'192.168.250.1'})})], 'meta': {u'injected': False, u'tenant_id': u'a10d7b579ee546ada9a9e5b70cfb9a25'}, 'id': u'70f730d8-aced-48e8-a08e-0395fa446e28', 'label': u'net3'}), 'devname': u'tap1e546c5b-f5', 'vnic_type': u'normal', 'qbh_params': None, 'meta': {}, 'details': {}, 'address': u'fa:16:3e:d2:bd:fb', 'active': False, 'type': u'binding_failed', 'id': u'1e546c5b-f5b5-4221-bfb2-7be46a323ea9', 'qbg_params': None}) plug /usr/lib/python2.7/site-packages/nova/virt/libvirt/vif.py:531 2015-02-08 16:00:27.698 5794 ERROR nova.openstack.common.threadgroup [-] Unexpected vif_type=binding_failed 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup Traceback (most recent call last): 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/openstack/common/threadgroup.py", line 125, in wait 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup x.wait() 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/openstack/common/threadgroup.py", line 47, in wait 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup return self.thread.wait() 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 173, in wait 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup return self._exit_event.wait() 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 121, in wait 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup return hubs.get_hub().switch() 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 293, in switch 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup return self.greenlet.switch() 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 212, in main 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup result = function(*args, **kwargs) 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/openstack/common/service.py", line 492, in run_service 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup service.start() 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/service.py", line 164, in start 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup self.manager.init_host() 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1139, in init_host 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup self._init_instance(context, instance) 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 982, in _init_instance 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup self.driver.plug_vifs(instance, net_info) 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 948, in plug_vifs 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup self.vif_driver.plug(instance, vif) 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/vif.py", line 541, in plug 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup _("Unexpected vif_type=%s") % vif_type) 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup NovaException: Unexpected vif_type=binding_failed 2015-02-08 16:00:27.698 5794 TRACE nova.openstack.common.threadgroup