libvirt error while attaching port to a VM

Bug #1634349 reported by Vedamurthy Joshi
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Juniper Openstack
Confirmed
High
Kiran
Trunk
Won't Fix
High
Kiran

Bug Description

Build : 3.2.0.0-2746
CoreLocation : /cs-shared/test_runs/nodeb2/jenkins-ubuntu-14-04_mitaka_Single_Node_sanity-75
cores : {}
LogsLocation : http://10.204.216.50/Docs/logs/3.2.0.0-2746_jenkins-ubuntu-14-04_mitaka_Single_Node_sanity-75/logs/
Report : http://10.204.216.50/Docs/logs/3.2.0.0-2746_jenkins-ubuntu-14-04_mitaka_Single_Node_sanity-75/junit-noframes.html
Topology :
DISTRO : "Ubuntu 14.04.4 LTS"
SKU : mitaka
Config Nodes : [u'nodeb2']
Control Nodes : [u'nodeb2']
Compute Nodes : [u'nodeb2']
Openstack Node : nodeb2
WebUI Node : nodeb2
Analytics Nodes : [u'nodeb2']
Physical Devices : [u'mx1', u"'mx1'"]

nova interface attach failed during sanity test. On later runs, the attach went fine.
Error in nova-compute log :

2016-10-16 05:42:28.636 3292 WARNING nova.network.neutronv2.api [req-646bfb54-4af3-4bb9-85e2-1d4cf6e078bb 3852ca3043be4c7cacafe943eb5df17f 76294a08f395461f9913b3414afd8446 - - -] [instance: 930a0380-8c90-4028-a08a-c64ab3bf5df0] 4f9d913a-347e-49bd-bfd7-6c6c559d287b already exists in list: networks containing: [u'4f9d913a-347e-49bd-bfd7-6c6c559d287b']. ignoring it
2016-10-16 05:42:59.565 3292 ERROR nova.virt.libvirt.driver [req-646bfb54-4af3-4bb9-85e2-1d4cf6e078bb 3852ca3043be4c7cacafe943eb5df17f 76294a08f395461f9913b3414afd8446 - - -] [instance: 930a0380-8c90-4028-a08a-c64ab3bf5df0] attaching network adapter failed.
2016-10-16 05:42:59.565 3292 ERROR nova.virt.libvirt.driver [instance: 930a0380-8c90-4028-a08a-c64ab3bf5df0] Traceback (most recent call last):
2016-10-16 05:42:59.565 3292 ERROR nova.virt.libvirt.driver [instance: 930a0380-8c90-4028-a08a-c64ab3bf5df0] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1508, in attach_interface
2016-10-16 05:42:59.565 3292 ERROR nova.virt.libvirt.driver [instance: 930a0380-8c90-4028-a08a-c64ab3bf5df0] guest.attach_device(cfg, persistent=True, live=live)
2016-10-16 05:42:59.565 3292 ERROR nova.virt.libvirt.driver [instance: 930a0380-8c90-4028-a08a-c64ab3bf5df0] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/guest.py", line 250, in attach_device
2016-10-16 05:42:59.565 3292 ERROR nova.virt.libvirt.driver [instance: 930a0380-8c90-4028-a08a-c64ab3bf5df0] self._domain.attachDeviceFlags(conf.to_xml(), flags=flags)
2016-10-16 05:42:59.565 3292 ERROR nova.virt.libvirt.driver [instance: 930a0380-8c90-4028-a08a-c64ab3bf5df0] File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 186, in doit
2016-10-16 05:42:59.565 3292 ERROR nova.virt.libvirt.driver [instance: 930a0380-8c90-4028-a08a-c64ab3bf5df0] result = proxy_call(self._autowrap, f, *args, **kwargs)
2016-10-16 05:42:59.565 3292 ERROR nova.virt.libvirt.driver [instance: 930a0380-8c90-4028-a08a-c64ab3bf5df0] File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 144, in proxy_call
2016-10-16 05:42:59.565 3292 ERROR nova.virt.libvirt.driver [instance: 930a0380-8c90-4028-a08a-c64ab3bf5df0] rv = execute(f, *args, **kwargs)
2016-10-16 05:42:59.565 3292 ERROR nova.virt.libvirt.driver [instance: 930a0380-8c90-4028-a08a-c64ab3bf5df0] File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 125, in execute
2016-10-16 05:42:59.565 3292 ERROR nova.virt.libvirt.driver [instance: 930a0380-8c90-4028-a08a-c64ab3bf5df0] six.reraise(c, e, tb)
2016-10-16 05:42:59.565 3292 ERROR nova.virt.libvirt.driver [instance: 930a0380-8c90-4028-a08a-c64ab3bf5df0] File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 83, in tworker
2016-10-16 05:42:59.565 3292 ERROR nova.virt.libvirt.driver [instance: 930a0380-8c90-4028-a08a-c64ab3bf5df0] rv = meth(*args, **kwargs)
2016-10-16 05:42:59.565 3292 ERROR nova.virt.libvirt.driver [instance: 930a0380-8c90-4028-a08a-c64ab3bf5df0] File "/usr/lib/python2.7/dist-packages/libvirt.py", line 560, in attachDeviceFlags
2016-10-16 05:42:59.565 3292 ERROR nova.virt.libvirt.driver [instance: 930a0380-8c90-4028-a08a-c64ab3bf5df0] if ret == -1: raise libvirtError ('virDomainAttachDeviceFlags() failed', dom=self)
2016-10-16 05:42:59.565 3292 ERROR nova.virt.libvirt.driver [instance: 930a0380-8c90-4028-a08a-c64ab3bf5df0] libvirtError: Timed out during operation: cannot acquire state change lock (held by remoteDispatchDomainDetachDeviceFlags)
2016-10-16 05:42:59.565 3292 ERROR nova.virt.libvirt.driver [instance: 930a0380-8c90-4028-a08a-c64ab3bf5df0]
2016-10-16 05:43:00.056 3292 WARNING nova.compute.manager [req-646bfb54-4af3-4bb9-85e2-1d4cf6e078bb 3852ca3043be4c7cacafe943eb5df17f 76294a08f395461f9913b3414afd8446 - - -] [instance: 930a0380-8c90-4028-a08a-c64ab3bf5df0] attach interface failed , try to deallocate port 9448cfb8-a2dd-46a6-bc51-c040307981e3, reason: Failed to attach network adapter device to 930a0380-8c90-4028-a08a-c64ab3bf5df0
2016-10-16 05:43:00.558 3292 INFO nova.network.neutronv2.api [req-646bfb54-4af3-4bb9-85e2-1d4cf6e078bb 3852ca3043be4c7cacafe943eb5df17f 76294a08f395461f9913b3414afd8446 - - -] [instance: 930a0380-8c90-4028-a08a-c64ab3bf5df0] Port 9448cfb8-a2dd-46a6-bc51-c040307981e3 from network info_cache is no longer associated with instance in Neutron. Removing from network info_cache.
2016-10-16 05:43:00.688 3292 ERROR oslo_messaging.rpc.dispatcher [req-646bfb54-4af3-4bb9-85e2-1d4cf6e078bb 3852ca3043be4c7cacafe943eb5df17f 76294a08f395461f9913b3414afd8446 - - -] Exception during message handling: Failed to attach network adapter device to 930a0380-8c90-4028-a08a-c64ab3bf5df0
2016-10-16 05:43:00.688 3292 ERROR oslo_messaging.rpc.dispatcher Traceback (most recent call last):
2016-10-16 05:43:00.688 3292 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 138, in _dispatch_and_reply
2016-10-16 05:43:00.688 3292 ERROR oslo_messaging.rpc.dispatcher incoming.message))
2016-10-16 05:43:00.688 3292 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 185, in _dispatch
2016-10-16 05:43:00.688 3292 ERROR oslo_messaging.rpc.dispatcher return self._do_dispatch(endpoint, method, ctxt, args)
2016-10-16 05:43:00.688 3292 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 127, in _do_dispatch
2016-10-16 05:43:00.688 3292 ERROR oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args)
2016-10-16 05:43:00.688 3292 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 110, in wrapped
2016-10-16 05:43:00.688 3292 ERROR oslo_messaging.rpc.dispatcher payload)
2016-10-16 05:43:00.688 3292 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-10-16 05:43:00.688 3292 ERROR oslo_messaging.rpc.dispatcher self.force_reraise()
2016-10-16 05:43:00.688 3292 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-10-16 05:43:00.688 3292 ERROR oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2016-10-16 05:43:00.688 3292 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 89, in wrapped
2016-10-16 05:43:00.688 3292 ERROR oslo_messaging.rpc.dispatcher return f(self, context, *args, **kw)
2016-10-16 05:43:00.688 3292 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 390, in decorated_function
2016-10-16 05:43:00.688 3292 ERROR oslo_messaging.rpc.dispatcher kwargs['instance'], e, sys.exc_info())
2016-10-16 05:43:00.688 3292 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-10-16 05:43:00.688 3292 ERROR oslo_messaging.rpc.dispatcher self.force_reraise()
2016-10-16 05:43:00.688 3292 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-10-16 05:43:00.688 3292 ERROR oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2016-10-16 05:43:00.688 3292 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 378, in decorated_function
2016-10-16 05:43:00.688 3292 ERROR oslo_messaging.rpc.dispatcher return function(self, context, *args, **kwargs)
2016-10-16 05:43:00.688 3292 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 5057, in attach_interface
2016-10-16 05:43:00.688 3292 ERROR oslo_messaging.rpc.dispatcher instance_uuid=instance.uuid)
2016-10-16 05:43:00.688 3292 ERROR oslo_messaging.rpc.dispatcher InterfaceAttachFailed: Failed to attach network adapter device to 930a0380-8c90-4028-a08a-c64ab3bf5df0
2016-10-16 05:43:00.688 3292 ERROR oslo_messaging.rpc.dispatcher
2016-10-16 05:43:00.691 3292 ERROR oslo_messaging._drivers.common [req-646bfb54-4af3-4bb9-85e2-1d4cf6e078bb 3852ca3043be4c7cacafe943eb5df17f 76294a08f395461f9913b3414afd8446 - - -] Returning exception Failed to attach network adapter device to 930a0380-8c90-4028-a08a-c64ab3bf5df0 to caller
2016-10-16 05:43:00.692 3292 ERROR oslo_messaging._drivers.common [req-646bfb54-4af3-4bb9-85e2-1d4cf6e078bb 3852ca3043be4c7cacafe943eb5df17f 76294a08f395461f9913b3414afd8446 - - -] ['Traceback (most recent call last):\n', ' File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 138, in _dispatch_and_reply\n incoming.message))\n', ' File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 185, in _dispatch\n return self._do_dispatch(endpoint, method, ctxt, args)\n', ' File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 127, in _do_dispatch\n result = func(ctxt, **new_args)\n', ' File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 110, in wrapped\n payload)\n', ' File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', ' File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', ' File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 89, in wrapped\n return f(self, context, *args, **kw)\n', ' File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 390, in decorated_function\n kwargs[\'instance\'], e, sys.exc_info())\n', ' File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', ' File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', ' File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 378, in decorated_function\n return function(self, context, *args, **kwargs)\n', ' File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 5057, in attach_interface\n instance_uuid=instance.uuid)\n', 'InterfaceAttachFailed: Failed to attach network adapter device to 930a0380-8c90-4028-a08a-c64ab3bf5df0\n']

Test run error log :

2016-10-16 05:43:17,617 - ERROR - Python 2.7.6: /usr/bin/python Sun Oct 16 05:43:00 2016 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /contrail-test/tcutils/wrappers.py in wrapper(self=, *args=(), **kwargs={}) 71 log.info('Initial checks done. Running the testcase now') 72 log.info('') 73 result = function(self, *args, **kwargs) 74 except KeyboardInterrupt: 75 raise result = None function = self = args = () kwargs = {} /contrail-test/scripts/neutron/test_ports.py in test_ports_attach_detach(self=) 82 # Now attach the interface again 83 port_obj = self.create_port(net_id=vn1_fixture.vn_id) 84 vm1_fixture.interface_attach(port_id=port_obj['id']) 85 time.sleep(5) 86 vm1_fixture.vm_obj.get() vm1_fixture = vm1_fixture.interface_attach = > port_id undefined port_obj = {u'admin_state_up': True, u'binding:host_id': None, u'binding:vif_details': {u'port_filter': True}, u'binding:vif_type': u'vrouter', u'binding:vnic_type': u'normal', u'device_id': u'', u'device_owner': u'', u'fixed_ips': [{u'ip_address': u'82.48.38.5', u'subnet_id': u'09b889cb-06ec-458f-950a-8f78858170bd'}], u'id': u'9448cfb8-a2dd-46a6-bc51-c040307981e3', u'mac_address': u'02:94:48:cf:b8:a2', ...} /contrail-test/fixtures/vm_test.py in interface_attach(self=, port_id=u'9448cfb8-a2dd-46a6-bc51-c040307981e3', net_id=None, fixed_ip=None) 2639 self.logger.info('Attaching port %s to VM %s' % 2640 (port_id, self.vm_obj.name)) 2641 return self.vm_obj.interface_attach(port_id, net_id, fixed_ip) 2642 2643 def interface_detach(self, port_id): self = self.vm_obj = self.vm_obj.interface_attach = > port_id = u'9448cfb8-a2dd-46a6-bc51-c040307981e3' net_id = None fixed_ip = None /usr/lib/python2.7/dist-packages/novaclient/v2/servers.py in interface_attach(self=, port_id=u'9448cfb8-a2dd-46a6-bc51-c040307981e3', net_id=None, fixed_ip=None) 515 Attach a network interface to an instance. 516 """ 517 return self.manager.interface_attach(self, port_id, net_id, fixed_ip) 518 519 def interface_detach(self, port_id): self = self.manager = self.manager.interface_attach = > port_id = u'9448cfb8-a2dd-46a6-bc51-c040307981e3' net_id = None fixed_ip = None /usr/lib/python2.7/dist-packages/novaclient/v2/servers.py in interface_attach(self=, server=, port_id=u'9448cfb8-a2dd-46a6-bc51-c040307981e3', net_id=None, fixed_ip=None) 1657 1658 return self._create('/servers/%s/os-interface' % base.getid(server), 1659 body, 'interfaceAttachment') 1660 1661 def interface_detach(self, server, port_id): body = {'interfaceAttachment': {'port_id': u'9448cfb8-a2dd-46a6-bc51-c040307981e3'}} /usr/lib/python2.7/dist-packages/novaclient/base.py in _create(self=, url=u'/servers/930a0380-8c90-4028-a08a-c64ab3bf5df0/os-interface', body={'interfaceAttachment': {'port_id': u'9448cfb8-a2dd-46a6-bc51-c040307981e3'}}, response_key='interfaceAttachment', return_raw=False, **kwargs={}) 343 def _create(self, url, body, response_key, return_raw=False, **kwargs): 344 self.run_hooks('modify_body_for_create', body, **kwargs) 345 resp, body = self.api.client.post(url, body=body) 346 if return_raw: 347 return self.convert_into_with_meta(body[response_key], resp) resp undefined body = {'interfaceAttachment': {'port_id': u'9448cfb8-a2dd-46a6-bc51-c040307981e3'}} self = self.api = self.api.client = self.api.client.post = > url = u'/servers/930a0380-8c90-4028-a08a-c64ab3bf5df0/os-interface' /usr/lib/python2.7/dist-packages/novaclient/client.py in post(self=, url=u'/servers/930a0380-8c90-4028-a08a-c64ab3bf5df0/os-interface', **kwargs={'body': {'interfaceAttachment': {'port_id': u'9448cfb8-a2dd-46a6-bc51-c040307981e3'}}}) 453 454 def post(self, url, **kwargs): 455 return self._cs_request(url, 'POST', **kwargs) 456 457 def put(self, url, **kwargs): self = self._cs_request = > url = u'/servers/930a0380-8c90-4028-a08a-c64ab3bf5df0/os-interface' kwargs = {'body': {'interfaceAttachment': {'port_id': u'9448cfb8-a2dd-46a6-bc51-c040307981e3'}}} /usr/lib/python2.7/dist-packages/novaclient/client.py in _cs_request(self=, url=u'http://10.204.216.33:8774/v2.1/76294a08f395461...30a0380-8c90-4028-a08a-c64ab3bf5df0/os-interface', method='POST', **kwargs={'body': {'interfaceAttachment': {'port_id': u'9448cfb8-a2dd-46a6-bc51-c040307981e3'}}, 'headers': {'Accept': 'application/json', 'Content-Type': 'application/json', 'User-Agent': 'python-novaclient', 'X-Auth-Project-Id': 'ctest-TestPorts-36924093', 'X-Auth-Token': u'70ac3b6a9c0142e2a36fc9d811aabdac'}}) 428 kwargs['headers']['X-Auth-Project-Id'] = self.projectid 429 430 resp, body = self._time_request(url, method, **kwargs) 431 return resp, body 432 except exceptions.Unauthorized as e: resp undefined body undefined self = self._time_request = > url = u'http://10.204.216.33:8774/v2.1/76294a08f395461...30a0380-8c90-4028-a08a-c64ab3bf5df0/os-interface' method = 'POST' kwargs = {'body': {'interfaceAttachment': {'port_id': u'9448cfb8-a2dd-46a6-bc51-c040307981e3'}}, 'headers': {'Accept': 'application/json', 'Content-Type': 'application/json', 'User-Agent': 'python-novaclient', 'X-Auth-Project-Id': 'ctest-TestPorts-36924093', 'X-Auth-Token': u'70ac3b6a9c0142e2a36fc9d811aabdac'}} /usr/lib/python2.7/dist-packages/novaclient/client.py in _time_request(self=, url=u'http://10.204.216.33:8774/v2.1/76294a08f395461...30a0380-8c90-4028-a08a-c64ab3bf5df0/os-interface', method='POST', **kwargs={'body': {'interfaceAttachment': {'port_id': u'9448cfb8-a2dd-46a6-bc51-c040307981e3'}}, 'headers': {'Accept': 'application/json', 'Content-Type': 'application/json', 'User-Agent': 'python-novaclient', 'X-Auth-Project-Id': 'ctest-TestPorts-36924093', 'X-Auth-Token': u'70ac3b6a9c0142e2a36fc9d811aabdac'}}) 401 def _time_request(self, url, method, **kwargs): 402 with utils.record_time(self.times, self.timings, method, url): 403 resp, body = self.request(url, method, **kwargs) 404 return resp, body 405 resp undefined body undefined self = self.request = > url = u'http://10.204.216.33:8774/v2.1/76294a08f395461...30a0380-8c90-4028-a08a-c64ab3bf5df0/os-interface' method = 'POST' kwargs = {'body': {'interfaceAttachment': {'port_id': u'9448cfb8-a2dd-46a6-bc51-c040307981e3'}}, 'headers': {'Accept': 'application/json', 'Content-Type': 'application/json', 'User-Agent': 'python-novaclient', 'X-Auth-Project-Id': 'ctest-TestPorts-36924093', 'X-Auth-Token': u'70ac3b6a9c0142e2a36fc9d811aabdac'}} /usr/lib/python2.7/dist-packages/novaclient/client.py in request(self=, url=u'http://10.204.216.33:8774/v2.1/76294a08f395461...30a0380-8c90-4028-a08a-c64ab3bf5df0/os-interface', method='POST', **kwargs={'data': '{"interfaceAttachment": {"port_id": "9448cfb8-a2dd-46a6-bc51-c040307981e3"}}', 'headers': {'Accept': 'application/json', 'Content-Type': 'application/json', 'User-Agent': 'python-novaclient', 'X-Auth-Project-Id': 'ctest-TestPorts-36924093', 'X-Auth-Token': u'70ac3b6a9c0142e2a36fc9d811aabdac'}, 'verify': False}) 395 if resp.headers else None) 396 if resp.status_code >= 400: 397 raise exceptions.from_response(resp, body, url, method) 398 399 return resp, body global exceptions = exceptions.from_response = resp = body = {u'computeFault': {u'code': 500, u'message': u'Failed to attach network adapter device to 930a0380-8c90-4028-a08a-c64ab3bf5df0'}} url = u'http://10.204.216.33:8774/v2.1/76294a08f395461...30a0380-8c90-4028-a08a-c64ab3bf5df0/os-interface' method = 'POST' : Failed to attach network adapter device to 930a0380-8c90-4028-a08a-c64ab3bf5df0 (HTTP 500) (Request-ID: req-646bfb54-4af3-4bb9-85e2-1d4cf6e078bb) __class__ = __delattr__ = __dict__ = {'code': 500, 'details': None, 'message': u'Failed to attach network adapter device to 930a0380-8c90-4028-a08a-c64ab3bf5df0', 'method': 'POST', 'request_id': 'req-646bfb54-4af3-4bb9-85e2-1d4cf6e078bb', 'url': u'http://10.204.216.33:8774/v2.1/76294a08f395461...30a0380-8c90-4028-a08a-c64ab3bf5df0/os-interface'} __doc__ = '\n The base exception class for all exceptions this library raises.\n ' __format__ = __getattribute__ = __getitem__ = __getslice__ = __hash__ = __init__ = __module__ = 'novaclient.exceptions' __new__ = __reduce__ = __reduce_ex__ = __repr__ = __setattr__ = __setstate__ = __sizeof__ = __str__ = __subclasshook__ = __unicode__ = __weakref__ = None args = () code = 500 details = None message = u'Failed to attach network adapter device to 930a0380-8c90-4028-a08a-c64ab3bf5df0' method = 'POST' request_id = 'req-646bfb54-4af3-4bb9-85e2-1d4cf6e078bb' url = u'http://10.204.216.33:8774/v2.1/76294a08f395461...30a0380-8c90-4028-a08a-c64ab3bf5df0/os-interface' The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "tcutils/wrappers.py", line 73, in wrapper result = function(self, *args, **kwargs) File "scripts/neutron/test_ports.py", line 84, in test_ports_attach_detach vm1_fixture.interface_attach(port_id=port_obj['id']) File "/contrail-test/fixtures/vm_test.py", line 2641, in interface_attach return self.vm_obj.interface_attach(port_id, net_id, fixed_ip) File "/usr/lib/python2.7/dist-packages/novaclient/v2/servers.py", line 517, in interface_attach return self.manager.interface_attach(self, port_id, net_id, fixed_ip) File "/usr/lib/python2.7/dist-packages/novaclient/v2/servers.py", line 1659, in interface_attach body, 'interfaceAttachment') File "/usr/lib/python2.7/dist-packages/novaclient/base.py", line 345, in _create resp, body = self.api.client.post(url, body=body) File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 455, in post return self._cs_request(url, 'POST', **kwargs) File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 430, in _cs_request resp, body = self._time_request(url, method, **kwargs) File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 403, in _time_request resp, body = self.request(url, method, **kwargs) File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 397, in request raise exceptions.from_response(resp, body, url, method) ClientException: Failed to attach network adapter device to 930a0380-8c90-4028-a08a-c64ab3bf5df0 (HTTP 500) (Request-ID: req-646bfb54-4af3-4bb9-85e2-1d4cf6e078bb) 2016-10-16 05:43:17,618 - INFO -
2016-10-16 05:43:17,618 - INFO - END TEST : test_ports_attach_detach : FAILED[0:02:39]

Jeba Paulaiyan (jebap)
tags: added: sanity
Revision history for this message
Kiran (kiran-kn80) wrote :

Vedamurthy,

I am unable to recreate this. Does this still happen? Also, I could not find the exact log. Pl close if it does not happen anymore.

Thanks

Revision history for this message
Vedamurthy Joshi (vedujoshi) wrote :

This continues to be seen even on 4.0 runs

http://10.204.216.50/Docs/logs/4.0.0.0-3_jenkins-SMLite_ubuntu-14-04_mitaka_Multi_Interface_Container_Sanity-13_1494596780.58/logs/

http://10.204.216.50/Docs/logs/4.0.0.0-3_jenkins-SMLite_ubuntu-14-04_mitaka_Multi_Interface_Container_Sanity-13_1494596780.58/logs//testports.html#test_ports_attach_detach

You could get all the logs from the nodes at /cs-shared/test_runs/nodec54/jenkins-SMLite_ubuntu-14-04_mitaka_Multi_Interface_Container_Sanity-13 on nodem2.englab.juniper.net(you can use your unix account to login)

Jeba Paulaiyan (jebap)
tags: added: blocker
tags: removed: blocker
Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] master

Review in progress for https://review.opencontrail.org/33867
Submitter: Vedamurthy Joshi (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R4.0

Review in progress for https://review.opencontrail.org/33868
Submitter: Vedamurthy Joshi (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R3.2

Review in progress for https://review.opencontrail.org/33869
Submitter: Vedamurthy Joshi (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R3.2.3.x

Review in progress for https://review.opencontrail.org/33870
Submitter: Vedamurthy Joshi (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/33867
Committed: http://github.com/Juniper/contrail-test/commit/f8862a32cfc82c4aa9e91c33c47bfc627873105f
Submitter: Zuul (<email address hidden>)
Branch: master

commit f8862a32cfc82c4aa9e91c33c47bfc627873105f
Author: Vedu Joshi <email address hidden>
Date: Fri Jul 21 11:55:23 2017 +0530

Add sleep as a workaround for bug 1634349

Change-Id: I5eb0338fc6cc7905efd3eacc608df041a62247a7

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/33870
Committed: http://github.com/Juniper/contrail-test/commit/233cf2f0311a936ae97eaaeddc34fb2a0bac024e
Submitter: Zuul (<email address hidden>)
Branch: R3.2.3.x

commit 233cf2f0311a936ae97eaaeddc34fb2a0bac024e
Author: Vedu Joshi <email address hidden>
Date: Fri Jul 21 11:55:23 2017 +0530

Add sleep as a workaround for bug 1634349

Change-Id: I5eb0338fc6cc7905efd3eacc608df041a62247a7

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/33869
Committed: http://github.com/Juniper/contrail-test/commit/88320614382b22de533ec8e0bc5da860da9cf780
Submitter: Zuul (<email address hidden>)
Branch: R3.2

commit 88320614382b22de533ec8e0bc5da860da9cf780
Author: Vedu Joshi <email address hidden>
Date: Fri Jul 21 11:55:23 2017 +0530

Add sleep as a workaround for bug 1634349

Change-Id: I5eb0338fc6cc7905efd3eacc608df041a62247a7

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/33868
Committed: http://github.com/Juniper/contrail-test/commit/e32d24abd8dcfafd56334f76440c415076da9ce4
Submitter: Zuul (<email address hidden>)
Branch: R4.0

commit e32d24abd8dcfafd56334f76440c415076da9ce4
Author: Vedu Joshi <email address hidden>
Date: Fri Jul 21 11:55:23 2017 +0530

Add sleep as a workaround for bug 1634349

Change-Id: I5eb0338fc6cc7905efd3eacc608df041a62247a7

Revision history for this message
Vedamurthy Joshi (vedujoshi) wrote :

Removing the assignee since i only added a workaround to the test scripts to avoid hitting the bug

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R4.0

Review in progress for https://review.opencontrail.org/34544
Submitter: Vedamurthy Joshi (<email address hidden>)

Rudra Rugge (rrugge)
no longer affects: juniperopenstack/r4.0
no longer affects: juniperopenstack/trunk
Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/34544
Committed: http://github.com/Juniper/contrail-test/commit/180a9004ececf3ef9546d74a656a284de37851f3
Submitter: Zuul (<email address hidden>)
Branch: R4.0

commit 180a9004ececf3ef9546d74a656a284de37851f3
Author: Vedu Joshi <email address hidden>
Date: Mon Aug 14 13:53:24 2017 +0530

Add WA for bug 1634349

Add WA for bug 1634349 in a second place

Change-Id: I69ae723d70d72cbef1fd52cd9a9b81b68d656946

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] master

Review in progress for https://review.opencontrail.org/34701
Submitter: Vedamurthy Joshi (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/34701
Committed: http://github.com/Juniper/contrail-test/commit/9ac58f8a82bc4a7824c26e483356c4dfde6fb221
Submitter: Zuul (<email address hidden>)
Branch: master

commit 9ac58f8a82bc4a7824c26e483356c4dfde6fb221
Author: Vedu Joshi <email address hidden>
Date: Mon Aug 14 13:53:24 2017 +0530

Add WA for bug 1634349

Add WA for bug 1634349 in a second place

Change-Id: I69ae723d70d72cbef1fd52cd9a9b81b68d656946
(cherry picked from commit 180a9004ececf3ef9546d74a656a284de37851f3)

Revision history for this message
Vedamurthy Joshi (vedujoshi) wrote :
Revision history for this message
Kiran (kiran-kn80) wrote :

         # Now attach the interface again
         port_obj = self.create_port(net_id=vn1_fixture.vn_id)
+ self.sleep(5)
         vm1_fixture.interface_attach(port_id=port_obj['id'])
         time.sleep(5)
         vm1_fixture.vm_obj.get()

Vedu,
Is self.sleep(5) and time.sleep(5) same?

tags: added: sanityblocker
removed: sanity
Rudra Rugge (rrugge)
tags: added: sanity
removed: sanityblocker
Revision history for this message
Rudra Rugge (rrugge) wrote :

We will not target libvirt fix for detach and then attach use cases. VM delete should trigger deletion of the port.

no longer affects: juniperopenstack/r3.2
no longer affects: juniperopenstack/r3.2.3.x
no longer affects: juniperopenstack/r4.0
Kiran (kiran-kn80)
Changed in juniperopenstack:
milestone: r4.1.0.0-fcs → none
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.