http://logs.openstack.org/50/524750/1/check/legacy-tempest-dsvm-neutron-full/eb8d805/logs/screen-n-api.txt.gz?level=TRACE#_Dec_04_13_34_20_635874
Dec 04 13:34:20.635874 ubuntu-xenial-inap-mtl01-0001196572 <email address hidden>[6808]: ERROR nova.api.openstack.extensions [None req-2d1b063f-1324-4498-af68-ce48c6d8e5a3 tempest-AttachInterfacesTestJSON-149718191 tempest-AttachInterfacesTestJSON-149718191] Unexpected exception in API method: TypeError: 'NoneType' object has no attribute '__getitem__'
Dec 04 13:34:20.636066 ubuntu-xenial-inap-mtl01-0001196572 <email address hidden>[6808]: Traceback (most recent call last):
Dec 04 13:34:20.636202 ubuntu-xenial-inap-mtl01-0001196572 <email address hidden>[6808]: File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 163, in _process_incoming
Dec 04 13:34:20.636336 ubuntu-xenial-inap-mtl01-0001196572 <email address hidden>[6808]: res = self.dispatcher.dispatch(message)
Dec 04 13:34:20.636474 ubuntu-xenial-inap-mtl01-0001196572 <email address hidden>[6808]: File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 220, in dispatch
Dec 04 13:34:20.636614 ubuntu-xenial-inap-mtl01-0001196572 <email address hidden>[6808]: return self._do_dispatch(endpoint, method, ctxt, args)
Dec 04 13:34:20.636745 ubuntu-xenial-inap-mtl01-0001196572 <email address hidden>[6808]: File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 190, in _do_dispatch
Dec 04 13:34:20.636892 ubuntu-xenial-inap-mtl01-0001196572 <email address hidden>[6808]: result = func(ctxt, **new_args)
Dec 04 13:34:20.637049 ubuntu-xenial-inap-mtl01-0001196572 <email address hidden>[6808]: File "/opt/stack/new/nova/nova/exception_wrapper.py", line 76, in wrapped
Dec 04 13:34:20.637187 ubuntu-xenial-inap-mtl01-0001196572 <email address hidden>[6808]: function_name, call_dict, binary)
Dec 04 13:34:20.637317 ubuntu-xenial-inap-mtl01-0001196572 <email address hidden>[6808]: File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
Dec 04 13:34:20.637442 ubuntu-xenial-inap-mtl01-0001196572 <email address hidden>[6808]: self.force_reraise()
Dec 04 13:34:20.637607 ubuntu-xenial-inap-mtl01-0001196572 <email address hidden>[6808]: File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
Dec 04 13:34:20.637761 ubuntu-xenial-inap-mtl01-0001196572 <email address hidden>[6808]: six.reraise(self.type_, self.value, self.tb)
Dec 04 13:34:20.637895 ubuntu-xenial-inap-mtl01-0001196572 <email address hidden>[6808]: File "/opt/stack/new/nova/nova/exception_wrapper.py", line 67, in wrapped
Dec 04 13:34:20.638044 ubuntu-xenial-inap-mtl01-0001196572 <email address hidden>[6808]: return f(self, context, *args, **kw)
Dec 04 13:34:20.638183 ubuntu-xenial-inap-mtl01-0001196572 <email address hidden>[6808]: File "/opt/stack/new/nova/nova/compute/utils.py", line 930, in decorated_function
Dec 04 13:34:20.638306 ubuntu-xenial-inap-mtl01-0001196572 <email address hidden>[6808]: return function(self, context, *args, **kwargs)
Dec 04 13:34:20.638433 ubuntu-xenial-inap-mtl01-0001196572 <email address hidden>[6808]: File "/opt/stack/new/nova/nova/compute/manager.py", line 215, in decorated_function
Dec 04 13:34:20.638566 ubuntu-xenial-inap-mtl01-0001196572 <email address hidden>[6808]: kwargs['instance'], e, sys.exc_info())
Dec 04 13:34:20.638696 ubuntu-xenial-inap-mtl01-0001196572 <email address hidden>[6808]: File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
Dec 04 13:34:20.638820 ubuntu-xenial-inap-mtl01-0001196572 <email address hidden>[6808]: self.force_reraise()
Dec 04 13:34:20.638953 ubuntu-xenial-inap-mtl01-0001196572 <email address hidden>[6808]: File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
Dec 04 13:34:20.639076 ubuntu-xenial-inap-mtl01-0001196572 <email address hidden>[6808]: six.reraise(self.type_, self.value, self.tb)
Dec 04 13:34:20.639200 ubuntu-xenial-inap-mtl01-0001196572 <email address hidden>[6808]: File "/opt/stack/new/nova/nova/compute/manager.py", line 203, in decorated_function
Dec 04 13:34:20.639325 ubuntu-xenial-inap-mtl01-0001196572 <email address hidden>[6808]: return function(self, context, *args, **kwargs)
Dec 04 13:34:20.639462 ubuntu-xenial-inap-mtl01-0001196572 <email address hidden>[6808]: File "/opt/stack/new/nova/nova/compute/manager.py", line 5563, in attach_interface
Dec 04 13:34:20.639607 ubuntu-xenial-inap-mtl01-0001196572 <email address hidden>[6808]: phase=fields.NotificationPhase.END)
Dec 04 13:34:20.639735 ubuntu-xenial-inap-mtl01-0001196572 <email address hidden>[6808]: File "/opt/stack/new/nova/nova/rpc.py", line 225, in wrapped
Dec 04 13:34:20.639856 ubuntu-xenial-inap-mtl01-0001196572 <email address hidden>[6808]: return f(*args, **kwargs)
Dec 04 13:34:20.639991 ubuntu-xenial-inap-mtl01-0001196572 <email address hidden>[6808]: File "/opt/stack/new/nova/nova/compute/utils.py", line 370, in notify_about_instance_action
Dec 04 13:34:20.640566 ubuntu-xenial-inap-mtl01-0001196572 <email address hidden>[6808]: bdms=bdms)
Dec 04 13:34:20.640746 ubuntu-xenial-inap-mtl01-0001196572 <email address hidden>[6808]: File "/opt/stack/new/nova/nova/notifications/objects/instance.py", line 140, in __init__
Dec 04 13:34:20.640884 ubuntu-xenial-inap-mtl01-0001196572 <email address hidden>[6808]: bdms=bdms)
Dec 04 13:34:20.641010 ubuntu-xenial-inap-mtl01-0001196572 <email address hidden>[6808]: File "/opt/stack/new/nova/nova/notifications/objects/instance.py", line 114, in __init__
Dec 04 13:34:20.641135 ubuntu-xenial-inap-mtl01-0001196572 <email address hidden>[6808]: self.ip_addresses = IpPayload.from_network_info(network_info)
Dec 04 13:34:20.641272 ubuntu-xenial-inap-mtl01-0001196572 <email address hidden>[6808]: File "/opt/stack/new/nova/nova/notifications/objects/instance.py", line 280, in from_network_info
Dec 04 13:34:20.641413 ubuntu-xenial-inap-mtl01-0001196572 <email address hidden>[6808]: for ip in vif.fixed_ips():
Dec 04 13:34:20.641559 ubuntu-xenial-inap-mtl01-0001196572 <email address hidden>[6808]: File "/opt/stack/new/nova/nova/network/model.py", line 415, in fixed_ips
Dec 04 13:34:20.641683 ubuntu-xenial-inap-mtl01-0001196572 <email address hidden>[6808]: return [fixed_ip for subnet in self['network']['subnets']
Dec 04 13:34:20.641824 ubuntu-xenial-inap-mtl01-0001196572 <email address hidden>[6808]: TypeError: 'NoneType' object has no attribute '__getitem__'
The instance info cache is probably not up to date at this point, or we're just being sloppy about checking that fields are set.
For some reason I'm only seeing this in one change right now, so maybe I just got unlucky but does seem like something we need to handle regardless:
http:// logstash. openstack. org/#dashboard/ file/logstash. json?query= message% 3A%5C%22return% 20%5Bfixed_ ip%20for% 20subnet% 20in%20self% 5B'network' %5D%5B' subnets' %5D%5C% 22%20AND% 20tags% 3A%5C%22screen- n-api.txt% 5C%22&from= 7d