virtual_machine_interface_bindings KeyError while when fatflow config is added to a port or on in-release upgrade

Bug #1540856 reported by Vedamurthy Joshi
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Juniper Openstack
Status tracked in Trunk
Trunk
Fix Committed
High
Sachin Bansal

Bug Description

R3.0 2709 Ubuntu 14.04 Kilo multi-node setup

Added fat flow config for tcp port 22. Then port-list starts to fail

In-release Upgrade from 3704 to 3708 too causes a similar keyerror on virtual_machine_interface_bindings

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/bottle.py", line 856, in _handle
    return route.call(**args)
  File "/usr/lib/python2.7/dist-packages/bottle.py", line 1721, in wrapper
    rv = callback(*a, **ka)
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_api_server.py", line 1532, in handler_trap_exception
    response = handler(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/cfgm_common/vnc_api_stats.py", line 17, in wrapper
    response = func(api_server_obj, resource_type, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_api_server.py", line 562, in http_resource_read
    (ok, result) = db_conn.dbe_read(resource_type, obj_ids, obj_fields)
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_ifmap.py", line 1873, in dbe_read
    obj_type, [obj_ids['uuid']], obj_fields)
  File "/usr/lib/python2.7/dist-packages/cfgm_common/vnc_cassandra.py", line 416, in object_read
    obj_class.prop_list_field_has_wrappers[prop_name]
KeyError: 'virtual_machine_interface_bindings'
127.0.0.1 - - [2016-02-02 16:13:21] "GET /virtual-machine-interface/559bc3fb-db99-4dee-9f6b-ff65f3716db3 HTTP/1.1" 500 156 0.005316

127.0.0.1 - - [2016-02-02 16:13:21] "GET /virtual-machine-interface/559bc3fb-db99-4dee-9f6b-ff65f3716db3 HTTP/1.1" 500 156 0.005316
ERROR:contrail-api:__default__ [SYS_ERR]: VncApiError: <type 'exceptions.KeyError'>
Python 2.7.6: /usr/bin/python
Tue Feb 2 16:13:21 2016

A problem occurred in a Python script. Here is the sequence of
function calls leading up to the error, in the order they occurred.

 /usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_api_server.py in handler_trap_exception(*args=(), **kwargs={'id': '559bc3fb-db99-4dee-9f6b-ff65f3716db3'})
 1530 (code, err_msg) = status
 1531 raise cfgm_common.exceptions.HttpError(code, err_msg)
 1532 response = handler(*args, **kwargs)
 1533 self._generate_rest_api_response_trace(trace, response)
 1534
response undefined
handler = <functools.partial object>
args = ()
kwargs = {'id': '559bc3fb-db99-4dee-9f6b-ff65f3716db3'}

 /usr/lib/python2.7/dist-packages/cfgm_common/vnc_api_stats.py in wrapper(api_server_obj=<vnc_cfg_api_server.vnc_cfg_api_server.VncApiServer object>, resource_type='virtual-machine-interface', *args=(), **kwargs={'id': '559bc3fb-db99-4dee-9f6b-ff65f3716db3'})
   15 statistics = VncApiStatistics(
   16 obj_type=resource_type.replace('-', '_'))
   17 response = func(api_server_obj, resource_type, *args, **kwargs)
   18 statistics.response_size = len(str(response))
   19 statistics.response_code = bottle.response.status_code
response undefined
func = <function http_resource_read>
api_server_obj = <vnc_cfg_api_server.vnc_cfg_api_server.VncApiServer object>
resource_type = 'virtual-machine-interface'
args = ()
kwargs = {'id': '559bc3fb-db99-4dee-9f6b-ff65f3716db3'}
 /usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_api_server.py in http_resource_read(self=<vnc_cfg_api_server.vnc_cfg_api_server.VncApiServer object>, resource_type='virtual-machine-interface', id='559bc3fb-db99-4dee-9f6b-ff65f3716db3')
  560
  561 try:
  562 (ok, result) = db_conn.dbe_read(resource_type, obj_ids, obj_fields)
  563 if not ok:
  564 self.config_object_error(id, None, obj_type, 'http_get', result)
ok = True
result = ''
db_conn = <vnc_cfg_api_server.vnc_cfg_ifmap.VncDbClient object>
db_conn.dbe_read = <bound method VncDbClient.dbe_read of <vnc_cfg_api_server.vnc_cfg_ifmap.VncDbClient object>>
resource_type = 'virtual-machine-interface'
obj_ids = {'uuid': '559bc3fb-db99-4dee-9f6b-ff65f3716db3'}
obj_fields = [u'ecmp_hashing_include_fields', u'virtual_machine_interface_host_routes', u'virtual_machine_interface_mac_addresses', u'virtual_machine_interface_dhcp_option_list', u'virtual_machine_interface_bindings', u'virtual_machine_interface_allowed_address_pairs', u'virtual_machine_interface_fat_flow_protocols', u'perms2', u'id_perms', u'virtual_machine_interface_device_owner', u'vrf_assign_table', u'virtual_machine_interface_properties', u'display_name', 'service_health_check_refs', 'routing_instance_refs', u'security_group_refs', u'physical_interface_refs', u'port_tuple_refs', 'interface_route_table_refs', 'virtual_machine_interface_refs', ...]

 /usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_ifmap.py in dbe_read(self=<vnc_cfg_api_server.vnc_cfg_ifmap.VncDbClient object>, obj_type='virtual-machine-interface', obj_ids={'uuid': '559bc3fb-db99-4dee-9f6b-ff65f3716db3'}, obj_fields=[u'ecmp_hashing_include_fields', u'virtual_machine_interface_host_routes', u'virtual_machine_interface_mac_addresses', u'virtual_machine_interface_dhcp_option_list', u'virtual_machine_interface_bindings', u'virtual_machine_interface_allowed_address_pairs', u'virtual_machine_interface_fat_flow_protocols', u'perms2', u'id_perms', u'virtual_machine_interface_device_owner', u'vrf_assign_table', u'virtual_machine_interface_properties', u'display_name', 'service_health_check_refs', 'routing_instance_refs', u'security_group_refs', u'physical_interface_refs', u'port_tuple_refs', 'interface_route_table_refs', 'virtual_machine_interface_refs', ...])
 1871 try:
 1872 (ok, cassandra_result) = self._cassandra_db.object_read(
 1873 obj_type, [obj_ids['uuid']], obj_fields)
 1874 except NoIdError as e:
 1875 # if NoIdError is for obj itself (as opposed to say for parent
obj_type = 'virtual-machine-interface'
obj_ids = {'uuid': '559bc3fb-db99-4dee-9f6b-ff65f3716db3'}
obj_fields = [u'ecmp_hashing_include_fields', u'virtual_machine_interface_host_routes', u'virtual_machine_interface_mac_addresses', u'virtual_machine_interface_dhcp_option_list', u'virtual_machine_interface_bindings', u'virtual_machine_interface_allowed_address_pairs', u'virtual_machine_interface_fat_flow_protocols', u'perms2', u'id_perms', u'virtual_machine_interface_device_owner', u'vrf_assign_table', u'virtual_machine_interface_properties', u'display_name', 'service_health_check_refs', 'routing_instance_refs', u'security_group_refs', u'physical_interface_refs', u'port_tuple_refs', 'interface_route_table_refs', 'virtual_machine_interface_refs', ...]

 /usr/lib/python2.7/dist-packages/cfgm_common/vnc_cassandra.py in object_read(self=<vnc_cfg_api_server.vnc_cfg_ifmap.VncServerCassandraClient object>, res_type='virtual-machine-interface', obj_uuids=['559bc3fb-db99-4dee-9f6b-ff65f3716db3'], field_names=[u'ecmp_hashing_include_fields', u'virtual_machine_interface_host_routes', u'virtual_machine_interface_mac_addresses', u'virtual_machine_interface_dhcp_option_list', u'virtual_machine_interface_bindings', u'virtual_machine_interface_allowed_address_pairs', u'virtual_machine_interface_fat_flow_protocols', u'perms2', u'id_perms', u'virtual_machine_interface_device_owner', u'vrf_assign_table', u'virtual_machine_interface_properties', u'display_name', 'service_health_check_refs', 'routing_instance_refs', u'security_group_refs', u'physical_interface_refs', u'port_tuple_refs', 'interface_route_table_refs', 'virtual_machine_interface_refs', ...])
  414 if self._re_match_prop_list.match(col_name):
  415 has_wrapper = \
  416 obj_class.prop_list_field_has_wrappers[prop_name]
  417 else:
  418 has_wrapper = \
obj_class = <class 'vnc_api.gen.resource_client.VirtualMachineInterface'>
obj_class.prop_list_field_has_wrappers = {'virtual_machine_interface_fat_flow_protocols': True}
prop_name = 'virtual_machine_interface_bindings'
<type 'exceptions.KeyError'>: 'virtual_machine_interface_bindings'
    __class__ = <type 'exceptions.KeyError'>
    __delattr__ = <method-wrapper '__delattr__' of exceptions.KeyError object>
    __dict__ = {}
    __doc__ = 'Mapping key not found.'
    __format__ = <built-in method __format__ of exceptions.KeyError object>
    __getattribute__ = <method-wrapper '__getattribute__' of exceptions.KeyError object>
    __getitem__ = <method-wrapper '__getitem__' of exceptions.KeyError object>
    __getslice__ = <method-wrapper '__getslice__' of exceptions.KeyError object>
    __hash__ = <method-wrapper '__hash__' of exceptions.KeyError object>
    __init__ = <method-wrapper '__init__' of exceptions.KeyError object>
    __new__ = <built-in method __new__ of type object>
    __reduce__ = <built-in method __reduce__ of exceptions.KeyError object>
    __reduce_ex__ = <built-in method __reduce_ex__ of exceptions.KeyError object>
    __repr__ = <method-wrapper '__repr__' of exceptions.KeyError object>
    __setattr__ = <method-wrapper '__setattr__' of exceptions.KeyError object>
    __setstate__ = <built-in method __setstate__ of exceptions.KeyError object>
    __sizeof__ = <built-in method __sizeof__ of exceptions.KeyError object>
    __str__ = <method-wrapper '__str__' of exceptions.KeyError object>
    __subclasshook__ = <built-in method __subclasshook__ of type object>
    __unicode__ = <built-in method __unicode__ of exceptions.KeyError object>
    args = ('virtual_machine_interface_bindings',)
    message = 'virtual_machine_interface_bindings'

The above is a description of an error in a Python program. Here is
the original traceback:

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_api_server.py", line 1532, in handler_trap_exception
    response = handler(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/cfgm_common/vnc_api_stats.py", line 17, in wrapper
    response = func(api_server_obj, resource_type, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_api_server.py", line 562, in http_resource_read
    (ok, result) = db_conn.dbe_read(resource_type, obj_ids, obj_fields)
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_ifmap.py", line 1873, in dbe_read
    obj_type, [obj_ids['uuid']], obj_fields)
  File "/usr/lib/python2.7/dist-packages/cfgm_common/vnc_cassandra.py", line 416, in object_read
    obj_class.prop_list_field_has_wrappers[prop_name]
KeyError: 'virtual_machine_interface_bindings'

127.0.0.1 - - [2016-02-02 16:13:25] "GET /virtual-machine-interface/cea2a148-43f0-4a69-ba82-32f0332653c5?fields=logical_router_back_refs%2Cinstance_ip_back_refs%2Cfloating_ip_back_refs HTTP/1.1" 200 2877 0.004334

Tags: blocker config
tags: added: blocker
Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] master

Review in progress for https://review.opencontrail.org/16812
Submitter: Sachin Bansal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/16812
Committed: http://github.org/Juniper/contrail-controller/commit/e09a1f20ee6ff6bc28812939bdc350c5d926c260
Submitter: Zuul
Branch: master

commit e09a1f20ee6ff6bc28812939bdc350c5d926c260
Author: Sachin Bansal <email address hidden>
Date: Wed Feb 3 00:08:41 2016 -0800

prop map must be written as map

At one place, it was being written as list. Fixed that.
Closes-Bug: 1540856

Change-Id: I098cf1761b9b0ff669a42bf6d3486bd770065cf7

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.