Contrail-Security:4.1:firewall policy rule creation fails

[Venkatesh Velpula]

=================================
build-id. :5(4.1.0.0-5)
build type :official
DISTRO :Ubuntu 16.04.2 LTS
SKU :newton
=================================

While creating the firewall rule, seeing the below error in the UI

Error: <type 'exceptions.KeyError'> Python 2.7.12: /usr/bin/python Tue Nov 28 11:06:49 2017 A problem occurred in a Python script.
Here is the sequence of function calls leading up to the error, in the order they occurred. /usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_api_server.py in http_resource_create(self=<vnc_cfg_api_server.vnc_cfg_api_server.VncApiServer object>, obj_type='firewall_rule') 636 637 try: 638 ok, result = stateful_create() 639 except Exception as e: 640 ok = False ok = False result = 'f759b38e-732d-4ae9-a639-56b793057034' stateful_create = <function stateful_create> /usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_api_server.py in stateful_create() 580 # type-specific hook 581 (ok, result) = r_class.pre_dbe_create( 582 tenant_name, obj_dict, db_conn) 583 if not ok: 584 return (ok, result) tenant_name = u'admin' obj_dict = {'action_list': {'simple_action': 'pass'}, 'direction': '<>', 'display_name': 'f759b38e-732d-4ae9-a639-56b793057034', 'endpoint_1': {'address_group': None, 'any': None, 'tags': ['global:tier=WEB'], 'virtual_network': None}, 'endpoint_2': {'address_group': None, 'any': None, 'tags': ['global:tier=APP'], 'virtual_network': None}, 'fq_name': ['default-policy-management', 'f759b38e-732d-4ae9-a639-56b793057034'], 'id_perms': {u'created': None, u'creator': None, u'description': None, u'enable': True, u'last_modified': None, u'permissions': {u'group': u'admin', u'group_access': 7, u'other_access': 7, u'owner': u'admin', u'owner_access': 7}, u'user_visible': True, u'uuid': {'uuid_lslong': 11977700030414155828L, 'uuid_mslong': 17823474424694524649L}}, 'match_tags': {'tag_list': ['site']}, 'name': 'f759b38e-732d-4ae9-a639-56b793057034', 'parent_type': 'policy-management', ...} db_conn = <vnc_cfg_api_server.vnc_db.VncDbClient object> /usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_types.py in pre_dbe_create(cls=<class 'vnc_cfg_api_server.vnc_cfg_types.FirewallRuleServer'>, tenant_name=u'admin', obj_dict={'action_list': {'simple_action': 'pass'}, 'direction': '<>', 'display_name': 'f759b38e-732d-4ae9-a639-56b793057034', 'endpoint_1': {'address_group': None, 'any': None, 'tags': ['global:tier=WEB'], 'virtual_network': None}, 'endpoint_2': {'address_group': None, 'any': None, 'tags': ['global:tier=APP'], 'virtual_network': None}, 'fq_name': ['default-policy-management', 'f759b38e-732d-4ae9-a639-56b793057034'], 'id_perms': {u'created': None, u'creator': None, u'description': None, u'enable': True, u'last_modified': None, u'permissions': {u'group': u'admin', u'group_access': 7, u'other_access': 7, u'owner': u'admin', u'owner_access': 7}, u'user_visible': True, u'uuid': {'uuid_lslong': 11977700030414155828L, 'uuid_mslong': 17823474424694524649L}}, 'match_tags': {'tag_list': ['site']}, 'name': 'f759b38e-732d-4ae9-a639-56b793057034', 'parent_type': 'policy-management', ...}, db_conn=<vnc_cfg_api_server.vnc_db.VncDbClient object>) 2028 obj_dict['fq_name'], 2029 obj_dict, 2030 ServiceGroupServer.object_type, 2031 ) 2032 if not ok: global ServiceGroupServer = <class 'vnc_cfg_api_server.vnc_cfg_types.ServiceGroupServer'> ServiceGroupServer.object_type = 'service_group' /usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_types.py in check_associated_firewall_resource_in_same_scope(cls=<class 'vnc_cfg_api_server.vnc_cfg_types.FirewallRuleServer'>, id='f759b38e-732d-4ae9-a639-56b793057034', fq_name=['default-policy-management', 'f759b38e-732d-4ae9-a639-56b793057034'], obj_dict={'action_list': {'simple_action': 'pass'}, 'direction': '<>', 'display_name': 'f759b38e-732d-4ae9-a639-56b793057034', 'endpoint_1': {'address_group': None, 'any': None, 'tags': ['global:tier=WEB'], 'virtual_network': None}, 'endpoint_2': {'address_group': None, 'any': None, 'tags': ['global:tier=APP'], 'virtual_network': None}, 'fq_name': ['default-policy-management', 'f759b38e-732d-4ae9-a639-56b793057034'], 'id_perms': {u'created': None, u'creator': None, u'description': None, u'enable': True, u'last_modified': None, u'permissions': {u'group': u'admin', u'group_access': 7, u'other_access': 7, u'owner': u'admin', u'owner_access': 7}, u'user_visible': True, u'uuid': {'uuid_lslong': 11977700030414155828L, 'uuid_mslong': 17823474424694524649L}}, 'match_tags': {'tag_list': ['site']}, 'name': 'f759b38e-732d-4ae9-a639-56b793057034', 'parent_type': 'policy-management', ...}, object_type='service_group') 230 231 for r_ref in obj_dict.get(ref_name, []): 232 ok, result = cls.dbe_read(cls.db_conn, object_type, r_ref['uuid'], 233 obj_fields=['parent_type']) 234 if not ok: ok undefined result undefined cls = <class 'vnc_cfg_api_server.vnc_cfg_types.FirewallRuleServer'> cls.dbe_read = <bound method __metaclass__.dbe_read of <class '...fg_api_server.vnc_cfg_types.FirewallRuleServer'>> cls.db_conn = <vnc_cfg_api_server.vnc_db.VncDbClient object> object_type = 'service_group' r_ref = {'to': ['default-policy-management', 'sg1']} obj_fields undefined <type 'exceptions.KeyError'>: 'uuid' __class__ = <type 'exceptions.KeyError'> __delattr__ = <method-wrapper '__delattr__' of exceptions.KeyError object> __dict__ = {} __doc__ = 'Mapping key not found.' __format__ = <built-in method __format__ of exceptions.KeyError object> __getattribute__ = <method-wrapper '__getattribute__' of exceptions.KeyError object> __getitem__ = <method-wrapper '__getitem__' of exceptions.KeyError object> __getslice__ = <method-wrapper '__getslice__' of exceptions.KeyError object> __hash__ = <method-wrapper '__hash__' of exceptions.KeyError object> __init__ = <method-wrapper '__init__' of exceptions.KeyError object> __new__ = <built-in method __new__ of type object> __reduce__ = <built-in method __reduce__ of exceptions.KeyError object> __reduce_ex__ = <built-in method __reduce_ex__ of exceptions.KeyError object> __repr__ = <method-wrapper '__repr__' of exceptions.KeyError object> __setattr__ = <method-wrapper '__setattr__' of exceptions.KeyError object> __setstate__ = <built-in method __setstate__ of exceptions.KeyError object> __sizeof__ = <built-in method __sizeof__ of exceptions.KeyError object> __str__ = <method-wrapper '__str__' of exceptions.KeyError object> __subclasshook__ = <built-in method __subclasshook__ of type object> __unicode__ = <built-in method __unicode__ of exceptions.KeyError object> args = ('uuid',) message = 'uuid' The above is a description of an error in a Python program.
Here is the original traceback: Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_api_server.py", line 638, in http_resource_create ok, result = stateful_create() File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_api_server.py", line 582, in stateful_create tenant_name, obj_dict, db_conn) File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_types.py", line 2030, in pre_dbe_create ServiceGroupServer.object_type, File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_types.py", line 232, in check_associated_firewall_resource_in_same_scope ok, result = cls.dbe_read(cls.db_conn, object_type, r_ref['uuid'], KeyError: 'uuid'

+---------+---------------------+----------------+-------------------+
| id | status | ip_address | mac_address |
+---------+---------------------+----------------+-------------------+
| nodec19 | provision_completed | 10.204.217.4 | 00:25:90:C3:AF:AA |
| nodec20 | provision_completed | 10.204.217.5 | 00:25:90:C3:08:6A |
| nodec21 | provision_completed | 10.204.217.6 | 00:25:90:C3:3F:12 |
| nodea4 | provision_completed | 10.204.216.120 | 00:25:90:A5:3B:12 |
| nodei18 | provision_completed | 10.204.217.130 | 00:25:90:E7:7E:FC |
| nodei16 | provision_completed | 10.204.217.128 | 00:25:90:E7:80:30 |
| nodea35 | provision_completed | 10.204.216.31 | 00:25:90:AA:A9:44 |
+---------+---------------------+----------------+-------------------+

+---------+---------------------------+-------------------+-----------------------+------------------------------------------------------------------------------------+
| id | cluster_id | provision_pending | provision_in_progress | provision_completed |
+---------+---------------------------+-------------------+-----------------------+------------------------------------------------------------------------------------+
| nodec19 | clusterc19c20c21a35i16i18 | [] | [] | ['openstack', 'contrail-controller', 'contrail-analyticsdb', 'contrail-analytics'] |
| nodec20 | clusterc19c20c21a35i16i18 | [] | [] | ['openstack', 'contrail-controller', 'contrail-analyticsdb', 'contrail-analytics'] |
| nodec21 | clusterc19c20c21a35i16i18 | [] | [] | ['openstack', 'contrail-controller', 'contrail-analyticsdb', 'contrail-analytics'] |
| nodea4 | clusterc19c20c21a35i16i18 | [] | [] | ['contrail-compute'] |
| nodei18 | clusterc19c20c21a35i16i18 | [] | [] | ['contrail-compute'] |
| nodei16 | clusterc19c20c21a35i16i18 | [] | [] | ['contrail-compute'] |
| nodea35 | clusterc19c20c21a35i16i18 | [] | [] | ['contrail-lb'] |
+---------+---------------------------+-------------------+-----------------------+------------------------------------------------------------------------------------+

[Naga Kiran (nagakiran)]

Able to create global/project-scope firewall policy with rules on this build.
Please share the steps to reproduce this issue

[Venkatesh Velpula (vvelpula)]

Hi Naga,

   Able to create the issue and Rajiv is looking into the setup ..below are the steps followed

-Create the global tags (teirs:web,app,application:HR,site:BLR,deployment:lab)
-Associated the tags to the VNs(VN1:web,HR,BLR,lab)(VN2:app,HR,BLR,lab)
-Create a service Group (SG1:ICMP,ICMP)
-created the firewall policy test1 with following rule
 Action :PASS
 Services :SG1
 endpoint1-tier:web
 endpoint2-tier:app
 match tags :Deployment

[Naga Kiran (nagakiran)]

Noticed this issue only when we try to associate service group while creating rule under global firewall policy.
As such there is no change in the data that's constructed in this case from UI, will check further and see whether the issue is on UI/API server?
It's working fine associating a service group to a rule under project scoped firewall policy.
Please update if you see any other case is failing.

Hi Venkatesh,
    Can you try to create a rule associating service group to a rule under global firewall policy from script and see if it goes through fine.

[Siva Bavanasi (kbsiva)]

Creation of Firewall rule is failing from api server if service_group_ref doesn't have uuid

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/37941
Submitter: ?douard Thuleau (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/37942
Submitter: ?douard Thuleau (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/37942
Committed: http://github.com/Juniper/contrail-controller/commit/e26bdf937299fa561f11168c9873b5ddee6c1318
Submitter: Zuul (<email address hidden>)
Branch: R4.1

commit e26bdf937299fa561f11168c9873b5ddee6c1318
Author: Édouard Thuleau <email address hidden>
Date: Tue Nov 28 23:01:17 2017 +0100

[config] Do not fail if UUID missing in a reference

Contrail API only requires either 'to' or 'uuid' in reference when a
resource is created/updated. That patch fetches reference UUID if missing
in the pre hook firewall rule methods.

Change-Id: I882826f2364445208b8a4f3e24939ac27be6b255
Closes-Bug: #1734816

[Venkatesh Velpula (vvelpula)]

verified with latest official build 7

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/37941
Committed: http://github.com/Juniper/contrail-controller/commit/d503ebd0c7cda139463edcb5cdec10909ac8287b
Submitter: Zuul (<email address hidden>)
Branch: master

commit d503ebd0c7cda139463edcb5cdec10909ac8287b
Author: Édouard Thuleau <email address hidden>
Date: Tue Nov 28 23:01:17 2017 +0100

[config] Do not fail if UUID missing in a reference

Contrail API only requires either 'to' or 'uuid' in reference when a
resource is created/updated. That patch fetches reference UUID if missing
in the pre hook firewall rule methods.

Change-Id: I882826f2364445208b8a4f3e24939ac27be6b255
Closes-Bug: #1734816