[mainline 2715] contrail-schema crash while deleting network policy

Bug #1550156 reported by Prakash Bailkeri
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Juniper Openstack
Status tracked in Trunk
R3.0
Fix Committed
High
Sachin Bansal
Trunk
Fix Committed
High
Sachin Bansal

Bug Description

<type 'exceptions.AttributeError'>
Python 2.7.6: /usr/bin/python
Fri Feb 26 11:09:43 2016

A problem occurred in a Python script. Here is the sequence of
function calls leading up to the error, in the order they occurred.

 /usr/lib/python2.7/dist-packages/schema_transformer/to_bgp.py in _vnc_subscribe_callback(self=<schema_transformer.to_bgp.SchemaTransformer object>, oper_info={u'oper': u'UPDATE', u'type': u'routing-instance', u'uuid': u'4b88e1ca-8e3d-4107-b792-e73e68a57c4b'})
  316 res_obj = cls.get(res_id)
  317 if res_obj is not None:
  318 res_obj.evaluate()
  319
  320 for vn_id in dependency_tracker.resources.get('virtual_network', []):
res_obj = <schema_transformer.config_db.VirtualNetworkST object>
res_obj.evaluate = <bound method VirtualNetworkST.evaluate of <schema_transformer.config_db.VirtualNetworkST object>>

 /usr/lib/python2.7/dist-packages/schema_transformer/config_db.py in evaluate(self=<schema_transformer.config_db.VirtualNetworkST object>)
 1345 ri = RoutingInstanceST.get(ri_name)
 1346 if ri:
 1347 ri.update_routing_policy_and_aggregates()
 1348 # end evaluate
 1349
ri = <schema_transformer.config_db.RoutingInstanceST object>
ri.update_routing_policy_and_aggregates = <bound method RoutingInstanceST.update_routing_p..._transformer.config_db.RoutingInstanceST object>>

 /usr/lib/python2.7/dist-packages/schema_transformer/config_db.py in update_routing_policy_and_aggregates(self=<schema_transformer.config_db.RoutingInstanceST object>)
 1931 return
 1932 sc = ServiceChain.get(self.service_chain)
 1933 for si_name in sc.service_list:
 1934 if not self.name.endswith(si_name.replace(':', '_')):
 1935 continue
si_name undefined
sc = None
sc.service_list undefined
<type 'exceptions.AttributeError'>: 'NoneType' object has no attribute 'service_list'
    __class__ = <type 'exceptions.AttributeError'>
    __delattr__ = <method-wrapper '__delattr__' of exceptions.AttributeError object>
    __dict__ = {}
    __doc__ = 'Attribute not found.'
    __format__ = <built-in method __format__ of exceptions.AttributeError object>
    __getattribute__ = <method-wrapper '__getattribute__' of exceptions.AttributeError object>
    __getitem__ = <method-wrapper '__getitem__' of exceptions.AttributeError object>
    __getslice__ = <method-wrapper '__getslice__' of exceptions.AttributeError object>
    __hash__ = <method-wrapper '__hash__' of exceptions.AttributeError object>
    __init__ = <method-wrapper '__init__' of exceptions.AttributeError object>
    __new__ = <built-in method __new__ of type object>
    __reduce__ = <built-in method __reduce__ of exceptions.AttributeError object>
    __reduce_ex__ = <built-in method __reduce_ex__ of exceptions.AttributeError object>
    __repr__ = <method-wrapper '__repr__' of exceptions.AttributeError object>
    __setattr__ = <method-wrapper '__setattr__' of exceptions.AttributeError object>
    __setstate__ = <built-in method __setstate__ of exceptions.AttributeError object>
    __sizeof__ = <built-in method __sizeof__ of exceptions.AttributeError object>
    __str__ = <method-wrapper '__str__' of exceptions.AttributeError object>
    __subclasshook__ = <built-in method __subclasshook__ of type object>
    __unicode__ = <built-in method __unicode__ of exceptions.AttributeError object>
    args = ("'NoneType' object has no attribute 'service_list'",)
    message = "'NoneType' object has no attribute 'service_list'"

The above is a description of an error in a Python program. Here is
the original traceback:

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/schema_transformer/to_bgp.py", line 318, in _vnc_subscribe_callback
    res_obj.evaluate()
  File "/usr/lib/python2.7/dist-packages/schema_transformer/config_db.py", line 1347, in evaluate
    ri.update_routing_policy_and_aggregates()
  File "/usr/lib/python2.7/dist-packages/schema_transformer/config_db.py", line 1933, in update_routing_policy_and_aggregates
    for si_name in sc.service_list:
AttributeError: 'NoneType' object has no attribute 'service_list'

Tags: blocker config
Nischal Sheth (nsheth)
information type: Proprietary → Public
Revision history for this message
Prakash Bailkeri (prakashmb) wrote :

Steps to recreate:
1. Create SI and attaching routing policy to SI
2. Create a network policy to apply the SI
3. Attach to networks
4. Restart contrail-schema
5. Delete the network policy
With this steps you will hit:

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/schema_transformer/to_bgp.py", line 318, in _vnc_subscribe_callback
    res_obj.evaluate()
  File "/usr/lib/python2.7/dist-packages/schema_transformer/config_db.py", line 1336, in evaluate
    service_chain.destroy()
  File "/usr/lib/python2.7/dist-packages/schema_transformer/config_db.py", line 2683, in destroy
    RoutingInstanceST.delete(service_name1)
  File "/usr/lib/python2.7/dist-packages/schema_transformer/config_db.py", line 117, in delete
    obj.delete_obj()
  File "/usr/lib/python2.7/dist-packages/schema_transformer/config_db.py", line 2250, in delete_obj
    DBBaseST._vnc_lib.routing_instance_delete(id=self.obj.uuid)
  File "/usr/lib/python2.7/dist-packages/vnc_api/vnc_api.py", line 39, in wrapper
    return func(self, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/vnc_api/vnc_api.py", line 512, in _object_delete
    content = self._request_server(rest.OP_DELETE, uri)
  File "/usr/lib/python2.7/dist-packages/vnc_api/vnc_api.py", line 712, in _request_server
    retry_count=retry_count)
  File "/usr/lib/python2.7/dist-packages/vnc_api/vnc_api.py", line 757, in _request
    raise RefsExistError(content)
RefsExistError: Delete when resource still referred: ['http://10.204.216.70:8082/routing-policy/4fb1094d-4135-4552-9427-fec882c6b650']

and

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/schema_transformer/to_bgp.py", line 318, in _vnc_subscribe_callback
    res_obj.evaluate()
  File "/usr/lib/python2.7/dist-packages/schema_transformer/config_db.py", line 1347, in evaluate
    ri.update_routing_policy_and_aggregates()
  File "/usr/lib/python2.7/dist-packages/schema_transformer/config_db.py", line 1933, in update_routing_policy_and_aggregates
    for si_name in sc.service_list:
AttributeError: 'NoneType' object has no attribute 'service_list'

amit surana (asurana-t)
tags: added: blocker
Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R3.0

Review in progress for https://review.opencontrail.org/17938
Submitter: Sachin Bansal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] master

Review in progress for https://review.opencontrail.org/17939
Submitter: Sachin Bansal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/17939
Committed: http://github.org/Juniper/contrail-controller/commit/cb0aabfe7cc2522c6c71d9fc63a0c8c954fa8e31
Submitter: Zuul
Branch: master

commit cb0aabfe7cc2522c6c71d9fc63a0c8c954fa8e31
Author: Sachin Bansal <email address hidden>
Date: Fri Feb 26 16:56:56 2016 -0800

Add a check for none before accessing variable

The root cause of the bug was fixed with this review:
https://review.opencontrail.org/#/c/17915/
However, adding a defensive check also in case we still hit the issue.

Change-Id: Ib17229d652c588e3698a5df6c399f8a5264b90f8
Closes-Bug: 1550156
(cherry picked from commit f279c16d747beb9945d920073a160680c6929484)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/17938
Committed: http://github.org/Juniper/contrail-controller/commit/f279c16d747beb9945d920073a160680c6929484
Submitter: Zuul
Branch: R3.0

commit f279c16d747beb9945d920073a160680c6929484
Author: Sachin Bansal <email address hidden>
Date: Fri Feb 26 16:56:56 2016 -0800

Add a check for none before accessing variable

The root cause of the bug was fixed with this review:
https://review.opencontrail.org/#/c/17915/
However, adding a defensive check also in case we still hit the issue.

Change-Id: Ib17229d652c588e3698a5df6c399f8a5264b90f8
Closes-Bug: 1550156

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.