schema crashes with NoIdError for a SI during gateway-related tests

Bug #1365839 reported by Vedamurthy Joshi
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Juniper Openstack
Status tracked in Trunk
R1.1
Fix Committed
High
Sachin Bansal
Trunk
Fix Committed
High
Sachin Bansal

Bug Description

R1.10 31

Was seen during tempest run :
Schema crashed multiple times with NoIDError on a snat si ID

http://anamika.englab.juniper.net:8080/job/ubuntu-12-04_havana_Single_node_Tempest/186/consoleText

Logs from all 3 config nodes will be in http://10.204.216.50/Docs/bugs/#

upstart/supervisor-config.log :
=============================
2014-09-04 12:02:20,239 INFO exited: contrail-schema (exit status 1; not expected)^M
2014-09-04 12:02:20,303 INFO spawned: 'contrail-schema' with pid 3798^M
2014-09-04 12:02:21,305 INFO success: contrail-schema entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)^M
2014-09-04 12:02:23,422 INFO exited: contrail-schema (exit status 1; not expected)^M
2014-09-04 12:02:23,543 INFO spawned: 'contrail-schema' with pid 3807^M
2014-09-04 12:02:24,649 INFO success: contrail-schema entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)^M
2014-09-04 12:02:39,356 INFO exited: contrail-schema (exit status 1; not expected)^M
2014-09-04 12:02:39,389 INFO spawned: 'contrail-schema' with pid 3825^M
2014-09-04 12:02:40,394 INFO success: contrail-schema entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)^M
2014-09-04 12:03:42,744 INFO exited: contrail-schema (exit status 1; not expected)^M
2014-09-04 12:03:42,798 INFO spawned: 'contrail-schema' with pid 3897^M
2014-09-04 12:03:43,799 INFO success: contrail-schema entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)^M

schema.err :
============

<class 'cfgm_common.exceptions.NoIdError'>
Python 2.7.3: /usr/bin/python
Thu Sep 4 12:03:42 2014

A problem occurred in a Python script. Here is the sequence of
function calls leading up to the error, in the order they occurred.

 /usr/lib/python2.7/dist-packages/schema_transformer/to_bgp.py in launch_arc(transformer=<schema_transformer.to_bgp.SchemaTransformer object>, ssrc_mapc=<cfgm_common.ifmap.client.client instance>)
 3513 result = arc_mapc.call('poll', pollreq)
 3514 try:
 3515 transformer.process_poll_result(result)
 3516 except Exception as e:
 3517 try:
transformer = <schema_transformer.to_bgp.SchemaTransformer object>
transformer.process_poll_result = <bound method SchemaTransformer.process_poll_res...ema_transformer.to_bgp.SchemaTransformer object>>
result = '<?xml version="1.0" encoding="UTF-8" standalone=...lResult></ns2:response></ns3:Body></ns3:Envelope>'

 /usr/lib/python2.7/dist-packages/schema_transformer/to_bgp.py in process_poll_result(self=<schema_transformer.to_bgp.SchemaTransformer object>, poll_result_str='<?xml version="1.0" encoding="UTF-8" standalone=...lResult></ns2:response></ns3:Body></ns3:Envelope>')
 3155 result_type.split('Result')[0].title(),
 3156 meta_name, idents, meta, funcname)
 3157 func(idents, meta)
 3158 # end for meta
 3159 # end for result_type
func = <bound method SchemaTransformer.add_service_inst...ema_transformer.to_bgp.SchemaTransformer object>>
idents = {'service-instance': 'default-domain:RoutersTest-1783388314:si_f330a427-6fdf-447f-9722-20b5c36502c1'}
meta = <Element {http://www.contrailsystems.com/vnc_cfg.xsd}service-instance-properties>

 /usr/lib/python2.7/dist-packages/schema_transformer/to_bgp.py in add_service_instance_properties(self=<schema_transformer.to_bgp.SchemaTransformer object>, idents={'service-instance': 'default-domain:RoutersTest-1783388314:si_f330a427-6fdf-447f-9722-20b5c36502c1'}, meta=<Element {http://www.contrailsystems.com/vnc_cfg.xsd}service-instance-properties>)
 3000 self.delete_service_instance_properties(idents, meta)
 3001 return
 3002 si = _vnc_lib.service_instance_read(fq_name_str=si_name)
 3003 si_props = si.get_service_instance_properties()
 3004 left_vn_str, right_vn_str = get_si_vns(si, si_props)
si undefined
global _vnc_lib = <vnc_api.vnc_api.VncApi object>
_vnc_lib.service_instance_read = <bound method VncApi.service_instance_read of <vnc_api.vnc_api.VncApi object>>
fq_name_str undefined
si_name = 'default-domain:RoutersTest-1783388314:si_f330a427-6fdf-447f-9722-20b5c36502c1'

 /usr/lib/python2.7/dist-packages/vnc_api/gen/vnc_api_client_gen.py in service_instance_read(self=<vnc_api.vnc_api.VncApi object>, fq_name=None, fq_name_str='default-domain:RoutersTest-1783388314:si_f330a427-6fdf-447f-9722-20b5c36502c1', id=None, ifmap_id=None, fields=None)
 2291
 2292 """
 2293 (args_ok, result) = self._read_args_to_id('service-instance', fq_name, fq_name_str, id, ifmap_id)
 2294 if not args_ok:
 2295 return result
args_ok undefined
result undefined
self = <vnc_api.vnc_api.VncApi object>
self._read_args_to_id = <bound method VncApi._read_args_to_id of <vnc_api.vnc_api.VncApi object>>
fq_name = None
fq_name_str = 'default-domain:RoutersTest-1783388314:si_f330a427-6fdf-447f-9722-20b5c36502c1'
id = None
ifmap_id = None

 /usr/lib/python2.7/dist-packages/vnc_api/vnc_api.py in _read_args_to_id(self=<vnc_api.vnc_api.VncApi object>, obj_type='service-instance', fq_name=None, fq_name_str='default-domain:RoutersTest-1783388314:si_f330a427-6fdf-447f-9722-20b5c36502c1', id=None, ifmap_id=None)
  314 return (True, self.fq_name_to_id(obj_type, fq_name))
  315 if fq_name_str:
  316 return (True, self.fq_name_to_id(obj_type, fq_name_str.split(':')))
  317 if ifmap_id:
  318 return (True, self.ifmap_to_id(ifmap_id))
builtinTrue = True
self = <vnc_api.vnc_api.VncApi object>
self.fq_name_to_id = <bound method VncApi.fq_name_to_id of <vnc_api.vnc_api.VncApi object>>
obj_type = 'service-instance'
fq_name_str = 'default-domain:RoutersTest-1783388314:si_f330a427-6fdf-447f-9722-20b5c36502c1'
fq_name_str.split = <built-in method split of str object>

 /usr/lib/python2.7/dist-packages/vnc_api/vnc_api.py in fq_name_to_id(self=<vnc_api.vnc_api.VncApi object>, obj_type='service-instance', fq_name=['default-domain', 'RoutersTest-1783388314', 'si_f330a427-6fdf-447f-9722-20b5c36502c1'])
  397 uri = self._action_uri['name-to-id']
  398 try:
  399 content = self._request_server(rest.OP_POST, uri, data=json_body)
  400 except HttpError as he:
  401 if he.status_code == 404:
content undefined
self = <vnc_api.vnc_api.VncApi object>
self._request_server = <bound method VncApi._request_server of <vnc_api.vnc_api.VncApi object>>
global rest = <module 'cfgm_common.rest' from '/usr/lib/python2.7/dist-packages/cfgm_common/rest.pyc'>
rest.OP_POST = 1
uri = u'/fqname-to-id'
data undefined
json_body = '{"fq_name": ["default-domain", "RoutersTest-1783...-9722-20b5c36502c1"], "type": "service-instance"}'

 /usr/lib/python2.7/dist-packages/vnc_api/vnc_api.py in _request_server(self=<vnc_api.vnc_api.VncApi object>, op=1, url=u'/fqname-to-id', data='{"fq_name": ["default-domain", "RoutersTest-1783...-9722-20b5c36502c1"], "type": "service-instance"}', retry_on_error=True, retry_after_authn=False)
  356 elif status == 404:
  357 raise NoIdError('Error: oper %s url %s body %s response %s'
  358 % (op, url, data, content))
  359 elif status == 403:
  360 raise PermissionDenied(content)
op = 1
url = u'/fqname-to-id'
data = '{"fq_name": ["default-domain", "RoutersTest-1783...-9722-20b5c36502c1"], "type": "service-instance"}'
content = u"Name [u'default-domain',\n u'RoutersTest-17833...f330a427-6fdf-447f-9722-20b5c36502c1'] not found"
<class 'cfgm_common.exceptions.NoIdError'>: Unknown id: Error: oper 1 url /fqname-to-id body {"fq_name": ["default-domain", "RoutersTest-1783388314", "si_f330a427-6fdf-447f-9722-20b5c36502c1"], "type": "service-instance"} response Name [u'default-domain',
 u'RoutersTest-1783388314',
 u'si_f330a427-6fdf-447f-9722-20b5c36502c1'] not found
    __class__ = <class 'cfgm_common.exceptions.NoIdError'>
    __delattr__ = <method-wrapper '__delattr__' of NoIdError object>
    __dict__ = {'_unknown_id': u'Error: oper 1 url /fqname-to-id body {"fq_name...330a427-6fdf-447f-9722-20b5c36502c1\'] not found'}
    __doc__ = None
    __format__ = <built-in method __format__ of NoIdError object>
    __getattribute__ = <method-wrapper '__getattribute__' of NoIdError object>
    __getitem__ = <method-wrapper '__getitem__' of NoIdError object>
    __getslice__ = <method-wrapper '__getslice__' of NoIdError object>
    __hash__ = <method-wrapper '__hash__' of NoIdError object>
    __init__ = <bound method NoIdError.__init__ of NoIdError()>
    __module__ = 'cfgm_common.exceptions'
    __new__ = <built-in method __new__ of type object>
    __reduce__ = <built-in method __reduce__ of NoIdError object>
    __reduce_ex__ = <built-in method __reduce_ex__ of NoIdError object>
    __repr__ = <method-wrapper '__repr__' of NoIdError object>
    __setattr__ = <method-wrapper '__setattr__' of NoIdError object>
    __setstate__ = <built-in method __setstate__ of NoIdError object>
    __sizeof__ = <built-in method __sizeof__ of NoIdError object>
    __str__ = <bound method NoIdError.__str__ of NoIdError()>
    __subclasshook__ = <built-in method __subclasshook__ of type object>
    __unicode__ = <built-in method __unicode__ of NoIdError object>
    __weakref__ = None
    _unknown_id = u'Error: oper 1 url /fqname-to-id body {"fq_name...330a427-6fdf-447f-9722-20b5c36502c1\'] not found'
    args = ()
    message = ''

The above is a description of an error in a Python program. Here is
the original traceback:

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/schema_transformer/to_bgp.py", line 3515, in launch_arc
    transformer.process_poll_result(result)
  File "/usr/lib/python2.7/dist-packages/schema_transformer/to_bgp.py", line 3157, in process_poll_result
    func(idents, meta)
  File "/usr/lib/python2.7/dist-packages/schema_transformer/to_bgp.py", line 3002, in add_service_instance_properties
    si = _vnc_lib.service_instance_read(fq_name_str=si_name)
  File "/usr/lib/python2.7/dist-packages/vnc_api/gen/vnc_api_client_gen.py", line 2293, in service_instance_read
    (args_ok, result) = self._read_args_to_id('service-instance', fq_name, fq_name_str, id, ifmap_id)
  File "/usr/lib/python2.7/dist-packages/vnc_api/vnc_api.py", line 316, in _read_args_to_id
    return (True, self.fq_name_to_id(obj_type, fq_name_str.split(':')))
  File "/usr/lib/python2.7/dist-packages/vnc_api/vnc_api.py", line 399, in fq_name_to_id
    content = self._request_server(rest.OP_POST, uri, data=json_body)
  File "/usr/lib/python2.7/dist-packages/vnc_api/vnc_api.py", line 358, in _request_server
    % (op, url, data, content))
NoIdError: Unknown id: Error: oper 1 url /fqname-to-id body {"fq_name": ["default-domain", "RoutersTest-1783388314", "si_f330a427-6fdf-447f-9722-20b5c36502c1"], "type": "service-instance"} response Name [u'default-domain',
 u'RoutersTest-1783388314',
 u'si_f330a427-6fdf-447f-9722-20b5c36502c1'] not found

tags: added: snat
Revision history for this message
Sachin Bansal (sbansal) wrote :

Fixed with commit id 32ccd5d5b463ef4842f0ac170815ee07f6a666b0 in mainline.

Revision history for this message
Sachin Bansal (sbansal) wrote :
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.