sanity: subnet creation failed

Bug #1647835 reported by Senthilnathan Murugappan
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Juniper Openstack
Status tracked in Trunk
R3.2
Invalid
High
Senthilnathan Murugappan
Trunk
Invalid
High
Senthilnathan Murugappan

Bug Description

During sanity run of 3.2.0.0-8 Kilo observed few instances of subnet_vnc object being None hence subnet creation has failed.
contrail logs are available at /cs-shared/test_runs/nodei13-vm2/jenkins-ubuntu-14-04_kilo_Multi_Node_Sanity-306 of anamika.englab.juniper.net (blr servers)

Python 2.7.6: /usr/bin/python
Mon Dec 5 01:07:36 2016

A problem occurred in a Python script. Here is the sequence of
function calls leading up to the error, in the order they occurred.

/usr/lib/python2.7/dist-packages/vnc_openstack/__init__.py in handler_trap_exception(*args=(), **kwargs={})
1100 def handler_trap_exception(*args, **kwargs):
1101 try:
1102 response = handler(*args, **kwargs)
1103 return response
1104 except Exception as e:
response undefined
handler = <bound method NeutronPluginInterface.plugin_http..._plugin_interface.NeutronPluginInterface object>>
args = ()
kwargs = {}

/usr/lib/python2.7/dist-packages/vnc_openstack/neutron_plugin_interface.py in plugin_http_post_subnet(self=<vnc_openstack.neutron_plugin_interface.NeutronPluginInterface object>)
  373 return self.plugin_get_subnet(context, subnet)
  374 elif context['operation'] == 'CREATE':
  375 return self.plugin_create_subnet(context, subnet)
  376 elif context['operation'] == 'UPDATE':
  377 return self.plugin_update_subnet(context, subnet)
self = <vnc_openstack.neutron_plugin_interface.NeutronPluginInterface object>
self.plugin_create_subnet = <bound method NeutronPluginInterface.plugin_crea..._plugin_interface.NeutronPluginInterface object>>
context = {'is_admin': True, 'operation': 'CREATE', 'roles': ['admin'], 'tenant': '7cbccb4d17aa489dae6212ce26191b82', 'tenant_id': '7cbccb4d17aa489dae6212ce26191b82', 'type': 'subnet', 'user_id': 'bb2a03248a3e423aa381c2e2fc365b43'}
subnet = {'fields': None, 'filters': None, 'resource': {'cidr': '30fb:8c65:b2ee:4cea::/64', 'contrail:ipam_fq_name': ['default-domain', 'default-project', 'default-network-ipam'], 'enable_dhcp': True, 'ip_version': 6, 'name': u'', 'network_id': 'c20f0493-da6d-4ec4-bbdd-a04e938ac5e5', 'tenant_id': '7cbccb4d17aa489dae6212ce26191b82'}}

/usr/lib/python2.7/dist-packages/vnc_openstack/neutron_plugin_interface.py in plugin_create_subnet(self=<vnc_openstack.neutron_plugin_interface.NeutronPluginInterface object>, context={'is_admin': True, 'operation': 'CREATE', 'roles': ['admin'], 'tenant': '7cbccb4d17aa489dae6212ce26191b82', 'tenant_id': '7cbccb4d17aa489dae6212ce26191b82', 'type': 'subnet', 'user_id': 'bb2a03248a3e423aa381c2e2fc365b43'}, subnet={'fields': None, 'filters': None, 'resource': {'cidr': '30fb:8c65:b2ee:4cea::/64', 'contrail:ipam_fq_name': ['default-domain', 'default-project', 'default-network-ipam'], 'enable_dhcp': True, 'ip_version': 6, 'name': u'', 'network_id': 'c20f0493-da6d-4ec4-bbdd-a04e938ac5e5', 'tenant_id': '7cbccb4d17aa489dae6212ce26191b82'}})
  299 try:
  300 cfgdb = self._get_user_cfgdb(context)
  301 subnet_info = cfgdb.subnet_create(subnet['resource'])
  302 return self._make_subnet_dict(subnet_info)
  303 except Exception as e:
subnet_info undefined
cfgdb = <vnc_openstack.neutron_plugin_db.DBInterface object>
cfgdb.subnet_create = <bound method DBInterface.wrapper of <vnc_openstack.neutron_plugin_db.DBInterface object>>
subnet = {'fields': None, 'filters': None, 'resource': {'cidr': '30fb:8c65:b2ee:4cea::/64', 'contrail:ipam_fq_name': ['default-domain', 'default-project', 'default-network-ipam'], 'enable_dhcp': True, 'ip_version': 6, 'name': u'', 'network_id': 'c20f0493-da6d-4ec4-bbdd-a04e938ac5e5', 'tenant_id': '7cbccb4d17aa489dae6212ce26191b82'}}

/usr/lib/python2.7/dist-packages/vnc_openstack/neutron_plugin_db.py in wrapper(self=<vnc_openstack.neutron_plugin_db.DBInterface object>, *args=({'cidr': '30fb:8c65:b2ee:4cea::/64', 'contrail:ipam_fq_name': ['default-domain', 'default-project', 'default-network-ipam'], 'enable_dhcp': True, 'ip_version': 6, 'name': u'', 'network_id': 'c20f0493-da6d-4ec4-bbdd-a04e938ac5e5', 'tenant_id': '7cbccb4d17aa489dae6212ce26191b82'},), **kwargs={})
2492 def wrapper(self, *args, **kwargs):
2493 self._connected_to_api_server.wait()
2494 return func(self, *args, **kwargs)
2495
2496 return wrapper
func = <function subnet_create>
self = <vnc_openstack.neutron_plugin_db.DBInterface object>
args = ({'cidr': '30fb:8c65:b2ee:4cea::/64', 'contrail:ipam_fq_name': ['default-domain', 'default-project', 'default-network-ipam'], 'enable_dhcp': True, 'ip_version': 6, 'name': u'', 'network_id': 'c20f0493-da6d-4ec4-bbdd-a04e938ac5e5', 'tenant_id': '7cbccb4d17aa489dae6212ce26191b82'},)
kwargs = {}

/usr/lib/python2.7/dist-packages/vnc_openstack/neutron_plugin_db.py in subnet_create(self=<vnc_openstack.neutron_plugin_db.DBInterface object>, subnet_q={'cidr': '30fb:8c65:b2ee:4cea::/64', 'contrail:ipam_fq_name': ['default-domain', 'default-project', 'default-network-ipam'], 'enable_dhcp': True, 'ip_version': 6, 'name': u'', 'network_id': 'c20f0493-da6d-4ec4-bbdd-a04e938ac5e5', 'tenant_id': '7cbccb4d17aa489dae6212ce26191b82'})
2779 subnet_vnc = self._subnet_read(net_id, subnet_prefix)
2780 subnet_info = self._subnet_vnc_to_neutron(subnet_vnc, net_obj,
2781 ipam_fq_name)
2782
2783 return subnet_info
ipam_fq_name = ['default-domain', 'default-project', 'default-network-ipam']

/usr/lib/python2.7/dist-packages/vnc_openstack/neutron_plugin_db.py in _subnet_vnc_to_neutron(self=<vnc_openstack.neutron_plugin_db.DBInterface object>, subnet_vnc=None, net_obj=<vnc_api.gen.resource_client.VirtualNetwork object>, ipam_fq_name=['default-domain', 'default-project', 'default-network-ipam'])
1462 sn_q_dict = {}
1463 extra_dict = {}
1464 sn_name = subnet_vnc.get_subnet_name()
1465 if sn_name is not None:
1466 sn_q_dict['name'] = sn_name
sn_name undefined
subnet_vnc = None
subnet_vnc.get_subnet_name undefined
<type 'exceptions.AttributeError'>: 'NoneType' object has no attribute 'get_subnet_name'
    __class__ = <type 'exceptions.AttributeError'>
    __delattr__ = <method-wrapper '__delattr__' of exceptions.AttributeError object>
    __dict__ = {}
    __doc__ = 'Attribute not found.'
    __format__ = <built-in method __format__ of exceptions.AttributeError object>
    __getattribute__ = <method-wrapper '__getattribute__' of exceptions.AttributeError object>
    __getitem__ = <method-wrapper '__getitem__' of exceptions.AttributeError object>
    __getslice__ = <method-wrapper '__getslice__' of exceptions.AttributeError object>
    __hash__ = <method-wrapper '__hash__' of exceptions.AttributeError object>
    __init__ = <method-wrapper '__init__' of exceptions.AttributeError object>
    __new__ = <built-in method __new__ of type object>
    __reduce__ = <built-in method __reduce__ of exceptions.AttributeError object>
    __reduce_ex__ = <built-in method __reduce_ex__ of exceptions.AttributeError object>
    __repr__ = <method-wrapper '__repr__' of exceptions.AttributeError object>
    __setattr__ = <method-wrapper '__setattr__' of exceptions.AttributeError object>
    __setstate__ = <built-in method __setstate__ of exceptions.AttributeError object>
    __sizeof__ = <built-in method __sizeof__ of exceptions.AttributeError object>
    __str__ = <method-wrapper '__str__' of exceptions.AttributeError object>
    __subclasshook__ = <built-in method __subclasshook__ of type object>
    __unicode__ = <built-in method __unicode__ of exceptions.AttributeError object>
    args = ("'NoneType' object has no attribute 'get_subnet_name'",)
    message = "'NoneType' object has no attribute 'get_subnet_name'"

The above is a description of an error in a Python program. Here is
the original traceback:

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/vnc_openstack/__init__.py", line 1102, in handler_trap_exception
    response = handler(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/vnc_openstack/neutron_plugin_interface.py", line 375, in plugin_http_post_subnet
    return self.plugin_create_subnet(context, subnet)
  File "/usr/lib/python2.7/dist-packages/vnc_openstack/neutron_plugin_interface.py", line 301, in plugin_create_subnet
    subnet_info = cfgdb.subnet_create(subnet['resource'])
  File "/usr/lib/python2.7/dist-packages/vnc_openstack/neutron_plugin_db.py", line 2494, in wrapper
    return func(self, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/vnc_openstack/neutron_plugin_db.py", line 2781, in subnet_create
    ipam_fq_name)
  File "/usr/lib/python2.7/dist-packages/vnc_openstack/neutron_plugin_db.py", line 1464, in _subnet_vnc_to_neutron
    sn_name = subnet_vnc.get_subnet_name()
AttributeError: 'NoneType' object has no attribute 'get_subnet_name'

Tags: config
Revision history for this message
Jeba Paulaiyan (jebap) wrote :

This happens in build 12 also

Logs in: /cs-shared/test_runs/nodei13-vm2/jenkins-ubuntu-14-04_liberty_Multi_Node_Sanity-210/nodei13-vm2

tags: added: blocker
Revision history for this message
Jeba Paulaiyan (jebap) wrote :

This happened in only one testbed and also not consistent. The testbed where it got re-created has VMs for controllers. Senthil is working on re-creating it. If it gets recreated, will look at this.

tags: removed: blocker
Jeba Paulaiyan (jebap)
tags: removed: sanity
Revision history for this message
Jeba Paulaiyan (jebap) wrote :

This issue did not happen after the nodes are properly NTP sync'd

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.