Can't start Neutron service when max VNI value is defined in ml2_conf.ini file

[Romil Gupta]

The VNI value is 24bits, i.e. <1 – 16777215>

When I defined the max value in the ml2_conf.ini file:
[ml2_type_vxlan]
vni_ranges = 1001:16777215

Neutron service fails to start.

2015-02-28 17:11:33.793 CRITICAL neutron [-] MemoryError

2015-02-28 17:11:33.793 TRACE neutron Traceback (most recent call last):
2015-02-28 17:11:33.793 TRACE neutron File "/usr/local/bin/neutron-server", line 9, in <module>
2015-02-28 17:11:33.793 TRACE neutron load_entry_point('neutron==2014.2.2.dev438', 'console_scripts', 'neutron-server')()
2015-02-28 17:11:33.793 TRACE neutron File "/opt/stack/neutron/neutron/server/__init__.py", line 48, in main
2015-02-28 17:11:33.793 TRACE neutron neutron_api = service.serve_wsgi(service.NeutronApiService)
2015-02-28 17:11:33.793 TRACE neutron File "/opt/stack/neutron/neutron/service.py", line 105, in serve_wsgi
2015-02-28 17:11:33.793 TRACE neutron LOG.exception(_('Unrecoverable error: please check log '
2015-02-28 17:11:33.793 TRACE neutron File "/opt/stack/neutron/neutron/openstack/common/excutils.py", line 82, in __exit__
2015-02-28 17:11:33.793 TRACE neutron six.reraise(self.type_, self.value, self.tb)
2015-02-28 17:11:33.793 TRACE neutron File "/opt/stack/neutron/neutron/service.py", line 102, in serve_wsgi
2015-02-28 17:11:33.793 TRACE neutron service.start()
2015-02-28 17:11:33.793 TRACE neutron File "/opt/stack/neutron/neutron/service.py", line 73, in start
2015-02-28 17:11:33.793 TRACE neutron self.wsgi_app = _run_wsgi(self.app_name)
2015-02-28 17:11:33.793 TRACE neutron File "/opt/stack/neutron/neutron/service.py", line 168, in _run_wsgi
2015-02-28 17:11:33.793 TRACE neutron app = config.load_paste_app(app_name)
2015-02-28 17:11:33.793 TRACE neutron File "/opt/stack/neutron/neutron/common/config.py", line 185, in load_paste_app
2015-02-28 17:11:33.793 TRACE neutron app = deploy.loadapp("config:%s" % config_path, name=app_name)
2015-02-28 17:11:33.793 TRACE neutron File "/usr/local/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 247, in loadapp
2015-02-28 17:11:33.793 TRACE neutron return loadobj(APP, uri, name=name, **kw)
2015-02-28 17:11:33.793 TRACE neutron File "/usr/local/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 272, in loadobj
2015-02-28 17:11:33.793 TRACE neutron return context.create()
2015-02-28 17:11:33.793 TRACE neutron File "/usr/local/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 710, in create
2015-02-28 17:11:33.793 TRACE neutron return self.object_type.invoke(self)
2015-02-28 17:11:33.793 TRACE neutron File "/usr/local/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 144, in invoke
2015-02-28 17:11:33.793 TRACE neutron **context.local_conf)
2015-02-28 17:11:33.793 TRACE neutron File "/usr/local/lib/python2.7/dist-packages/paste/deploy/util.py", line 55, in fix_call
2015-02-28 17:11:33.793 TRACE neutron val = callable(*args, **kw)
2015-02-28 17:11:33.793 TRACE neutron File "/usr/local/lib/python2.7/dist-packages/paste/urlmap.py", line 25, in urlmap_factory
2015-02-28 17:11:33.793 TRACE neutron app = loader.get_app(app_name, global_conf=global_conf)
2015-02-28 17:11:33.793 TRACE neutron File "/usr/local/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 350, in get_app
2015-02-28 17:11:33.793 TRACE neutron name=name, global_conf=global_conf).create()
2015-02-28 17:11:33.793 TRACE neutron File "/usr/local/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 710, in create
2015-02-28 17:11:33.793 TRACE neutron return self.object_type.invoke(self)
2015-02-28 17:11:33.793 TRACE neutron File "/usr/local/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 144, in invoke
2015-02-28 17:11:33.793 TRACE neutron **context.local_conf)
2015-02-28 17:11:33.793 TRACE neutron File "/usr/local/lib/python2.7/dist-packages/paste/deploy/util.py", line 55, in fix_call
2015-02-28 17:11:33.793 TRACE neutron val = callable(*args, **kw)
2015-02-28 17:11:33.793 TRACE neutron File "/opt/stack/neutron/neutron/auth.py", line 71, in pipeline_factory
2015-02-28 17:11:33.793 TRACE neutron app = loader.get_app(pipeline[-1])
2015-02-28 17:11:33.793 TRACE neutron File "/usr/local/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 350, in get_app
2015-02-28 17:11:33.793 TRACE neutron name=name, global_conf=global_conf).create()
2015-02-28 17:11:33.793 TRACE neutron File "/usr/local/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 710, in create
2015-02-28 17:11:33.793 TRACE neutron return self.object_type.invoke(self)
2015-02-28 17:11:33.793 TRACE neutron File "/usr/local/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 146, in invoke
2015-02-28 17:11:33.793 TRACE neutron return fix_call(context.object, context.global_conf, **context.local_conf)
2015-02-28 17:11:33.793 TRACE neutron File "/usr/local/lib/python2.7/dist-packages/paste/deploy/util.py", line 55, in fix_call
2015-02-28 17:11:33.793 TRACE neutron val = callable(*args, **kw)
2015-02-28 17:11:33.793 TRACE neutron File "/opt/stack/neutron/neutron/api/v2/router.py", line 69, in factory
2015-02-28 17:11:33.793 TRACE neutron return cls(**local_config)
2015-02-28 17:11:33.793 TRACE neutron File "/opt/stack/neutron/neutron/api/v2/router.py", line 73, in __init__
2015-02-28 17:11:33.793 TRACE neutron plugin = manager.NeutronManager.get_plugin()
2015-02-28 17:11:33.793 TRACE neutron File "/opt/stack/neutron/neutron/manager.py", line 219, in get_plugin
2015-02-28 17:11:33.793 TRACE neutron return weakref.proxy(cls.get_instance().plugin)
2015-02-28 17:11:33.793 TRACE neutron File "/opt/stack/neutron/neutron/manager.py", line 213, in get_instance
2015-02-28 17:11:33.793 TRACE neutron cls._create_instance()
2015-02-28 17:11:33.793 TRACE neutron File "/opt/stack/neutron/neutron/openstack/common/lockutils.py", line 272, in inner
2015-02-28 17:11:33.793 TRACE neutron return f(*args, **kwargs)
2015-02-28 17:11:33.793 TRACE neutron File "/opt/stack/neutron/neutron/manager.py", line 199, in _create_instance
2015-02-28 17:11:33.793 TRACE neutron cls._instance = cls()
2015-02-28 17:11:33.793 TRACE neutron File "/opt/stack/neutron/neutron/manager.py", line 114, in __init__
2015-02-28 17:11:33.793 TRACE neutron plugin_provider)
2015-02-28 17:11:33.793 TRACE neutron File "/opt/stack/neutron/neutron/manager.py", line 140, in _get_plugin_instance
2015-02-28 17:11:33.793 TRACE neutron return plugin_class()
2015-02-28 17:11:33.793 TRACE neutron File "/opt/stack/neutron/neutron/plugins/ml2/plugin.py", line 126, in __init__
2015-02-28 17:11:33.793 TRACE neutron self.type_manager.initialize()
2015-02-28 17:11:33.793 TRACE neutron File "/opt/stack/neutron/neutron/plugins/ml2/managers.py", line 152, in initialize
2015-02-28 17:11:33.793 TRACE neutron driver.obj.initialize()
2015-02-28 17:11:33.793 TRACE neutron File "/opt/stack/neutron/neutron/plugins/ml2/drivers/type_vxlan.py", line 80, in initialize
2015-02-28 17:11:33.793 TRACE neutron self._initialize(cfg.CONF.ml2_type_vxlan.vni_ranges)
2015-02-28 17:11:33.793 TRACE neutron File "/opt/stack/neutron/neutron/plugins/ml2/drivers/type_tunnel.py", line 65, in _initialize
2015-02-28 17:11:33.793 TRACE neutron self.sync_allocations()
2015-02-28 17:11:33.793 TRACE neutron File "/opt/stack/neutron/neutron/plugins/ml2/drivers/type_vxlan.py", line 96, in sync_allocations
2015-02-28 17:11:33.793 TRACE neutron vxlan_vnis |= set(moves.xrange(tun_min, tun_max + 1))
2015-02-28 17:11:33.793 TRACE neutron MemoryError
2015-02-28 17:11:33.793 TRACE neutron

[Kevin Benton (kevinbenton)]

Unless the fix is trivial, don't put a lot of work into this bug. There are many other scaling issues we will run into before neutron can handle millions of networks.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/251940

[OpenStack Infra (hudson-openstack)]

Change abandoned by Sreekumar S (<email address hidden>) on branch: master
Review: https://review.openstack.org/251940
Reason: There are many other factors to be considered to make this scale out to millions. Abandoning.

[Sreekumar S (sreesiv)]

As Kevin suggested above the fix is not trivial and there are many other issues to be considered along the way to make it scale to millions of networks. A lone fix like the one abandoned above is not going to make much difference. Anyone planning to pick up scale issues, should consider all the scale factors and other issues mentioned in the PS review for the above patch.