check-grenade-dsvm-neutron is failing after the services split. Below is the screen log of q-svc http://logs.openstack.org/44/141144/1/check/check-grenade-dsvm-neutron/8d5bf1b/logs/new/screen-q-svc.txt.gz 2014-12-12 04:38:17.575 10352 INFO neutron.manager [-] Loading Plugin: neutron.services.l3_router.l3_router_plugin.L3RouterPlugin 2014-12-12 04:38:17.648 10352 INFO oslo.messaging._drivers.impl_rabbit [-] Connecting to AMQP server on localhost:5672 2014-12-12 04:38:17.657 10352 INFO oslo.messaging._drivers.impl_rabbit [-] Connected to AMQP server on 127.0.0.1:5672 2014-12-12 04:38:17.667 10352 INFO neutron.db.l3_agentschedulers_db [-] Skipping period L3 agent status check because automatic router rescheduling is disabled. 2014-12-12 04:38:17.667 10352 DEBUG neutron.manager [-] Successfully loaded L3_ROUTER_NAT plugin. Description: L3 Router Service Plugin for basic L3 forwarding between (L2) Neutron networks and access to external networks via a NAT gateway. _load_service_plugins /opt/stack/new/neutron/neutron/manager.py:193 2014-12-12 04:38:17.667 10352 INFO neutron.manager [-] Loading Plugin: neutron.services.loadbalancer.plugin.LoadBalancerPlugin 2014-12-12 04:38:17.743 10352 ERROR neutron.services.service_base [-] Error loading provider 'neutron.services.loadbalancer.drivers.haproxy.plugin_driver.HaproxyOnHostPluginDriver' for service LOADBALANCER 2014-12-12 04:38:17.743 10352 TRACE neutron.services.service_base Traceback (most recent call last): 2014-12-12 04:38:17.743 10352 TRACE neutron.services.service_base File "/opt/stack/new/neutron/neutron/services/service_base.py", line 80, in load_drivers 2014-12-12 04:38:17.743 10352 TRACE neutron.services.service_base provider['driver'], plugin 2014-12-12 04:38:17.743 10352 TRACE neutron.services.service_base File "/usr/local/lib/python2.7/dist-packages/oslo/utils/importutils.py", line 38, in import_object 2014-12-12 04:38:17.743 10352 TRACE neutron.services.service_base return import_class(import_str)(*args, **kwargs) 2014-12-12 04:38:17.743 10352 TRACE neutron.services.service_base File "/usr/local/lib/python2.7/dist-packages/oslo/utils/importutils.py", line 27, in import_class 2014-12-12 04:38:17.743 10352 TRACE neutron.services.service_base __import__(mod_str) 2014-12-12 04:38:17.743 10352 TRACE neutron.services.service_base ImportError: No module named drivers.haproxy.plugin_driver 2014-12-12 04:38:17.743 10352 TRACE neutron.services.service_base 2014-12-12 04:38:17.744 10352 DEBUG neutron.openstack.common.lockutils [-] Releasing semaphore "manager" lock /opt/stack/new/neutron/neutron/openstack/common/lockutils.py:238 2014-12-12 04:38:17.744 10352 DEBUG neutron.openstack.common.lockutils [-] Semaphore / lock released "_create_instance" inner /opt/stack/new/neutron/neutron/openstack/common/lockutils.py:275 2014-12-12 04:38:17.744 10352 ERROR neutron.common.config [-] Unable to load neutron from configuration file /etc/neutron/api-paste.ini. 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config Traceback (most recent call last): 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config File "/opt/stack/new/neutron/neutron/common/config.py", line 189, in load_paste_app 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config app = deploy.loadapp("config:%s" % config_path, name=app_name) 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 247, in loadapp 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config return loadobj(APP, uri, name=name, **kw) 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 272, in loadobj 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config return context.create() 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 710, in create 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config return self.object_type.invoke(self) 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 144, in invoke 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config **context.local_conf) 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config File "/usr/lib/python2.7/dist-packages/paste/deploy/util.py", line 55, in fix_call 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config val = callable(*args, **kw) 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config File "/usr/lib/python2.7/dist-packages/paste/urlmap.py", line 28, in urlmap_factory 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config app = loader.get_app(app_name, global_conf=global_conf) 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 350, in get_app 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config name=name, global_conf=global_conf).create() 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 710, in create 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config return self.object_type.invoke(self) 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 144, in invoke 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config **context.local_conf) 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config File "/usr/lib/python2.7/dist-packages/paste/deploy/util.py", line 55, in fix_call 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config val = callable(*args, **kw) 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config File "/opt/stack/new/neutron/neutron/auth.py", line 71, in pipeline_factory 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config app = loader.get_app(pipeline[-1]) 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 350, in get_app 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config name=name, global_conf=global_conf).create() 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 710, in create 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config return self.object_type.invoke(self) 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 146, in invoke 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config return fix_call(context.object, context.global_conf, **context.local_conf) 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config File "/usr/lib/python2.7/dist-packages/paste/deploy/util.py", line 55, in fix_call 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config val = callable(*args, **kw) 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config File "/opt/stack/new/neutron/neutron/api/v2/router.py", line 69, in factory 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config return cls(**local_config) 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config File "/opt/stack/new/neutron/neutron/api/v2/router.py", line 73, in __init__ 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config plugin = manager.NeutronManager.get_plugin() 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config File "/opt/stack/new/neutron/neutron/manager.py", line 219, in get_plugin 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config return weakref.proxy(cls.get_instance().plugin) 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config File "/opt/stack/new/neutron/neutron/manager.py", line 213, in get_instance 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config cls._create_instance() 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config File "/opt/stack/new/neutron/neutron/openstack/common/lockutils.py", line 272, in inner 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config return f(*args, **kwargs) 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config File "/opt/stack/new/neutron/neutron/manager.py", line 199, in _create_instance 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config cls._instance = cls() 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config File "/opt/stack/new/neutron/neutron/manager.py", line 125, in __init__ 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config self._load_service_plugins() 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config File "/opt/stack/new/neutron/neutron/manager.py", line 172, in _load_service_plugins 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config provider) 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config File "/opt/stack/new/neutron/neutron/manager.py", line 140, in _get_plugin_instance 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config return plugin_class() 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config File "/opt/stack/new/neutron-lbaas/neutron_lbaas/services/loadbalancer/plugin.py", line 55, in __init__ 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config self._load_drivers() 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config File "/opt/stack/new/neutron-lbaas/neutron_lbaas/services/loadbalancer/plugin.py", line 60, in _load_drivers 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config constants.LOADBALANCER, self) 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config File "/opt/stack/new/neutron/neutron/services/service_base.py", line 91, in load_drivers 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config 'service_type': service_type}) 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config File "/usr/local/lib/python2.7/dist-packages/oslo/utils/excutils.py", line 82, in __exit__ 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config six.reraise(self.type_, self.value, self.tb) 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config File "/opt/stack/new/neutron/neutron/services/service_base.py", line 80, in load_drivers 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config provider['driver'], plugin 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config File "/usr/local/lib/python2.7/dist-packages/oslo/utils/importutils.py", line 38, in import_object 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config return import_class(import_str)(*args, **kwargs) 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config File "/usr/local/lib/python2.7/dist-packages/oslo/utils/importutils.py", line 27, in import_class 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config __import__(mod_str) 2014-12-12 04:38:17.744 10352 TRACE neutron.common.config ImportError: No module named drivers.haproxy.plugin_driver