neutron_server not working when lbaas enabled

Bug #1626958 reported by bjolo on 2016-09-23
18
This bug affects 3 people
Affects Status Importance Assigned to Milestone
kolla
Critical
Paul Bourke
Newton
Critical
Paul Bourke
Ocata
Critical
Paul Bourke

Bug Description

neutron_server container fails to start when lbaas is enabled due to missing drivers.

env: kolla master, ubuntu source

full log
http://paste.openstack.org/show/582715

snipplet
2016-09-23 10:53:42.876 7 WARNING stevedore.named [req-18c27add-9e5c-4e5d-a86f-1ede0cd3d918 - - - - -] Could not load neutron_lbaas.drivers.haproxy.plugin_driver.HaproxyOnHostPluginDriver
2016-09-23 10:53:42.876 7 WARNING stevedore.named [req-18c27add-9e5c-4e5d-a86f-1ede0cd3d918 - - - - -] Could not load neutron_vpnaas.services.vpn.service_drivers.ipsec.IPsecVPNDriver
2016-09-23 10:53:42.878 7 INFO neutron.services.service_base [req-18c27add-9e5c-4e5d-a86f-1ede0cd3d918 - - - - -] Default provider is not specified for service type L3_ROUTER_NAT
2016-09-23 10:53:42.879 7 INFO neutron.manager [req-18c27add-9e5c-4e5d-a86f-1ede0cd3d918 - - - - -] Loading Plugin: neutron_lbaas.services.loadbalancer.plugin.LoadBalancerPluginv2
2016-09-23 10:53:42.879 7 WARNING stevedore.named [req-18c27add-9e5c-4e5d-a86f-1ede0cd3d918 - - - - -] Could not load neutron_lbaas.services.loadbalancer.plugin.LoadBalancerPluginv2
2016-09-23 10:53:42.880 7 ERROR neutron.common.utils [req-18c27add-9e5c-4e5d-a86f-1ede0cd3d918 - - - - -] Error loading class by alias
2016-09-23 10:53:42.880 7 ERROR neutron.common.utils Traceback (most recent call last):
2016-09-23 10:53:42.880 7 ERROR neutron.common.utils File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/neutron/common/utils.py", line 403, in load_class_by_alias_or_classname
2016-09-23 10:53:42.880 7 ERROR neutron.common.utils mgr = driver.DriverManager(namespace, name)
2016-09-23 10:53:42.880 7 ERROR neutron.common.utils File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/stevedore/driver.py", line 46, in __init__
2016-09-23 10:53:42.880 7 ERROR neutron.common.utils verify_requirements=verify_requirements,
2016-09-23 10:53:42.880 7 ERROR neutron.common.utils File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/stevedore/named.py", line 77, in __init__
2016-09-23 10:53:42.880 7 ERROR neutron.common.utils self._init_plugins(extensions)
2016-09-23 10:53:42.880 7 ERROR neutron.common.utils File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/stevedore/driver.py", line 98, in _init_plugins
2016-09-23 10:53:42.880 7 ERROR neutron.common.utils (self.namespace, name))
2016-09-23 10:53:42.880 7 ERROR neutron.common.utils NoMatches: No 'neutron.service_plugins' driver found, looking for 'neutron_lbaas.services.loadbalancer.plugin.LoadBalancerPluginv2'
2016-09-23 10:53:42.880 7 ERROR neutron.common.utils
2016-09-23 10:53:42.881 7 ERROR neutron.common.utils [req-18c27add-9e5c-4e5d-a86f-1ede0cd3d918 - - - - -] Error loading class by class name
2016-09-23 10:53:42.881 7 ERROR neutron.common.utils Traceback (most recent call last):
2016-09-23 10:53:42.881 7 ERROR neutron.common.utils File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/neutron/common/utils.py", line 409, in load_class_by_alias_or_classname
2016-09-23 10:53:42.881 7 ERROR neutron.common.utils class_to_load = importutils.import_class(name)
2016-09-23 10:53:42.881 7 ERROR neutron.common.utils File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_utils/importutils.py", line 30, in import_class
2016-09-23 10:53:42.881 7 ERROR neutron.common.utils __import__(mod_str)
2016-09-23 10:53:42.881 7 ERROR neutron.common.utils ImportError: No module named neutron_lbaas.services.loadbalancer.plugin

bjolo (bjorn-lofdahl) wrote :

this bug should be extended for vpnaas as well. same issue as above.

Steven Dake (sdake) on 2016-10-05
Changed in kolla:
status: New → Triaged
importance: Undecided → Critical
milestone: none → newton-rc2
Changed in kolla:
assignee: nobody → Swapnil Kulkarni (coolsvap)
Jeffrey Zhang (jeffrey4l) wrote :

In default, neutron-server do not install neutron lbaasv2 package. you can get some help from http://docs.openstack.org/developer/kolla/image-building.html#plugin-functionality

Swapnil Kulkarni (coolsvap) wrote :

I am not able to reproduce this error with ubuntu source deployment.

bjolo (bjorn-lofdahl) wrote :

strange. i can reproduce consistently on both centos and ubuntu source using builts from today. Are you testing multinode or AIO?

Paul Bourke (pauldbourke) wrote :

It seems like as Jeffrey says the plugins are simply missing from the neutron-server container. They can be included, arguably this could be punted to Ocata as it is possible for the user to add them themselves using the link above.

Changed in kolla:
importance: Critical → High
importance: High → Medium
status: Triaged → Confirmed
Steven Dake (sdake) on 2016-10-13
Changed in kolla:
milestone: newton-rc2 → ocata-1
Steven Dake (sdake) wrote :

I think the fundamental disconnect here is that we are expecting ops to custom build images using the custom building feature for things that are enabled in globals.yml. I'm not sure what the correct answer is, but I think it warrants more discussion. I've changed this back to high (because I think it has backport potential). I think the solution for the short term is to document this in the image building. The problem with that is we probably have alot of cases beyond vpnaas and fwaas where this applies (such as networking-sfc?).

Changed in kolla:
importance: Medium → High
Jeffrey Zhang (jeffrey4l) wrote :

the better solution is: kolla provides some customization files. like add lbaas, vpnaas etc.

bjolo (bjorn-lofdahl) wrote :

my concern is that we are lacking consistency. Looking at the code, binary installs take care of the dependencies, but for source it is omitted.

kolla-newton/docker/neutron/neutron-base/Dockerfile.j2
8 {% if install_type == 'binary' %}
9 {% if base_distro in ['centos', 'fedora', 'oraclelinux', 'rhel'] %}
10
11 {% set neutron_base_packages = [
12 'openstack-neutron',
13 'openstack-neutron-lbaas',
14 'openstack-neutron-ml2',
15 'openvswitch',
16 'python-openvswitch'
17 ] %}
18
19 {% elif base_distro in ['ubuntu'] %}
20
21 {% set neutron_base_packages = [
22 'iproute2',
23 'neutron-plugin-ml2',
24 'neutron-server',
25 'openvswitch-switch',
26 'neutron-lbaas-common',
27 'python-openvswitch'
28 ] %}
29
30 {% endif %}

Changed in kolla:
assignee: Swapnil Kulkarni (coolsvap) → Paul Bourke (pauldbourke)

Fix proposed to branch: master
Review: https://review.openstack.org/393435

Changed in kolla:
status: Confirmed → In Progress
Steven Dake (sdake) wrote :

removing mitaka since this problem only exists in newton tmk.

no longer affects: kolla/mitaka
Changed in kolla:
milestone: ocata-1 → ocata-2

Change abandoned by Paul Bourke (pbourke) (<email address hidden>) on branch: master
Review: https://review.openstack.org/393435
Reason: dupe of https://review.openstack.org/#/c/329487/

Changed in kolla:
milestone: ocata-2 → ocata-3
Christian Berendt (berendt) wrote :

https://review.openstack.org/#/c/329487/ was merged. This should be fixed.

Christian Berendt (berendt) wrote :

Backport for Newton still required.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers