linuxbridge vif plugging error with tap device

Bug #1694719 reported by James Page on 2017-05-31
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
nova-lxd
High
James Page

Bug Description

As reported by OSA:

http://logs.openstack.org/21/468321/5/check/gate-openstack-ansible-os_nova-ansible-func_lxd-ubuntu-xenial/1aee1cf/logs/host/nova/nova-compute.log.txt.gz#_2017-05-31_10_13_57_425

interface plugging fails with trunk of Nova/Neutron/Nova-LXD due to changes in behaviour in neutron under:

  https://github.com/openstack/neutron/commit/1b987be2b55558dcc276fcfc8af6e39f8b6bac16

Specifically, neutron reports back a device type of 'tap' rather than 'linuxbridge'; this type is not currently supported in os_vif_util; under the libvirt driver, this falls back on the non os-vif codebase to execute the plug event for the tap device requests.

2017-05-31 10:13:57.423 28321 DEBUG nova.network.os_vif_util [req-753b239a-c6ff-490b-b8bd-3d6e39335aa0 530917a961524690abb77872c7bb128c f1b40fe190244f4dadddb3906584a2ba - default default] Converting VIF {"profile": {}, "ovs_interfaceid": null, "preserve_on_delete": false, "network": {"bridge": null, "subnets": [{"ips": [{"meta": {}, "version": 4, "type": "fixed", "floating_ips": [], "address": "192.168.74.8"}], "version": 4, "meta": {"dhcp_server": "192.168.74.2"}, "dns": [], "routes": [], "cidr": "192.168.74.0/28", "gateway": {"meta": {}, "version": 4, "type": "gateway", "address": "192.168.74.1"}}], "meta": {"injected": false, "tenant_id": "f1b40fe190244f4dadddb3906584a2ba", "mtu": 1450}, "id": "cc3504a1-94f4-45d3-8cb3-37cec86e720a", "label": "tempest-TestServerBasicOps-155769487-network"}, "devname": "tapf92d08ee-9b", "vnic_type": "normal", "qbh_params": null, "meta": {}, "details": {"port_filter": true}, "address": "fa:16:3e:74:2f:7f", "active": false, "type": "tap", "id": "f92d08ee-9bd4-4cc1-83ea-69f57ff4c2df", "qbg_params": null}: NotFound: not found nova_to_osvif_vif /openstack/venvs/nova-testing/local/lib/python2.7/site-packages/nova/network/os_vif_util.py:413
2017-05-31 10:13:57.424 28321 DEBUG nova.network.os_vif_util [req-753b239a-c6ff-490b-b8bd-3d6e39335aa0 530917a961524690abb77872c7bb128c f1b40fe190244f4dadddb3906584a2ba - default default] No conversion for VIF type tap yet: NotImplementedError nova_to_osvif_vif /openstack/venvs/nova-testing/local/lib/python2.7/site-packages/nova/network/os_vif_util.py:429
2017-05-31 10:13:57.425 28321 ERROR nova.compute.manager [req-753b239a-c6ff-490b-b8bd-3d6e39335aa0 530917a961524690abb77872c7bb128c f1b40fe190244f4dadddb3906584a2ba - default default] [instance: 4ed90174-e8e1-4c98-b0fc-2611ba587e92] Instance failed to spawn: AttributeError: 'NoneType' object has no attribute 'plugin'
2017-05-31 10:13:57.425 28321 ERROR nova.compute.manager [instance: 4ed90174-e8e1-4c98-b0fc-2611ba587e92] Traceback (most recent call last):
2017-05-31 10:13:57.425 28321 ERROR nova.compute.manager [instance: 4ed90174-e8e1-4c98-b0fc-2611ba587e92] File "/openstack/venvs/nova-testing/local/lib/python2.7/site-packages/nova/compute/manager.py", line 2157, in _build_resources
2017-05-31 10:13:57.425 28321 ERROR nova.compute.manager [instance: 4ed90174-e8e1-4c98-b0fc-2611ba587e92] yield resources
2017-05-31 10:13:57.425 28321 ERROR nova.compute.manager [instance: 4ed90174-e8e1-4c98-b0fc-2611ba587e92] File "/openstack/venvs/nova-testing/local/lib/python2.7/site-packages/nova/compute/manager.py", line 1963, in _build_and_run_instance
2017-05-31 10:13:57.425 28321 ERROR nova.compute.manager [instance: 4ed90174-e8e1-4c98-b0fc-2611ba587e92] block_device_info=block_device_info)
2017-05-31 10:13:57.425 28321 ERROR nova.compute.manager [instance: 4ed90174-e8e1-4c98-b0fc-2611ba587e92] File "/openstack/venvs/nova-testing/local/lib/python2.7/site-packages/nova/virt/lxd/driver.py", line 421, in spawn
2017-05-31 10:13:57.425 28321 ERROR nova.compute.manager [instance: 4ed90174-e8e1-4c98-b0fc-2611ba587e92] self.plug_vifs(instance, network_info)
2017-05-31 10:13:57.425 28321 ERROR nova.compute.manager [instance: 4ed90174-e8e1-4c98-b0fc-2611ba587e92] File "/openstack/venvs/nova-testing/local/lib/python2.7/site-packages/nova/virt/lxd/driver.py", line 951, in plug_vifs
2017-05-31 10:13:57.425 28321 ERROR nova.compute.manager [instance: 4ed90174-e8e1-4c98-b0fc-2611ba587e92] self.vif_driver.plug(instance, vif)
2017-05-31 10:13:57.425 28321 ERROR nova.compute.manager [instance: 4ed90174-e8e1-4c98-b0fc-2611ba587e92] File "/openstack/venvs/nova-testing/local/lib/python2.7/site-packages/nova/virt/lxd/vif.py", line 79, in plug
2017-05-31 10:13:57.425 28321 ERROR nova.compute.manager [instance: 4ed90174-e8e1-4c98-b0fc-2611ba587e92] os_vif.plug(vif_obj, instance_info)
2017-05-31 10:13:57.425 28321 ERROR nova.compute.manager [instance: 4ed90174-e8e1-4c98-b0fc-2611ba587e92] File "/openstack/venvs/nova-testing/local/lib/python2.7/site-packages/os_vif/__init__.py", line 72, in plug
2017-05-31 10:13:57.425 28321 ERROR nova.compute.manager [instance: 4ed90174-e8e1-4c98-b0fc-2611ba587e92] plugin_name = vif.plugin
2017-05-31 10:13:57.425 28321 ERROR nova.compute.manager [instance: 4ed90174-e8e1-4c98-b0fc-2611ba587e92] AttributeError: 'NoneType' object has no attribute 'plugin'
2017-05-31 10:13:57.425 28321 ERROR nova.compute.manager [instance: 4ed90174-e8e1-4c98-b0fc-2611ba587e92]

This is slightly tricky, as currently the nova-lxd driver depends on the pre-creation of the bridge (which to-date was done by the nova-compute driver) as the actual tapXXX device is not created until the instance is launched under LXD.

James Page (james-page) wrote :

Related bug 1681758

James Page (james-page) on 2017-05-31
Changed in nova-lxd:
status: New → Triaged
importance: Undecided → High
assignee: nobody → James Page (james-page)
status: Triaged → In Progress

Reviewed: https://review.openstack.org/469556
Committed: https://git.openstack.org/cgit/openstack/nova-lxd/commit/?id=16866d2cf8ce8c399a23e80668e4c4b53ff07f23
Submitter: Jenkins
Branch: master

commit 16866d2cf8ce8c399a23e80668e4c4b53ff07f23
Author: James Page <email address hidden>
Date: Wed May 31 17:15:50 2017 +0100

    Refactor container VIF handling for linuxbridge

    Neutron recently changed behaviour to complete all bridge setup
    and configuration for the linuxbridge driver via the neutron
    linuxbridge agent, requiring Nova simply to setup the tap device
    that subsequently gets plugged into a linuxbridge.

    Rework plug/unplug handling based on libvirt driver to fallback
    to a legacy style plug/unplug driver for tap devices of this kind.

    In the case of LXD we actually still use a veth pair so that:

      a) security group rules are applied to the tap named
         device on the host.
      b) the container still gets part of a nic to use, named
         ethX internally to the container.
      c) the neutron linuxbridge agent can see the required tapXXX
         device prior to the container being created, allowing
         VIF plugging to be completed.

    This looks something like this once wired and running:

               Host | Container

      [bridge] <-> [tapXXX|tinXXX] <-> [ethX]

    The veth pair is mapping into a LXD container using the physical
    LXD nic type.

    As the drive now creates the veth pair for unbridged network
    types, unplug must occur after the device has been removed
    from the container during interface_detach.

    Rework LXD device profile naming for consistency:

      a) VIFs attaching to bridges will be named inline with the
         bridge (no-change)
      b) VIFs not being attached to a bridge will be named with
         the VIF devname (changed from 'unbridged' which did not
         support any multiplicity).

    Change-Id: I2fdf41e5640f5ca5e3bcd7df1aa159a65b706138
    Closes-Bug: 1694719

Changed in nova-lxd:
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers