cannot connect to VMs via gre tunnel with openvswitch

[Tomokazu Hirai]

I can not connect to VM via GRE tunnel with openvswitch.

my environments are

* 3 nodes (controller, network, compute)
* openvswitch and tunnel_type : gre

network node and compute node have such /etc/neutron/plugins/openvswitch/ovs_neuron_plugin.ini

    [ovs]
    tenant_network_type = gre
    network_vlan_ranges =
    enable_tunneling = True
    tunnel_type = gre
    tunnel_id_ranges = 1:1000
    integration_bridge = br-int
    tunnel_bridge = br-tun
    local_ip = 10.200.10.59
    bridge_mappings = physnet1:br-eth1
    [agent]
    [securitygroup]
    firewall_driver = neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver
    [database]
    connection = mysql://ovsuser:ovsuser@10.200.10.57:3306/ovs_neutron

openvswitch agent on network node has no error. on compute node has such error messages.

    2013-10-11 14:57:59.880 11214 ERROR neutron.agent.linux.ovs_lib [-] Unable to execute ['ovs-ofctl', 'add-flow', 'br-tun', 'hard_timeout=0,idle_timeout=0,priority=1,in_port=-1,actions=resubmit(,2)']. Exception:
Command: ['sudo', '/usr/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ovs-ofctl', 'add-flow', 'br-tun', 'hard_timeout=0,idle_timeout=0,priority=1,in_port=-1,actions=resubmit(,2)']
    Exit code: 1
    Stdout: ''
    Stderr: 'ovs-ofctl: -1: negative values not supported for in_port\n'

this error is openvswitch's standard error. setup_tunnel_port() function on neutron/plugins/openvswitch/agent/ovs_neutron_agent.py has code such as ...

        if ofport < 0:
            LOG.error(_("Failed to set-up %(type)s tunnel port to %(ip)s"),
                      {'type': tunnel_type, 'ip': remote_ip})
            return 0

ofport include strings. "if int(ofport) < 0" is correct ?

this is not problem.. So network node and compute node have tunnel settings with openvswitch.

    network# ovs-vsctl show | grep remote
                    options: {in_key=flow, local_ip="10.200.10.58", out_key=flow, remote_ip="10.200.10.59"}
    compute# ovs-vsctl show | grep remote
                    options: {in_key=flow, local_ip="10.200.10.59", out_key=flow, remote_ip="10.200.10.58"}

but ovs-ofctl shows that there is no gre-1 on compute node.

    compute# ovs-ofctl show br-tun
    <...snip...>
     1(patch-int): addr:12:06:8c:21:76:a0
         config: 0
         state: 0
         speed: 0 Mbps now, 0 Mbps max
     LOCAL(br-tun): addr:36:20:55:99:a1:45
         config: 0
         state: 0
         speed: 0 Mbps now, 0 Mbps max
    OFPT_GET_CONFIG_REPLY (xid=0x4): frags=normal miss_send_len=0

There is gre-2 on network node.

    # ovs-ofctl show br-tun
     <...snip...>
     1(patch-int): addr:7a:2d:5e:02:e3:fe
         config: 0
         state: 0
         speed: 0 Mbps now, 0 Mbps max
     2(gre-2): addr:32:86:3c:69:0e:c3
         config: 0
         state: 0
         speed: 0 Mbps now, 0 Mbps max
     LOCAL(br-tun): addr:96:ea:54:a0:fb:4e
         config: 0
         state: 0
         speed: 0 Mbps now, 0 Mbps max
    OFPT_GET_CONFIG_REPLY (xid=0x4): frags=normal miss_send_len=0

Any Idea ? Thanks.

[tuanvc (tuanvcw)]

Having the same problem, any suggestion?

Stderr: 'ovs-ofctl: -1: negative values not supported for in_port\n'

[Danie van Zyl (danie-l)]

Hello,

I'm getting the same error on one of my compute nodes,

2013-10-18 07:22:16.823 19398 ERROR neutron.agent.linux.ovs_lib [-] Unable to execute ['ovs-ofctl', 'add-flow', 'br-tun', 'hard_timeout=0,idle_timut=0,priority=1,in_port=-1,actions=resubmit(,2)']. Exception:
Command: ['sudo', '/usr/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ovs-ofctl', 'add-flow', 'br-tun', 'hard_timeout=0,idle_timeout=0,priity=1,in_port=-1,actions=resubmit(,2)']
Exit code: 1
Stdout: ''
Stderr: 'ovs-ofctl: -1: negative values not supported for in_port\n'

And when I tried to run the command manually, I get this.

# sudo /usr/bin/neutron-rootwrap /etc/neutron/rootwrap.conf ovs-ofctl add-flow br-tun hard_timeout=0,idle_timeout=0,priority=1,in_port=-1,actions=resubmit\(,2\)
ovs-ofctl: -1: negative values not supported for in_port

Please is there a work around?

[Michael Petersen (mpetason)]

Same Issue:

2013-10-18 14:21:11.189 4527 ERROR neutron.agent.linux.ovs_lib [-] Unable to execute ['ovs-ofctl', 'add-flow', 'br-tun', 'hard_timeout=0,idle_timeout=0,priority=1,in_port=-1,actions=resubmit(,2)']. Exception:
Command: ['sudo', '/usr/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ovs-ofctl', 'add-flow', 'br-tun', 'hard_timeout=0,idle_timeout=0,priority=1,in_port=-1,actions=resubmit(,2)']
Exit code: 1
Stdout: ''
Stderr: 'ovs-ofctl: -1: negative values not supported for in_port\n'

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/53636

[Tomokazu Hirai (tomokazu-hirai)]

Thanks for your comment. :D

Using Saucy (13.10) is workarrond for this issue ?

my environments are ..

% lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 12.04.2 LTS
Release: 12.04
Codename: precise

% uname -a
Linux grizzly0402 3.5.0-23-generic #35~precise1-Ubuntu SMP Fri Jan 25 17:13:26 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

% dpkg -l | grep openvswitch
ii neutron-plugin-openvswitch 1:2013.2-0ubuntu1~cloud0 Neutron is a virtual network service for Openstack - Open vSwitch plugin
ii neutron-plugin-openvswitch-agent 1:2013.2-0ubuntu1~cloud0 Neutron is a virtual network service for Openstack - Open vSwitch plugin agent
ii openvswitch-common 1.10.2-0ubuntu2~cloud0 Open vSwitch common components
ii openvswitch-datapath-dkms 1.10.2-0ubuntu2~cloud0 Open vSwitch datapath module source - DKMS version
ii openvswitch-switch 1.10.2-0ubuntu2~cloud0 Open vSwitch switch implementations

[Tomokazu Hirai (tomokazu-hirai)]

My OS version is Ubuntu 12.04.2 LTS. Actuary this is old .
and OVS kernel module is installed when installing OVS.

So I try to do clean installation new OS of precise 12.04.3 LTS and deploy OpenStack.

Thanks.

[Arata Notsu (arata776)]

Hello Tomokazu,

It seems you have a right version of OVS....

Anyway, with Saucy, you will not be affected by this issue without something special.
With Precise, to use OVS 1.10.2, you may have to use cloud-archive:havana and install openvswitch-datapath-dkms explicitly.

[Akihiro Motoki (amotoki)]

Thank you for diving into the detail.

OVS 1.10 is not a requirement for OVS agent with GRE.
version 1.10 of OVS is just a requirement to use VXLAN.
It seems something is wrong when using GRE with OVS <1.10.

[Arata Notsu (arata776)]

This comment is correction for #5: use 127.0.0.1 as remote_ip, not a.b.c.d
-----

This issue occur if we try to use gre with old kernel module of openvswitch (maybe before 1.9.0).

This is an example. A gre port is created but actually does not work. The "ofport" is "-1", which is passed to ovs-ofctl as "in_port" and makes it fail.

$ sudo ovs-vsctl add-port br0 gre0 -- set interface gre0 type=gre options:remote_ip=127.0.0.1
$ sudo ovs-vsctl list Interface gre0
_uuid : 5e582134-37bf-49bd-91ec-fe602154e68f
admin_state : []
cfm_fault : []
cfm_fault_status : []
cfm_health : []
cfm_mpid : []
cfm_remote_mpids : []
cfm_remote_opstate : []
duplex : []
external_ids : {}
ingress_policing_burst: 0
ingress_policing_rate: 0
lacp_current : []
link_resets : []
link_speed : []
link_state : []
mac : []
mac_in_use : []
mtu : []
name : "gre0"
ofport : -1
ofport_request : []
options : {remote_ip="127.0.0.1"}
other_config : {}
statistics : {}
status : {}
type : gre

[Joseph P (etlars)]

I've suffered from the same problems like this:

Command: ['sudo', '/usr/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'gre-2', 'ofport']
Exit code: 0
Stdout: '-1\n'
Stderr: '' execute /usr/lib/python2.7/dist-packages/neutron/agent/linux/utils.py:60
2013-10-28 15:13:46.247 29385 ERROR neutron.plugins.openvswitch.agent.ovs_neutron_agent [-] Failed to set-up gre tunnel port

and finally I found a mistake in external network setting as following:

# neutron net-show ext-net
+---------------------------+--------------------------------------+
| Field | Value |
+---------------------------+--------------------------------------+
| admin_state_up | True |
| name | ext-net |
| provider:network_type | local |
| provider:physical_network | |
| provider:segmentation_id | |
| router:external | True |

I set provider:network_type as "local" therefore external network couldn't get segmentation_id.
So I changed provider:network_type as "gre" and rebooted all nodes.
And finally I can get following results without errors on gre tunnels.

# neutron net-show ext-net
+---------------------------+--------------------------------------+
| Field | Value |
+---------------------------+--------------------------------------+
| admin_state_up | True |
| id | 475639ac-8d37-4cb9-8aea-3b7cc688faab |
| name | ext-net |
| provider:network_type | gre |
| provider:physical_network | |
| provider:segmentation_id | 2 |
| router:external | True |
| shared | False |

root@compute1:~# ovs-ofctl show br-tun
OFPT_FEATURES_REPLY (xid=0x2): dpid:0000e68187fbbb4f
n_tables:254, n_buffers:256
capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP
actions: OUTPUT SET_VLAN_VID SET_VLAN_PCP STRIP_VLAN SET_DL_SRC SET_DL_DST SET_NW_SRC SET_NW_DST SET_NW_TOS SET_TP_SRC SET_TP_DST ENQUEUE
 1(patch-int): addr:d6:1b:92:8a:43:7b
     config: 0
     state: 0
     speed: 0 Mbps now, 0 Mbps max
 2(gre-1): addr:56:a9:f8:70:1d:6c
     config: 0
     state: 0
     speed: 0 Mbps now, 0 Mbps max
 LOCAL(br-tun): addr:e6:81:87:fb:bb:4f
     config: 0
     state: 0
     speed: 0 Mbps now, 0 Mbps max
OFPT_GET_CONFIG_REPLY (xid=0x4): frags=normal miss_send_len=0

root@network:~/log# ovs-ofctl show br-tun
OFPT_FEATURES_REPLY (xid=0x2): dpid:00007e6773dc2344
n_tables:254, n_buffers:256
capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP
actions: OUTPUT SET_VLAN_VID SET_VLAN_PCP STRIP_VLAN SET_DL_SRC SET_DL_DST SET_NW_SRC SET_NW_DST S...

[Vivekanandan Narasimhan (vivekanandan-narasimhan)]

Have been hit by this problem very hard.

The fix proposed being proposed is throwing the right error. However, could someone please let me know what is
the root cause for the error itself.

Why would Openvswitch issue -1 as the OFPort number for the tunnel endpoint ports?

When digged through the /var/log/openvswitch/ovs-vswitchd.log, I found this:
2013-10-30T14:22:41Z|00076|dpif|WARN|system@ovs-system: failed to add vxlan-1 as port: Address family not supported by protocol

Could someone please let me know would this error appear? This causes the openflow port not to be allocated to the tunnel endpoint.

It looks to me that the kernel module of openvswitch I have is the right one : 1.10.2 and I've havana rc3 running uniformly on my CN:
cn1:/var/log/openvswitch$
cn1:/var/log/openvswitch$ vi ovs-vswitchd.log
cn1:/var/log/openvswitch$ modinfo openvswitch
filename: /lib/modules/3.5.0-32-generic/updates/dkms/openvswitch.ko
version: 1.10.2
license: GPL
description: Open vSwitch switching datapath
srcversion: C57BFF12C03B30A057F0B4B
depends:
vermagic: 3.5.0-32-generic SMP mod_unload modversions
cn1:/var/log/openvswitch$
cn1:/var/log/openvswitch$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 12.04.3 LTS
Release: 12.04
Codename: precise
cn1:/var/log/openvswitch$ dpkg -l | grep openvswitch
ii neutron-plugin-openvswitch 1:2013.2~rc3-0ubuntu1~cloud0 Neutron is a virtual network service for Openstack - Open vSwitch plugin
ii neutron-plugin-openvswitch-agent 1:2013.2~rc3-0ubuntu1~cloud0 Neutron is a virtual network service for Openstack - Open vSwitch plugin agent
ii openvswitch-common 1.10.2-0ubuntu2~cloud0 Open vSwitch common components
ii openvswitch-datapath-dkms 1.10.2-0ubuntu2~cloud0 Open vSwitch datapath module source - DKMS version
ii openvswitch-switch 1.10.2-0ubuntu2~cloud0 Open vSwitch switch implementations
cn1:/var/log/openvswitch$

[Kyle Mestery (mestery)]

You need to make sure you are running the OVS 1.10 kernel module in addition to the OVS 1.10 userspace. Both are required for VXLAN support. Please make sure you are running the right kernel module. Make sure "modinfo openvswitch" shows the right version. Also check "dmesg" for the version of the OVS module being loaded.

[Vivekanandan Narasimhan (vivekanandan-narasimhan)]

Thanks for the response Kyle.

I enclosed the 'mod-info openvswitch' in my comment above. It shows 1.10.2 which also tallies with what was in the Ubuntu Canonical Repo for openvswitch-switch deb pkg for Havana RC3.

The module is the same that I ran on the other CN node in my 3-node setup. That CN (cn-2) is able to create Tunnel endpoint with positive OFPort numbers with the cn1 and the even to nn in the setup.

Though cn1 and nn are able to create the OVS ports, but when requested for OFPort number they return -1 as the port is not available in the OF Datapath, thereby unable to do tunneling between themselves and cn2.

[tsjsdbd (tsjsdbd)]

I fellow this guide, and resolve this problem, the same problem.

sudo apt-get install openvswitch-datapath-dkms
Then reboot

GRE tunnels not being created
https://ask.openstack.org/en/question/6598/gre-tunnels-not-being-created/

[Igor D.C. (igordcard)]

For tsjsdbd and others that found openvswitch-datapath-dkms useful for fixing their problem, consider taking a look at this bug report for OpenStack Manuals, specifically the install guide for Havana, where it should be clear that openvswitch-datapath-dkms or something similar is necessary for a proper deployment of Havana on Ubuntu 12.04 LTS with GRE as the networking technology: https://bugs.launchpad.net/openstack-manuals/+bug/1249589.

[Michaël Van de Borne (mikemowgli)]

I'm afraid that didn't do the trick for me... I installed the kernel module, and I rebooted, but the GRE tunnel is still missing in OpenFlow output...

[Michaël Van de Borne (mikemowgli)]

Ok that finally with the openvswitch kernel module installation.

[Igor D.C. (igordcard)]

How did you manage to?

On 12 November 2013 08:48, Michaël Van de Borne <<email address hidden>
> wrote:

> Ok that finally with the openvswitch kernel module installation.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1238445
>
> Title:
> cannot connect to VMs via gre tunnel with openvswitch
>
> Status in OpenStack Neutron (virtual network service):
> In Progress
>
> Bug description:
> I can not connect to VM via GRE tunnel with openvswitch.
>
> my environments are
>
> * 3 nodes (controller, network, compute)
> * openvswitch and tunnel_type : gre
>
> network node and compute node have such
> /etc/neutron/plugins/openvswitch/ovs_neuron_plugin.ini
>
> [ovs]
> tenant_network_type = gre
> network_vlan_ranges =
> enable_tunneling = True
> tunnel_type = gre
> tunnel_id_ranges = 1:1000
> integration_bridge = br-int
> tunnel_bridge = br-tun
> local_ip = 10.200.10.59
> bridge_mappings = physnet1:br-eth1
> [agent]
> [securitygroup]
> firewall_driver =
> neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver
> [database]
> connection = mysql://ovsuser:ovsuser@10.200.10.57:3306/ovs_neutron
>
> openvswitch agent on network node has no error. on compute node has
> such error messages.
>
> 2013-10-11 14:57:59.880 11214 ERROR neutron.agent.linux.ovs_lib [-]
> Unable to execute ['ovs-ofctl', 'add-flow', 'br-tun',
> 'hard_timeout=0,idle_timeout=0,priority=1,in_port=-1,actions=resubmit(,2)'].
> Exception:
> Command: ['sudo', '/usr/bin/neutron-rootwrap',
> '/etc/neutron/rootwrap.conf', 'ovs-ofctl', 'add-flow', 'br-tun',
> 'hard_timeout=0,idle_timeout=0,priority=1,in_port=-1,actions=resubmit(,2)']
> Exit code: 1
> Stdout: ''
> Stderr: 'ovs-ofctl: -1: negative values not supported for in_port\n'
>
> this error is openvswitch's standard error. setup_tunnel_port()
> function on neutron/plugins/openvswitch/agent/ovs_neutron_agent.py has
> code such as ...
>
> if ofport < 0:
> LOG.error(_("Failed to set-up %(type)s tunnel port to
> %(ip)s"),
> {'type': tunnel_type, 'ip': remote_ip})
> return 0
>
> ofport include strings. "if int(ofport) < 0" is correct ?
>
> this is not problem.. So network node and compute node have tunnel
> settings with openvswitch.
>
> network# ovs-vsctl show | grep remote
> options: {in_key=flow, local_ip="10.200.10.58",
> out_key=flow, remote_ip="10.200.10.59"}
> compute# ovs-vsctl show | grep remote
> options: {in_key=flow, local_ip="10.200.10.59",
> out_key=flow, remote_ip="10.200.10.58"}
>
>
> but ovs-ofctl shows that there is no gre-1 on compute node.
>
> compute# ovs-ofctl show br-tun
> <...snip...>
> 1(patch-int): addr:12:06:8c:21:76:a0
> config: 0
> state: 0
> speed: 0 Mbps now, 0 Mbps max
> LOCAL(br-tun): addr:36:20:55:99:a1:45
> config: 0
> state: 0
> speed: 0 Mbps...

[Baodong (Robert) Li (baoli)]

This is caused by the incompatible/old version of OVS running on the
compute node. I manually installed a recent version of OVS, and the
problem went away.

--Robert

On 11/12/13 4:18 AM, "Igor Duarte Cardoso" <email address hidden>
wrote:

>How did you manage to?
>
>
>On 12 November 2013 08:48, Michaël Van de Borne
><<email address hidden>
>> wrote:
>
>> Ok that finally with the openvswitch kernel module installation.
>>
>> --
>> You received this bug notification because you are subscribed to the bug
>> report.
>> https://bugs.launchpad.net/bugs/1238445
>>
>> Title:
>> cannot connect to VMs via gre tunnel with openvswitch
>>
>> Status in OpenStack Neutron (virtual network service):
>> In Progress
>>
>> Bug description:
>> I can not connect to VM via GRE tunnel with openvswitch.
>>
>> my environments are
>>
>> * 3 nodes (controller, network, compute)
>> * openvswitch and tunnel_type : gre
>>
>> network node and compute node have such
>> /etc/neutron/plugins/openvswitch/ovs_neuron_plugin.ini
>>
>> [ovs]
>> tenant_network_type = gre
>> network_vlan_ranges =
>> enable_tunneling = True
>> tunnel_type = gre
>> tunnel_id_ranges = 1:1000
>> integration_bridge = br-int
>> tunnel_bridge = br-tun
>> local_ip = 10.200.10.59
>> bridge_mappings = physnet1:br-eth1
>> [agent]
>> [securitygroup]
>> firewall_driver =
>> neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver
>> [database]
>> connection = mysql://ovsuser:ovsuser@10.200.10.57:3306/ovs_neutron
>>
>> openvswitch agent on network node has no error. on compute node has
>> such error messages.
>>
>> 2013-10-11 14:57:59.880 11214 ERROR neutron.agent.linux.ovs_lib
>>[-]
>> Unable to execute ['ovs-ofctl', 'add-flow', 'br-tun',
>>
>>'hard_timeout=0,idle_timeout=0,priority=1,in_port=-1,actions=resubmit(,2)
>>'].
>> Exception:
>> Command: ['sudo', '/usr/bin/neutron-rootwrap',
>> '/etc/neutron/rootwrap.conf', 'ovs-ofctl', 'add-flow', 'br-tun',
>>
>>'hard_timeout=0,idle_timeout=0,priority=1,in_port=-1,actions=resubmit(,2)
>>']
>> Exit code: 1
>> Stdout: ''
>> Stderr: 'ovs-ofctl: -1: negative values not supported for
>>in_port\n'
>>
>> this error is openvswitch's standard error. setup_tunnel_port()
>> function on neutron/plugins/openvswitch/agent/ovs_neutron_agent.py has
>> code such as ...
>>
>> if ofport < 0:
>> LOG.error(_("Failed to set-up %(type)s tunnel port to
>> %(ip)s"),
>> {'type': tunnel_type, 'ip': remote_ip})
>> return 0
>>
>> ofport include strings. "if int(ofport) < 0" is correct ?
>>
>> this is not problem.. So network node and compute node have tunnel
>> settings with openvswitch.
>>
>> network# ovs-vsctl show | grep remote
>> options: {in_key=flow, local_ip="10.200.10.58",
>> out_key=flow, remote_ip="10.200.10.59"}
>> compute# ovs-vsctl show | grep remote
>> options: {in_key=flow, local_ip="10.200.10.59",
>> out_key=flow, remote_ip="10.200.10.58"}
>>
>>
>> but ovs-ofctl shows that there is no gre-1 ...

[Michaël Van de Borne (mikemowgli)]

I just rebooted one again the network node and the compute node. It happens I have played a bit with modprobe after my first reboot and it seems OVS didn't liked it...

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/53636
Committed: http://github.com/openstack/neutron/commit/edddad586332335578477df464be13f5a2b4a45f
Submitter: Jenkins
Branch: master

commit edddad586332335578477df464be13f5a2b4a45f
Author: Arata Notsu <email address hidden>
Date: Fri Oct 25 16:34:22 2013 +0900

    Do not run "ovs-ofctl add-flow" with an invalid in_port

    Since ofport is a string, we have to convert it to integer to
    check it expresses a negative number or not.

    Also unit tests are added/corrected.

    Change-Id: If85595f1b8ecb40ea41edda52706fb1bd41cb1ab
    Related-Bug: 1238445

[sankalp82 (sankalpjain202)]

i Have also same problem
My VM do not get IP from dhcp !!!

[Michaël Van de Borne (mikemowgli)]

sankalp82, just install openvswitch-datapath-dkms, then reboot

[sankalp82 (sankalpjain202)]

@mike Thanks ,
still same problem my VM do not get IP at booting time

[Michaël Van de Borne (mikemowgli)]

What are the output of "sudo ovs-vsctl show" and "sudo ovs-ofctl show br-tun" ?

[sankalp82 (sankalpjain202)]

Hi @Michael !!!!!

sudo ovs-vsctl show
7fe8eef6-f4e5-4f80-b877-39dc02f08a66
    Bridge br-int
        Port "tap2dc1c852-d5"
            tag: 1
            Interface "tap2dc1c852-d5"
                type: internal
        Port br-int
            Interface br-int
                type: internal
        Port patch-tun
            Interface patch-tun
                type: patch
                options: {peer=patch-int}
    Bridge br-tun
        Port "gre-1"
            Interface "gre-1"
                type: gre
                options: {in_key=flow, local_ip="192.168.20.143", out_key=flow, remote_ip="192.168.20.114"}
        Port patch-int
            Interface patch-int
                type: patch
                options: {peer=patch-tun}
        Port br-tun
            Interface br-tun
                type: internal
    Bridge br-ex
        Port "eth1"
            Interface "eth1"
        Port br-ex
            Interface br-ex
                type: internal
    ovs_version: "1.10.2"

sudo ovs-ofctl show br-tun
OFPT_FEATURES_REPLY (xid=0x2): dpid:000036d80bc2754d
n_tables:254, n_buffers:256
capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP
actions: OUTPUT SET_VLAN_VID SET_VLAN_PCP STRIP_VLAN SET_DL_SRC SET_DL_DST SET_NW_SRC SET_NW_DST SET_NW_TOS SET_TP_SRC SET_TP_DST ENQUEUE
 1(patch-int): addr:e6:44:48:07:ad:c2
     config: 0
     state: 0
     speed: 0 Mbps now, 0 Mbps max
 2(gre-1): addr:c2:35:e9:dc:8b:35
     config: 0
     state: 0
     speed: 0 Mbps now, 0 Mbps max
 LOCAL(br-tun): addr:36:d8:0b:c2:75:4d
     config: 0
     state: 0
     speed: 0 Mbps now, 0 Mbps max
OFPT_GET_CONFIG_REPLY (xid=0x4): frags=normal miss_send_len=0

[Arata Notsu (arata776)]

What is the result of "sudo ovs-vsctl list Interface gre-1" ?

[Arata Notsu (arata776)]

If this bug just results from the mismatch of version between kernel and userspace, I have no idea what I can do further in neutron.

[Arata Notsu (arata776)]

Is this considered still not resolved?

[Darren Birkett (darren-birkett)]

Just to add to this,

I am trying to use gre tunnels as above. My kernel and userpace versions of openvswitch DO match (1.10.2). I HAVE got openvswitch-datapath-dkms. I get similar errors to those described above. I have tried forcing a reload of the kernel module, but still the errors persist.

But when I reboot, everything is resolved. I'd like to avoid having to reboot as I am deploying via chef. So I'd say there is still some kind of issue here, even after applying the suggested 'fixes'

[Darren Birkett (darren-birkett)]

For anyone else reading, it turns out that sometimes the openvswitch kernel module that gets compiled against your current kernel by the installation of openvswitch-datapath-dkms, does not get reloaded properly.

There is a service on ubuntu that is meant to do this, called openvswitch-force-reload-kmod. In my case, running this did not work. I had to manually do the following:

/usr/share/openvswitch/scripts/ovs-ctl force-reload-kmod
 * Detected internal interfaces: br-int br-tun
 * Saving flows
 * Killing ovsdb-server (373)
 * Starting ovsdb-server
 * system ID not configured, please use --system-id
 * Configuring Open vSwitch system IDs
 * Killing ovs-vswitchd (385)
 * Saving interface configuration
 * Removing datapath: system@ovs-system
 * Removing openvswitch module
 * Inserting openvswitch module
 * Starting ovs-vswitchd
 * Enabling remote OVSDB managers
 * Restoring saved flows
 * Restoring interface configuration

This then loaded the correct kernel version and did not require a reboot to do so.

[Launchpad Janitor (janitor)]

[Expired for neutron because there has been no activity for 60 days.]