GRETAP Tunnels Drop Packets When Configured With a Bridge As Master

[Steve Williams]

The network setup is to create a layer 2 GRETAP tunnel to connect to segments of a LAN across an OpenVPN tunnel using intervening layer 3 network (in this case a GPRS network, but it could be the Internet).

I found a kernel bug report that matched my problem and described the fix - https://bugzilla.kernel.org/show_bug.cgi?id=14837

I need to be able to set the following options on the GRE tap tunnel device - "ignore-df nopmtudisc" - when the gretap device is created.

On each machine a bridge is created and a local interface is added. A GRETAP tunnel is created and the tunnel is then added to the bridge. Here are the yaml files to achieve that:

/etc/netplan/00-static-remote-gateway.yaml

network:
  version: 2
  renderer: networkd
  ethernets:
    enp1s0:
      dhcp4: no
      dhcp6: no
      addresses:
        - 10.35.2.2/30
      gateway4: 10.35.2.1
      routes:
        - to: 0.0.0.0/0
          via: 10.35.2.1
          metric: 10
      nameservers:
        addresses: []
    enp2s0:
      dhcp4: no
      dhcp6: no
      addresses:
        - 10.35.2.6/30
      routes:
        - to: 10.35.2.4/30
          via: 10.35.2.5
          table: 11
      routing-policy:
        - from: 10.35.2.4/30
          table: 11
    enp3s0:
      dhcp4: no
      dhcp6: no
    enp4s0:
      dhcp4: no
      dhcp6: no

/etc/netplan/20-tunnel-config.yaml

network:
  version: 2
  tunnels:
    gretap1:
      mode: gretap
      local: 10.8.0.3
      remote: 10.8.0.1

/etc/netplan/30-bridge-config.yaml

network:
  version: 2
  bridges:
    lanbr0:
      interfaces: [enp3s0,enp4s0,gretap1]
      parameters:
        stp: yes
      dhcp4: no
      dhcp6: no

A similar config exists on the remote end, with the local and remote addresses swapped. The addresses used are the IP addresses associated with the endpoints of OpenVPN tunnel.

As configured, the tunnel drops larger packets. To fix the issue, the following commands are used to modify the gretap1 device on each end.

ip link del gretap1
ip link add gretap1 mtu 1454 type gretap local 10.8.0.3 remote 10.8.0.1 ignore-df nopmtudisc

The link then shows that DF is ignored and path MTU discovery is off.

ip -d link show gretap1
14: gretap1@NONE: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1462 qdisc fq_codel master lanbr0 state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether 0a:0a:3d:bc:6e:70 brd ff:ff:ff:ff:ff:ff promiscuity 1 minmtu 68 maxmtu 0
    gretap remote 10.8.0.1 local 10.8.0.3 ttl inherit nopmtudisc ignore-df erspan_ver 0
    bridge_slave state forwarding priority 32 cost 100 hairpin off guard off root_block off fastleave off learning on flood on port_id 0x8003 port_no 0x3 designated_port 32771 designated_cost 0 designated_bridge 8000.0:e0:4c:83:e:d0 designated_root 8000.0:e0:4c:83:e:d0 hold_timer 0.00 message_age_timer 19.41 forward_delay_timer 0.00 topology_change_ack 0 config_pending 0 proxy_arp off proxy_arp_wifi off mcast_router 1 mcast_fast_leave off mcast_flood on mcast_to_unicast off neigh_suppress off group_fwd_mask 0 group_fwd_mask_str 0x0 vlan_tunnel off isolated off addrgenmode none numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535

[Andrew Liebman (andrewl733)]

I would vote for fixing this bug. Today I tried to create a gretap tunnel and configure it in netplan but, because I need to add the options "ignore-df nopmtudisc", I had to settle for making a Systemd service to run a script which used "ip link" commands to set up gretap.

I love netplan. It is so logical. But it's a bummer when something major is missing from its configuration capabilities.

[Danilo Egea Gondolfo (danilogondolfo)]

Hello,

I see networkd has the option [TUNNEL].DiscoverPathMTU=

Support for "ignore-df" was merged a few months ago in systemd and apparently are not yet present in systemd releases: https://github.com/systemd/systemd/pull/28131

But we definitely want to support it. We have plans to support everything that is available in networkd so this will be included.