Neutron advertises too high MTU for vxlan

[Dr. Jens Harbott]

When creating a tenant network with type vxlan, the MTU is automatically set to 1450:

# neutron net-show net2
+---------------------------+--------------------------------------+
| Field | Value |
+---------------------------+--------------------------------------+
| admin_state_up | True |
| availability_zone_hints | |
| availability_zones | nova |
| created_at | 2016-04-08T11:11:42 |
| description | |
| id | f44e9e2c-8a60-46c7-98bb-3f1824fc09e9 |
| ipv4_address_scope | |
| ipv6_address_scope | |
| mtu | 1450 |
| name | net2 |
| provider:network_type | vxlan |
| provider:physical_network | |
| provider:segmentation_id | 65633 |
| router:external | False |
| shared | False |
| status | ACTIVE |
| subnets | 288021c1-7073-41c5-a233-529226971dd3 |
| tags | |
| tenant_id | e3c253d3e8344a8796e70bc4f96b6166 |
| updated_at | 2016-04-08T11:11:42 |
+---------------------------+--------------------------------------+

This is the maximum Ethernet MTU possible for the tenant assuming the encapsulated packet has to fit into an IP MTU of 1500 on the tunnel network.

Now neutron tells DHCP to set --dhcp-option-force=option:mtu,1450 but the DHCP option refers to the IP layer MTU, see https://tools.ietf.org/html/rfc2132 section 5.1. So a correctly behaving client will set its interface (Ethernet) MTU to 1454, implying an IP MTU of 1450.

But now it will send Ethernet frames of size 1454, which encapsulated have size 1504, and thus get dropped on the tunnel network.

The correct behaviour here would be to advertise an MTU reduced by 4 via DHCP.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/303399

[Dr. Jens Harbott (j-harbott)]

The error lies in the way the vxlan driver calculates the inner MTU, it uses a value of 50 bytes to subtract from the outside MTU. However that value neglects the 4 bytes of ethernet header on the inner client side, the correct fix will be to use a value of 54 and generate an inner MTU of 1446 for the default outside MTU of 1500 bytes.

[Dr. Jens Harbott (j-harbott)]

Hmm, no, there is something else going on here, I'm confused. Looking closer, all my cloud instances set their MTU to 1454, regardless of whether they are in an Vxlan or Vlan based network, so no matter whether dnsmasq is configured with MTU 1450 or 1500.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Jens Rosenboom (<email address hidden>) on branch: master
Review: https://review.openstack.org/303399
Reason: Invalid, see bug report

[Dr. Jens Harbott (j-harbott)]

Gah, turns out that chef installed a file /etc/neutron/dnsmasq.conf containing

+dhcp-option=26,1454

which overrides all other options. Sorry to the Neutron folks for the false alarm, I'll go and fix our cookbook.