ifupdown does not set ipv6-only large mtu

Bug #1609367 reported by Dan Streetman on 2016-08-03
28
This bug affects 5 people
Affects Status Importance Assigned to Milestone
ifupdown (Ubuntu)
Medium
Unassigned

Bug Description

ifupdown changes a device's mtu differently, between "inet" section mtu and "inet6" section mtu. For "inet", ifupdown changes the device's mtu, using 'ip link set DEV mtu NNN'. However for "inet6", ifupdown changes the interface's IPv6 mtu, using 'sysctl -q -e -w net.ipv6.conf.DEV.mtu=NNN'.

The problem is that a device's ipv6 mtu cannot be larger than the device mtu. Normally this is not a problem; the main reason a device needs a separate ipv6 mtu is to support ipv6 tunneling inside ipv4 (e.g. 6in4 or 6rd), so ifupdown would also have a ipv4 "inet" section that contained a larger mtu (or, the ipv6 section would have a lower-than-default mtu).

But, in the case of an interface that is configured only for ipv6 *and* large frames (e.g. 9000 mtu), there is only an "inet6" section in the ifupdown config, and so ifupdown fails to set the device's mtu, which causes its attempt to set the ipv6 mtu to fail.

I believe the correct behavior is for ifupdown to check if there is any "inet" config section for the same device as the "inet6" section, and if there is not then it should change the device mtu as well as the ipv6 mtu. Additionally, the "inet6" configuration must be done after the "inet" configuration for the same device, since the ipv6 mtu can't be higher than the device mtu.

I don't see any easy way to add that logic to inet6.defn, however...

Related branches

Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in ifupdown (Ubuntu):
status: New → Confirmed
Ryan Harper (raharper) wrote :

Can you explain which part fails?

"ifupdown fails to set the device's mtu, which causes its attempt to set the ipv6 mtu to fail"

This is the change that drove the use of sysctl to set mtu:

https://anonscm.debian.org/git/collab-maint/ifupdown.git/commit/?id=a1b2c5555fea935d91961dc2df3ac5cfe6668bcf

which comes from bug:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=809714

and the tl;dr is that IPv6 supports per protocol, rather than interface.

It appears that we want the opposite here; so it's not quite clear how to resolve this.

In the case that the user doesn't want the underlying interface MTU to change, then the sysctl is needed to change the MTU *only* for IPv6.

In the original bug where one is comparing ip output of MTU; is there something *not* working ? Or did we just observe that the bond interface isn't the same MTU? Can we confirm that without the bond interface itself set to MTU 9202 that we don't see the correct jumbo frames over ipv6?

Changed in ifupdown (Ubuntu):
status: Confirmed → Incomplete
Dan Streetman (ddstreet) wrote :

> and the tl;dr is that IPv6 supports per protocol, rather than interface.

the ipv6 mtu is for situations where the host's ipv6 connection uses (at some point down the line) ipv6 tunneling, for example 6in4 or 6rd. The ipv6 mtu is strictly a subset of the device mtu; it can never be more than the device mtu. For cases where the system has native ipv6 connectivity, typically the device mtu == ipv6 mtu, so the device mtu should be set and the ipv6 mtu should not be changed. Anytime the device mtu is set, the ipv6 mtu is also changed to match it.

I think the ifupdown api is not actually in sync with what the ipv6 mtu really means. Part of this comes from the fact that ifupdown separates conf into "inet" for ipv4 and "inet6" for ipv6, but mtu isn't actually a networking layer setting; it's a link layer setting.

Within the inet/ipv4 and inet6/ipv6 constraints, one possibility is instead of setting the ipv6 mtu for all cases of a "inet6" section, ifupdown should restore the original behavior of setting the device mtu for either a "inet" or "inet6" section mtu param, and an additional "mtu6" param, valid only in the "inet6" section, should be added. That new param should be used to configure a lower ipv6 mtu for cases of ipv6 tunneling. Something like:

iface eth1 inet manual
  mtu 1500

iface eth1 inet6 manual
  mtu 1500 # this is equivalent to the "inet" section mtu
  mtu6 1480

However, that's obviously an api change that may require other changes to things that create ifupdown configuration.

Alternately, ifupdown could add logic to decide whether or not to change the device mtu or the ipv6 mtu, e.g. pseudocode:

if (interface_does_not_have_inet_section)
  # the "inet6" mtu really meant device mtu, not ipv6 mtu
  device_mtu = inet6_section_mtu
else if (inet_section_has_mtu)
  device_mtu = inet_section_mtu
  ipv6_mtu = inet6_section_mtu
else
  # interface has inet config,
  # but mtu specified only in ipv6,
  # must infer meaning
  if (current_device_mtu < inet6_section_mtu)
    # assume large device mtu desired
    device_mtu = inet6_section_mtu
  else
    # assume ipv6 tunneling
    ipv6_mtu = inet6_section_mtu

Dan Streetman (ddstreet) wrote :

> In the original bug where one is comparing ip output of MTU; is there something *not* working ?

in that original bug, the user is trying to set up ipv6 tunneling:

"This scenario is quiet common when using tunnels (e.g. Sixxs) to provide IPv6
connectivity. My IPv6 MTU needs to be 20 bytes smaller than may IPv4 MTU
because of the tunnel's overhead."

> Or did we just observe that the bond interface isn't the same MTU? Can we confirm that without
> the bond interface itself set to MTU 9202 that we don't see the correct jumbo frames over ipv6?

this isn't bonding-specific, the example from bug 1609861 just happens to use bonding. But an regular interface configured with a "inet6" section mtu of e.g. 9000 and no "inet" section (or an "inet" section but no mtu in it) will not correctly set the device mtu to 9000.

On Thu, Aug 4, 2016 at 5:21 PM, Dan Streetman <
<email address hidden>> wrote:

> > In the original bug where one is comparing ip output of MTU; is there
> something *not* working ?
>
> in that original bug, the user is trying to set up ipv6 tunneling:
>
> "This scenario is quiet common when using tunnels (e.g. Sixxs) to provide
> IPv6
> connectivity. My IPv6 MTU needs to be 20 bytes smaller than may IPv4 MTU
> because of the tunnel's overhead."
>
> > Or did we just observe that the bond interface isn't the same MTU? Can
> we confirm that without
> > the bond interface itself set to MTU 9202 that we don't see the correct
> jumbo frames over ipv6?
>
> this isn't bonding-specific, the example from bug 1609861 just happens
> to use bonding. But an regular interface configured with a "inet6"
> section mtu of e.g. 9000 and no "inet" section (or an "inet" section but
> no mtu in it) will not correctly set the device mtu to 9000.
>

I understand this scenario; however, what I don't understand is why
if we're setting mtu 9002 on the underlying devices, why the mtu on the
"virtual" device (bond0)
matters vs. the mtu setting of the ipv6 link, especially since this is ipv6
only.

Do we see packets actually get fragmented at the bond ?

>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1609367
>
> Title:
> ifupdown does not set ipv6-only large mtu
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/ubuntu/+source/ifupdown/+bug/
> 1609367/+subscriptions
>

Ryan Harper (raharper) wrote :

There also seems to be another bug for ipv6 only; the sysctl setting of mtu happens before the ip link set dev up; which means, at up time, the interface gets the default mtu which clobbers the ipv6 mtu.

/sbin/sysctl -q -e -w net.ipv6.conf.ens8.autoconf=0
/sbin/sysctl -q -e -w net.ipv6.conf.ens8.mtu=1480
/bin/ip link set dev ens8 up
/bin/ip -6 addr add fd0d:ffe0:5771::1:0:0/64 dev ens8

I'm testing this post-up hook which I think covers all of the cases.

# cat /etc/network/if-up.d/mtuipv6
#!/bin/sh

[ "$IFACE" != "lo" ] || exit 0

# Trigger only if MTU configured
[ -n "$IF_MTU" ] || exit 0

# Target ipv6 MTU settings only
[ "$ADDRFAM" = "inet6" ] || exit 0

# We need to check the underlying interface MTU and
# raise it if the IPV6 mtu is larger
CUR_DEV_MTU=$(/bin/cat /sys/class/net/$IFACE/mtu)
if [ $CUR_DEV_MTU -lt $IF_MTU ]; then
    /bin/ip link set $IFACE mtu $IF_MTU
fi
/sbin/sysctl -q -e -w net.ipv6.conf.$IFACE.mtu=$IF_MTU

Thoughts?

Dan Streetman (ddstreet) wrote :

> I understand this scenario; however, what I don't understand is why
> if we're setting mtu 9002 on the underlying devices, why the mtu on the
> "virtual" device (bond0)
> matters vs. the mtu setting of the ipv6 link, especially since this is ipv6
> only.

I think the slave interface mtu will be used.

> I'm testing this post-up hook which I think covers all of the cases.

yep that script looks good.

only thing i can think of is, what if an interface's inet6 section is specified (and setup) first, before the interface's inet section? e.g.

auto eth0
iface eth0 inet6 static
    address 2001:dbe::1/64
    mtu 1480

iface eth0 inet static
     address 10.0.0.1
     mtu 1500

would that cause the device mtu to be (re)set to 1500, after the ipv6 mtu had been set? That would reset the ipv6 mtu to 1500. Can the ordering of inet/inet6 sections always be inet before inet6?

Ryan Harper (raharper) wrote :

On Tue, Aug 9, 2016 at 12:56 PM, Dan Streetman <
<email address hidden>> wrote:

> > I understand this scenario; however, what I don't understand is why
> > if we're setting mtu 9002 on the underlying devices, why the mtu on the
> > "virtual" device (bond0)
> > matters vs. the mtu setting of the ipv6 link, especially since this is
> ipv6
> > only.
>
> I think the slave interface mtu will be used.
>
> > I'm testing this post-up hook which I think covers all of the cases.
>
> yep that script looks good.
>
> only thing i can think of is, what if an interface's inet6 section is
> specified (and setup) first, before the interface's inet section? e.g.
>
> auto eth0
> iface eth0 inet6 static
> address 2001:dbe::1/64
> mtu 1480
>
> iface eth0 inet static
> address 10.0.0.1
> mtu 1500
>
> would that cause the device mtu to be (re)set to 1500, after the ipv6
>

Testing the above scenario works fine. The interface has 1500 mtu, and
sysctl shows ipv6 link at 1480.

However,

if we have a higher mtu onthe ipv6 link, like 4800, then the inet section
mtu setting clobbers it and they both end up at 1500.

Now, that case I still believe is *invalid* configuration;

> mtu had been set? That would reset the ipv6 mtu to 1500. Can the
> ordering of inet/inet6 sections always be inet before inet6?
>

Curtin can provide some ordering here, but ifupdown always runs from
top-down.

>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1609367
>
> Title:
> ifupdown does not set ipv6-only large mtu
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/ubuntu/+source/ifupdown/+bug/
> 1609367/+subscriptions
>

Ryan Harper (raharper) on 2016-08-18
Changed in ifupdown (Ubuntu):
importance: Undecided → Medium
status: Incomplete → Confirmed
Dan Streetman (ddstreet) wrote :

> if we have a higher mtu onthe ipv6 link, like 4800, then the inet section
> mtu setting clobbers it and they both end up at 1500.
>
> Now, that case I still believe is *invalid* configuration;

I agree - if the ifupdown config has the inet mtu set lower than the inet6 mtu, that's a misconfiguration, since it's not possible for the device mtu to be lower than the ipv6 mtu.

Dan Streetman (ddstreet) wrote :

Ryan are you working on adding the script to ifupdown in ubuntu and/or debian?

Ryan Harper (raharper) wrote :

Hi Dan,

Yes; I have a curtin modification and test-cases to handle all of our
scenarios.

https://code.launchpad.net/~raharper/curtin/trunk.more-ipv6

In particular, you can see the final scripts (a pre-hook and post is needed)

http://bazaar.launchpad.net/~raharper/curtin/trunk.more-ipv6/view/head:/curtin/commands/apply_net.py#L29

And the various test-cases that we now pass on this branch, on precise,
trusty, xenial and yakkety with different versions of ifupdown.

http://bazaar.launchpad.net/~raharper/curtin/trunk.more-ipv6/view/head:/examples/tests/network_mtu.yaml

While we could see about getting the script included in yakkety (or y+1);
curtin will still need to
handle this unless we want to SRU ifdowndown in precise, trusty and
xenial. This has an enormous
impact; so it's not something we should take lightly.

On Fri, Aug 26, 2016 at 2:23 PM, Dan Streetman <
<email address hidden>> wrote:

> Ryan are you working on adding the script to ifupdown in ubuntu and/or
> debian?
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1609367
>
> Title:
> ifupdown does not set ipv6-only large mtu
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/ubuntu/+source/ifupdown/+bug/
> 1609367/+subscriptions
>

Dan Streetman (ddstreet) wrote :

> While we could see about getting the script included in yakkety (or y+1);
> curtin will still need to
> handle this unless we want to SRU ifdowndown in precise, trusty and
> xenial. This has an enormous
> impact; so it's not something we should take lightly.

As long as curtin handles it correctly, I don't see any need to SRU ifupdown; just adding the scripts in y and forward should be enough. Anyone manually configuring ifupdown in X or earlier can manually setup both inet and inet6 sections to get the mtus right.

Dan Streetman (ddstreet) wrote :

To follow up, I believe the decision is to add the scripts to ifupdown in y or z only (no SRU), and fix this for older releases in curtin via bug 1609861.

Dan Streetman (ddstreet) wrote :

> I believe the decision is to add the scripts to ifupdown in y or z only (no SRU)

or, we can just close this as wontfix and explain how to work around this by adding a manual inet section with the increased mtu value, since that's quite easy to do when manually configuring ifupdown.

Dan Streetman (ddstreet) on 2019-05-25
Changed in ifupdown (Ubuntu):
status: Confirmed → Won't Fix
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers