IPv6 shouldn’t be disabled by default in libvirt

[Jeremy Visser]

libvirt by default disables IPv6 as a result of this patch: https://bugzilla.redhat.com/attachment.cgi?id=355689&action=diff

This is incorrect behaviour. IPv6 should never be disabled by default, and the need for IPv6–enabled software grows greater every day.

Currently, libvirt both sets net.ipv6.conf.virbrX.disable_ipv6=1, and net.ipv6.conf.virbrX.accept_ra=0. The reason why IPv6 was disabled by Red Hat was to stop the virbrX interface from accidentally receiving Router Advertisements from virtual machines running on the host. The solution accept_ra=0 could have been sufficient to fix this, but due to reasons unknown (likely a fear and non-understanding of IPv6), IPv6 was instead disabled altogether, which breaks IPv6 functionality.

This can be fixed by only setting accept_ra=0, while not touching disable_ipv6. (If a sysadmin still wants IPv6 disabled, they can do so on the kernel command line, or set net.ipv6.conf.all.disable_ipv6=1.)

It is important that this is fixed, because Ubuntu 10.04 LTS is supported until 2015 — three years after the predicted exhaustion of the RIR IPv4 pool (and four years after the exhaustion of the IANA pool).

[Peter (peter-redhat-bugs)]

Description of problem:
A bridge interface generated by libvirt/qemu is IPv6 enabled by default. Ok, can be, but this should be made switchable by network.xml (e.g. net.ipv6.conf.virbr1.disable_ipv6=1)

The bad thing is that, it accepts router advertisements. I have a VM, which acts as an IPv6 router for an isolated network (testing purposes) and has a running radvd (only on this isolated network). Suddenly, virbr1 also gets an RA, configures a related IPv6 address and add a default route to this VM.

This destroys the IPv6 default routing for the host itself, if done e.g. later via aiccu.

Version-Release number of selected component (if applicable):
# rpm -qa |grep virt
python-virtinst-0.400.3-1.fc10.noarch
virt-viewer-0.0.3-3.fc10.i386
virt-mem-0.2.9-6.fc10.i386
libvirt-0.5.1-2.fc10.i386
virt-manager-0.6.0-5.fc10.i386
libvirt-python-0.5.1-2.fc10.i386
virt-top-1.0.3-2.fc10.i386

How reproducible:
always

Steps to Reproduce:
1. setup a router VM with 2 nics, one on a NAT bridge shared with the host (virbr0) and one isolated bridge (virbr1)

2. Configure radvd on this router VM for isolated network only

3. Wait...

Actual results:

# ip -6 addr show dev virbr1
16: virbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500
    inet6 fec0::1:4c3:8bff:fe**:****/64 scope site dynamic
       valid_lft 2591985sec preferred_lft 604785sec
    inet6 fe80::4c3:8bff:fe**:****/64 scope link
       valid_lft forever preferred_lft forever

# ip -6 route show dev virbr1
fe80::/64 proto kernel metric 256 mtu 1500 advmss 1440 hoplimit 4294967295
fec0:0:0:1::/64 proto kernel metric 256 expires 0sec mtu 1500 advmss 1440 hoplimit 4294967295
default via fe80::5652:ff:fe**:**** proto kernel metric 1024 expires 0sec mtu 1500 advmss 1440 hoplimit 64

and starting then aiccu on the host results in:

# ip -6 route show |grep ^default
default via fe80::5652:ff:fe**:**** dev virbr1 proto kernel metric 1024 expires 0sec mtu 1500 advmss 1440 hoplimit 64
default via 2001:6f8:900:****::1 dev sixxs metric 1024 mtu 1280 advmss 1220 hoplimit 4294967295

So now 2 default IPv6 routes are existing, but the first one didn't work.

Expected results:
Neither autoconfigured IPv6 address no autoconfigured default route on a bridge interface.

Additional info:

Imho, a bridge should be *IPv4- and IPv6-less* by default and get only such addresses by explicit configuration, because in Linux currently the isolation does not really exist.

Afair I have already detected this on a Xen installation more than one year before, same issue...don't remember whether I had already reported this.

[Peter (peter-redhat-bugs)]

Workaround:

disable IPv6 and accepting of RAs and autoconf on this interface and remove routes through it

# sysctl -w net.ipv6.conf.virbr1.disable_ipv6=1
# sysctl -w net.ipv6.conf.virbr1.accept_ra=0
# sysctl -w net.ipv6.conf.virbr1.autoconf=0

and remove auto-configured IPv6 address from this interface.

[Daniel (daniel-redhat-bugs)]

Aieeee, this is a nasty bug.

Shouldn't it be sufficient to just blow away the link-local address on the bridge device. This should stop it doing any auto-config, even if the sysctls you mention have it turned on.

[Peter (peter-redhat-bugs)]

Hmm, more or less bad idea to remove the link-local IPv6 address because its afaik lost then forever (until interface restart). I would prefer to add a block
between

brctl addbr virbrX

and

ip link set virbrX up

# sysctl -w net.ipv6.conf.virbrX.disable_ipv6=1
# sysctl -w net.ipv6.conf.virbrX.accept_ra=0
# sysctl -w net.ipv6.conf.virbrX.autoconf=0

should be added.

[Daniel (daniel-redhat-bugs)]

Ok, yes that makes sense & avoids the race condition.

[Peter (peter-redhat-bugs)]

Are you sure the subject was proper changed to "libvirt bridge should not have IPv6 disabled"??

The libvirt bridge is currently IPv6 enabled by default, problem is that it is accepting router advertisements (which should be avoided in any case).

It can be that sometime in the future one request similiar like in IPv4, means configure the libvirt bridge as a routing interface to the outside, in this case, a permanent IPv6 address and a running radvd providing prefixes to the VM clients on this interface can be the case.

But for the meantime, at least accept of router advertisements should be disabled, because this can destroy IPv6 routing by a malicious client or even redirect IPv6 upstream to a malicious VM.

[Mark (mark-redhat-bugs)]

Yeah, sorry - just a typo

[Daniel (daniel-redhat-bugs)]

Created attachment 355689
Disable ipv6

In my testing this prevented the host being impacted by guest radvd advertisements.

[Mark (mark-redhat-bugs)]

Okay, this patch is now applied in rawhide

See also bug #516497 - the patch didn't handle ipv6-less kernels very well

[Jeremy (jeremy-redhat-bugs)]

This "fix" is incorrect. It disables IPv6, which is completely incorrect behaviour in any case.

Comment #5 alludes to this -- you have managed to take the original meaning of the bug, which is that RAs get received on the virbr interface, and twisted it into somehow disabling IPv6 on the interface altogether.

I'm working on a patch that removes the "disable_ipv6=1" portion of the function.

[Jeremy (jeremy-redhat-bugs)]

Created attachment 396453
Remove IPv6 disabling functionality, while keeping accept_ra=0

Attached is a patch which leaves IPv6 enabled, while still setting accept_ra=0, which means that from the original perspective of this bug, the problem is still fixed, without the side-effect of disabling IPv6 altogether.

[Daniel (daniel-redhat-bugs)]

Disabling IPv6 completely is 100% intentional. In the future we will be re-adding explicit support for IPv6 in the network XML configuration format to allow various well defined configurations. We don't want to rely on any default IPv6 behaviour that may or may not be setup

[Jeremy (jeremy-redhat-bugs)]

Are you basically saying you won't apply the patch? I would say the urgency to apply this patch is *critical*. Call me a pessimist, but I somehow doubt you'll be motivated to ship explicit IPv6 support in the XML config any time soon, given your current attitude, so this patch means the behaviour can be fixed, and can be fixed now.

RHEL 6 will obviously be shipping with this code. If it's not, that doesn't devalue my point, because Ubuntu 10.04 LTS is on track to ship with this "functionality" too, and this OS will be supported on the server until 2015.

The IPv4 Address Report [0], generated by a methodology that makes it more accurate over time, predicts an IANA IPv4 pool exhaustion by Sep 2011, and RIR pool exhaustion by Oct 2012.

Do you think that there will be RHEL 6 and Ubuntu 10.04 systems in production in the year 2013 and 2014, during which there will be a crunch to have IPv6 addresses added to all machines, including libvirt VMs? Of course!

Not meaning to be rude, but do you really want to have to explain to your customers, "well, the OS does support IPv6, but you'll actually have to manually patch the software because we couldn't see past the end of our own nose back in 2010"?

If you cannot see the problem here, I suggest you take your ability to make long term decisions into serious question.

[0] http://www.potaroo.net/tools/ipv4/index.html

[Peter (peter-redhat-bugs)]

I 100% agree to https://bugzilla.redhat.com/show_bug.cgi?id=501934#c12, just note that Red Hat has already the problem with RHEL5, where they selected the used kernel too early (2.6.18), where stateful IPv6 firewalling is still missing and probably would never be integrated (because the change to xtables were done in upstream some minor versions later, too much work for backporting).

Best way would be to involve now Red Hat Enterprise Linux Product management, if they want proper IPv6 support also in virtualized environment for RHEL6, then at at least one should work on this particular enhancement, which includes also solutions for https://bugzilla.redhat.com/show_bug.cgi?id=514749 and https://bugzilla.redhat.com/show_bug.cgi?id=567124.

[Mathias Gug (mathiaz)]

Could you please add a link to the upstream bug in order to be able to review the complete conversation leading to the patch you've linked to?

Even if we revert the upstream patch (by applying your patch) would ipv6 be working correctly under libvirt?

[Jeremy Visser (jeremy-visser)]

I’ve added the upstream bug.

And yes, IPv6 would work correctly when you apply this patch. The original intent of the upstream bug report was that the virbrX interface was receiving RA’s from client VMs. Upstream patched it by way of both settings accept_ra=0 and disable_ipv6=1, though only the former is only necessary to fix the problem, which is exactly the behaviour that my patch fixes.

[Philipp Kern (pkern)]

On Fri, Mar 05, 2010 at 01:04:43AM -0000, Jeremy Visser wrote:
> And yes, IPv6 would work correctly when you apply this patch. The
> original intent of the upstream bug report was that the virbrX interface
> was receiving RA’s from client VMs. Upstream patched it by way of both
> settings accept_ra=0 and disable_ipv6=1, though only the former is only
> necessary to fix the problem, which is exactly the behaviour that my
> patch fixes.

I somehow recall that the ipv6 forwarding sysctl sets accept_ra=0.
Furthermore I don't like software messing with my sysctls at all, but
maybe that's just me. (I.e. one should be able to set sysctls by
yourself too and being able to turn that sysctl setting off entirely.)

Kind regards,
Philipp Kern

[Jeremy Visser (jeremy-visser)]

I do agree with that. In fact, one of my locally patched copies of libvirt does just that — doesn’t touch the sysctls at all. If I really don’t want to accept RA’s, I can set net.ipv6.conf.default.accept_ra=0 (or specifically for the interface).

However, I do also recognise that the bug report and suggestion was written by Peter Bieringer, who wrote the IPv6 tutorial for Linux, and knows a tad more about IPv6 than I do, so out of respect I hesitated to subvert his bug’s original intent.

[Launchpad Janitor (janitor)]

This bug was fixed in the package libvirt - 0.7.5-5ubuntu13

---------------
libvirt (0.7.5-5ubuntu13) lucid; urgency=low

  * debian/patches/9017-dont-disable-ipv6.patch: don't disable IPv6
    on the bridge interface. The original intent of disabling IPv6 was
    to prevent RAs received from client VMs from accidentally adding
    addresses to the bridge interface. However, only accept_ra=0 is
    necessary to fix this, without the side-effect of disabling IPv6.
    (LP: #528934)
 -- Jeremy Visser <email address hidden> Thu, 04 Mar 2010 17:33:11 +1100

[Tore (tore-redhat-bugs)]

I wholeheartedly agree with Jeremy's comments in comment #12. The implemented «fix» feels about equivalent to amputating an arm to heal a fracture...

Tore

[Konstantin (konstantin-redhat-bugs)]

After upgrade from F11 to F12 this broke IPv6 autoconfiguring for my guests completely. I have quagga on host doing RA to subnets, including libvirt's briges (this is use case described in comment #5).
So how do I workaround your "fix" to fix this?

[Jeremy (jeremy-redhat-bugs)]

(In reply to comment #15)
> So how do I workaround your "fix" to fix this?

Two ways: (1) manually set /proc/sys/net/ipv6/conf/$VIRBR_INTERFACE/disable_ipv6 to 0 each time the virbr interface comes up, or (2) patch the software with the patch I have added to this bug report.

[Jeremy (jeremy-redhat-bugs)]

Daniel, do you have any further thoughts on this bug?

[Matt (matt-redhat-bugs)]

In my case, my subnet's router advertises IPv6 subnets using radvd. I have a second non-NAT bridge on my hosts, and connect my VMs to that non-NAT bridge. They're receiving route advertisements from the subnet router, and works just fine. However, the non-NAT bridge somehow isn't getting an IPv6 autoconf address, even though disable_ipv6 = 0 and accept_ra (and related) = 1.

While I can see the concern about guests running radvd could cause problems, in the case where this isn't happening, I'd like IPv6 to be usable on both the host and guests.

[Bernie (bernie-redhat-bugs)]

(In reply to comment #18)
> While I can see the concern about guests running radvd could cause problems, in
> the case where this isn't happening, I'd like IPv6 to be usable on both the
> host and guests.

Can I say "me too"? :-)

[Jeremy (jeremy-redhat-bugs)]

Daniel, if you read these bug mails, can you please re-open the bug status? Keeping this bug with status 'closed' isn't going to help you pretend the problem doesn't exist.

(In reply to comment #18)
> However, the non-NAT bridge somehow isn't getting an IPv6 autoconf
> address, even though disable_ipv6 = 0 and accept_ra (and related) = 1.

Offtopic for this bug, but that problem is caused because net.ipv6.conf.all.forwarding = 1. That behaviour is badly documented, but basically if the machine is configured to forward packets, it won't accept any autoconf addresses.

[Laine (laine-redhat-bugs)]

Since someone just referenced this bug from a different BZ as the possible cause of IPv6 not working on libvirt virtual networks, I thought I should comment here that IPv6 support for libvirt virtual networks was added in libvirt-0.8.7, released on Jan 4, 2011. Any version of libvirt after that will properly set the abovementioned sysctl settings according to whether or not an IPv6 address is configured for the network.