Network Manager copies IPv6 route cache entries to main routing table

Bug #1038541 reported by Ben Jencks on 2012-08-19
42
This bug affects 7 people
Affects Status Importance Assigned to Milestone
NetworkManager
Fix Released
Medium
network-manager (Ubuntu)
High
Mathieu Trudel-Lapierre
Precise
High
Mathieu Trudel-Lapierre

Bug Description

[Impact]
This tends to break network connectivity to specific hosts if such hosts were accessed before a change in network topology.

Specifically:
- User pings a system;
- User tries to connect to that system with SSH, it fails and the user realises they are not connected to a VPN required to reach that device.
- User connects to the VPN
- System is still unreachable because it is listed as reachable via the original network device rather than the new VPN-created network device.

Or; the user changes from wired to wifi and loses connectivity to some systems because they were initially accessed over wired and the IPv6 route remains via the old device.

[Test Case]
Requires a valid IPv6-enabled network.
1) Connect to an IPv6 network.
2) Ping Google via ipv6: 'ping6 ipv6.google.com'
3) Observe whether 2607:f8b0:400c:c03::6a gets added to the IPv6 routing table:
'ip -6 route'

Without the patch, any host that gets packets sent to it will be added to the routing table; with the patch, such hosts are not listed in the routing table with the 'ip -6 route' command.

[Regression Potential]
Risk is minimal; the messages processed are not meant to be received by NetworkManager. A possible regression scenario would be if valid netlink messages for new routes would be generated by the kernel with the RTM_F_CLONED message when they indicate new network routes required due to a topology change, rather than new temporary cache routes for accessing a particular host; and would cause the new route to not be added (causing limited network unreachability).

---

Any time an IPv6 route lookup happens, the kernel generates a new routing cache entry and notifies userspace using a netlink "new route" message with the RTM_F_CLONED flag set on the route.

Network Manager doesn't check for this flag, so it accepts it as a new real route and adds it to its internal route cache. Then, because the event triggers an interface update, it synchronizes its route cache with the kernel table, putting the cache entry in as a real host route.

I think NM might also overwrite the next hop of the route based on its internal idea of the default route, but I'm not sure.

This causes problems if you have interfaces not managed by NetworkManager, such as manually configured VPNs, since these host routes override the route entries that send traffic down those interfaces rather than to the default router.

To reproduce:
Ensure "Ignore automatically obtained routes" is unchecked in Edit Connection -> IPv6 -> Routes, because this blocks the last stage (copying the spurious route back to the kernel).
Connect to an IPv6 network.
run `ip -6 route` and observe no extra routes
run `ip -6 route get 2600::`
run `ip -6 route` again and observe the new static route to 2600:: via your default gateway

Expected behavior:
No changes to routing table due to a route lookup.

Ben Jencks (bjencks) wrote :

Patch attached that checks whether a new route notification is for a route cache entry, and if so, ignores it. Solves the problem for me, but I haven't checked for regressions or more complex use cases.

Diff is against 0.9.4.0-0ubuntu4.1.

The attachment "nm-ipv6-route-cache.patch" of this bug report has been identified as being a patch. The ubuntu-reviewers team has been subscribed to the bug report so that they can review the patch. In the event that this is in fact not a patch you can resolve this situation by removing the tag 'patch' from the bug report and editing the attachment so that it is not flagged as a patch. Additionally, if you are member of the ubuntu-reviewers team please also unsubscribe the team from this bug report.

[This is an automated message performed by a Launchpad user owned by Brian Murray. Please contact him regarding any issues with the action taken in this bug report.]

tags: added: patch
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in network-manager (Ubuntu):
status: New → Confirmed

This causes the babel routing daemon (package babeld) to trigger its route propagation and flood the network with the routing cache entries.

Changed in network-manager:
importance: Unknown → Medium
status: Unknown → New
Changed in network-manager:
status: New → Invalid
Changed in network-manager:
importance: Medium → Unknown
status: Invalid → Unknown
Changed in network-manager:
importance: Unknown → Medium
status: Unknown → In Progress

Nice patch there. I'm happy to include this in the package, though I'll change the debug message to be a bit more meaningful.

Still I'll test it before applying to Quantal :)

Setting up the tasks for Precise as well, since this *does* affect Precise and it should be applicable without issues.

Changed in network-manager (Ubuntu):
status: Confirmed → Triaged
importance: Undecided → High
assignee: nobody → Mathieu Trudel-Lapierre (mathieu-tl)
Changed in network-manager (Ubuntu Precise):
status: New → Triaged
importance: Undecided → High
assignee: nobody → Mathieu Trudel-Lapierre (mathieu-tl)
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package network-manager - 0.9.6.0-0ubuntu5

---------------
network-manager (0.9.6.0-0ubuntu5) quantal; urgency=low

  * debian/patches/nm-ipv6-route-cache.patch: don't re-add routes we receive
    from kernel notifications to the routing table if they have the
    RTM_F_CLONED flag; since that means they're host routes used temporarily
    by the kernel to reach a specific destination; keeping them would cause at
    least issues with VPNs, and possibly confusion with routing daemons.
    Thanks to Ben Jenks for the patch. (LP: #1038541)
 -- Mathieu Trudel-Lapierre <email address hidden> Wed, 29 Aug 2012 16:38:51 -0400

Changed in network-manager (Ubuntu):
status: Triaged → Fix Released
Changed in network-manager:
status: In Progress → Fix Released
Changed in network-manager (Ubuntu Precise):
status: Triaged → In Progress
description: updated

Hello Ben, or anyone else affected,

Accepted network-manager into precise-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/network-manager/0.9.4.0-0ubuntu4.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please change the bug tag from verification-needed to verification-done. If it does not, change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in network-manager (Ubuntu Precise):
status: In Progress → Fix Committed
tags: added: verification-needed
Ben Jencks (bjencks) wrote :

Confirmed, this fixes the issue. By the way, is it too late to get my last name fixed in the changelog (Jencks vs. Jenks)?

tags: added: verification-done
removed: verification-needed

Oops, sorry :/

Yeah, it's kind of late, because then the package would need to go through validation again and wait at least 7 days in proposed.

The verification of this Stable Release Update has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regresssions.

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package network-manager - 0.9.4.0-0ubuntu4.2

---------------
network-manager (0.9.4.0-0ubuntu4.2) precise-proposed; urgency=low

  * debian/patches/git_use_wpa_wext_methods_50435e1.patch: use the same kind
    of logic as wpasupplicant instead of just looking at whether a driver
    reports signal over nl80211 to decide whether to use that or fallback
    to wext. (LP: #1035590)
  * debian/patches/ipw2x00-no-nl80211.patch: replaced by the patch above.
  * debian/patches/git_ignore_ipconfig_dups_cca4052.patch: silently ignore
    duplicates in NMIP[46]Config objects; which tends to happen fairly
    regularly with RDNSS. (LP: #996032)
  * debian/patches/nm-ipv6-route-cache.patch: don't re-add routes we receive
    from kernel notifications to the routing table if they have the
    RTM_F_CLONED flag; since that means they're host routes used temporarily
    by the kernel to reach a specific destination; keeping them would cause at
    least issues with VPNs, and possibly confusion with routing daemons.
    Thanks to Ben Jenks for the patch. (LP: #1038541)
  * debian/ifblacklist_migrate.sh: make sure that "iface X inet6" entries added
    by d-i also get commented out if set up for dhcp or auto -- this follows
    what is already done for IPv4, where such devices should be handled by
    NetworkManager. This fixes issues where people installing using d-i and
    where IPv6 autoconfiguration is available would get their interfaces
    ignored by NetworkManager when installing desktop. (LP: #995165)
 -- Mathieu Trudel-Lapierre <email address hidden> Wed, 10 Oct 2012 17:31:55 -0400

Changed in network-manager (Ubuntu Precise):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.