Backport netplan to xenial

Bug #1627641 reported by Martin Pitt on 2016-09-26
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
network-manager (Ubuntu)
Wishlist
Unassigned
Xenial
Wishlist
Martin Pitt
nplan (Ubuntu)
Wishlist
Unassigned
Xenial
Wishlist
Martin Pitt
systemd (Ubuntu)
Low
Unassigned
Xenial
Low
Martin Pitt

Bug Description

For snappy (at first at least) we need to provide netplan in xenial, as for the first snappy GA release we must not use any PPAs any more.

netplan's NetworkManager backend depends on two patches to read configuration and connections from /run/NetworkManager/. These will need to be backported for full netplan support; but they are not required for snappy as this will use a snapped NM. However, this will need a temporary hack (https://code.launchpad.net/%7Emorphis/netplan/+git/netplan/+merge/306607) until snaps can actually properly support OS components like NetworkManager.

PATCHES:
https://git.launchpad.net/~network-manager/network-manager/+git/ubuntu/commit/?h=xenial&id=6dcdb85
https://anonscm.debian.org/cgit/pkg-systemd/systemd.git/commit/?h=ubuntu-xenial&id=4e9c52b0bb

REGRESSION POTENTIAL:
netplan: The risk for existing installations is practically zero as nplan does not exist in xenial yet and thus will not be pulled in during upgrades.

NetworkManager: Nothing in xenial expects/uses /run/NetworkManager/ and as it's an ephemeral tmpfs there is no risk of existing files there. If the patches are broken it could in theory happen that NetworkManager also does not properly read files from /etc/NetworkManager/ any more, so the -proposed package must verify that existing connections still work.

systemd: This does change behavior of networkd quite a bit: RA is now being handled in userpsace instead of the kernel, there are some new virtual device types, LLDP support, etc., and there are no (known) backwards incompatibilities. The 229 version was known buggy with DHCPv6 (we disabled these two test cases), and judging by the feedback in Debian 231 is now reasonably stable. networkd is not being used by default or advertised in Ubuntu 16.04 (so far), so this will not affect the vast majority of installations. But while we have quite good test coverage, it cannot be ruled out that we break some custom setup that uses networkd.

TEST PLAN:
1. Run "NetworkManager --print-config" and save the output.
2. Install the proposed NetworkManager and confirm that existing connections (from /etc/NetworkManager/system-connections) still work.
3. Run "NetworkManager --print-config" again and verify that the output is the same as in step 1.
4. netplan has a very comprehensive integration test suite run as autopkgtest, which covers NetworkManager (including the /run patches) and network. Confirm that it succeeds.

Martin Pitt (pitti) on 2016-09-26
summary: - Provide nplan to xenial
+ Backport netplan to xenial
Changed in network-manager (Ubuntu):
status: New → Fix Released
Changed in nplan (Ubuntu):
status: New → Fix Released
Changed in network-manager (Ubuntu Xenial):
status: New → Triaged
assignee: nobody → Martin Pitt (pitti)
importance: Undecided → Wishlist
Changed in nplan (Ubuntu Xenial):
importance: Undecided → Wishlist
status: New → Triaged
assignee: nobody → Martin Pitt (pitti)
Martin Pitt (pitti) on 2016-09-26
description: updated
Changed in network-manager (Ubuntu Xenial):
status: Triaged → In Progress
Martin Pitt (pitti) wrote :

With the backported nplan one test case fails:

======================================================================
FAIL: test_manual_addresses (__main__.TestNetworkd)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/autopkgtest.RBASGN/build.kj6/nplan-0.12~16.04/tests/integration.py", line 458, in test_manual_addresses
    'inet6 1234']) # old static IPv6
  File "/tmp/autopkgtest.RBASGN/build.kj6/nplan-0.12~16.04/tests/integration.py", line 303, in assert_iface_up
    self.assertNotRegex(out, r, out)
AssertionError: Regex matched: 'inet 192.168.5' matches 'inet 192.168.5'

I. e. after "netplan apply" the old DHCP address from the interface does not disappear when switching to a pure "static addresses" config. This works in yakkety, and I need to investigate this first (might need a backported fix to networkd).

Martin Pitt (pitti) wrote :

We need to backport a networkd fix: https://github.com/systemd/systemd/commit/6fc2549711 . This will clean up old addresses from interfaces when restarting networkd, so that changed netplan configuration will actually reflect reality.

Changed in systemd (Ubuntu):
status: New → Fix Released
Changed in systemd (Ubuntu Xenial):
assignee: nobody → Martin Pitt (pitti)
importance: Undecided → High
status: New → In Progress
Martin Pitt (pitti) on 2016-09-26
description: updated
description: updated
Martin Pitt (pitti) on 2016-09-26
description: updated
Martin Pitt (pitti) on 2016-09-26
Changed in systemd (Ubuntu Xenial):
importance: High → Wishlist
importance: Wishlist → Low

Hello Martin, or anyone else affected,

Accepted systemd into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/229-4ubuntu9 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in systemd (Ubuntu Xenial):
status: In Progress → Fix Committed
tags: added: verification-needed
Andy Whitcroft (apw) wrote :

Hello Martin, or anyone else affected,

Accepted network-manager into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/network-manager/1.2.2-0ubuntu0.16.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in network-manager (Ubuntu Xenial):
status: In Progress → Fix Committed
Martin Pitt (pitti) on 2016-09-27
Changed in nplan (Ubuntu Xenial):
status: Triaged → In Progress
Changed in network-manager (Ubuntu):
importance: Undecided → Wishlist
Changed in nplan (Ubuntu):
importance: Undecided → Wishlist
Changed in systemd (Ubuntu):
importance: Undecided → Low
Andy Whitcroft (apw) wrote :

Hello Martin, or anyone else affected,

Accepted nplan into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/nplan/0.12~16.04 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in nplan (Ubuntu Xenial):
status: In Progress → Fix Committed
Andy Whitcroft (apw) wrote :

Hello Martin, or anyone else affected,

Accepted network-manager into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/network-manager/1.2.2-0ubuntu0.16.04.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Martin Pitt (pitti) wrote :

This isn't reproducible in yakkety, but it does reproduce locally in a QEMU xenial instance (although a lot harder than on the infra, which is why I didn't see it at first). It seems systemd 229's systemd-networkd-wait-online has a bug that it does not actually wait for the interfaces to be fully configured. When it exits, the status is

● 412: eth42
       Link File: /lib/systemd/network/99-default.link
    Network File: /run/systemd/network/10-netplan-eth42.network
            Type: ether
           State: degraded (configured)
          Driver: veth
      HW Address: 0a:9b:04:93:f9:c0
             MTU: 1500
         Address: fe80::89b:4ff:fe93:f9c0
             DNS: 2600:0000:0000:0000:0000:0000:0000:0001

i. e. the interface is still being set up. I would not like to work around that in the tests, because if we are actually going to use networkd on xenial the integration with network-online.target must work.

Changed in systemd (Ubuntu Xenial):
status: Fix Committed → In Progress
Martin Pitt (pitti) wrote :

I understand this better now. networkd actually went to handling DHCPv6/RA in userspace (https://github.com/systemd/systemd/commit/3b015d40c19) so that it actually *can* wait for IPv6 RA addresses -- if the kernel handles them, it does not know what to expect.

In yakkety we let networkd do that, but in xenial we revert it (https://anonscm.debian.org/cgit/pkg-systemd/systemd.git/tree/debian/patches/Revert-Revert-networkd-ndisc-revert-to-letting-the-k.patch?h=ubuntu-xenial) as networkd v229 has some regressions wrt. IPv6 handling compared to the kernel. These are fixed in 231, so we could drop the Debian reversion patch and instead backport these fixes.

nplan and wait-online works fine with the patch dropped, for the record.

Martin Pitt (pitti) wrote :

I just realized that the DHCPv6 failure that netplan's tests detect were already known in xenial's networkd tests; these tests got marked as "expected failure".

I started to backport individual networkd fixes, but this quickly became a frankensoftware which has never been tested in that form, and it would take prohibitively long to finish. Since 231, networkd development has slowed down considerably and it's by and large stable (judging by Debian bug reports a lot of people actually use it, and we did not get complaints since 231 any more).

So in summary I think it is better to backport networkd 231 wholesale. In the running system it is completely independent of systemd and other tools (standalone binary), it has good test coverage, has been tried and tested in yakkety (unlike xenial when we did not yet use/support networkd), so IMHO the risk of this is lower than spending days on reengineering fixes on top of 229.

The main noise of the backport is that this also requires backporting some common utility code.

Martin Pitt (pitti) on 2016-09-29
description: updated
Chris Halse Rogers (raof) wrote :

Hello Martin, or anyone else affected,

Accepted systemd into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/229-4ubuntu11 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in systemd (Ubuntu Xenial):
status: In Progress → Fix Committed
Martin Pitt (pitti) wrote :

systemd's and netplan's tests now work:
  http://people.canonical.com/~ubuntu-archive/proposed-migration/xenial/update_excuses.html#systemd
  http://people.canonical.com/~ubuntu-archive/proposed-migration/xenial/update_excuses.html#nplan

nplan's tests still show a race condition on ppc64el, but this isn't a regression.

network-manager's tests also all work:
  http://people.canonical.com/~ubuntu-archive/proposed-migration/xenial/update_excuses.html#network-manager

I ran the above test cases on a 16.04.1 desktop (--print-config is identical, verifying existing connections still work), and NM still generally works.

tags: added: verification-done
removed: verification-needed
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package network-manager - 1.2.2-0ubuntu0.16.04.3

---------------
network-manager (1.2.2-0ubuntu0.16.04.3) xenial; urgency=medium

  * debian/tests/wpa-dhclient: Don't assume that the IPv6 prefix length from
    the DHCP server is /64. (LP: #1609898)

network-manager (1.2.2-0ubuntu0.16.04.2) xenial; urgency=medium

  [ Martin Pitt ]
  * Read config and system connections from /run/NetworkManager/ to support
    netplan (LP: #1627641)
  * debian/gbp.conf: Set debian-branch to xenial

  [ Mathieu Trudel-Lapierre ]
  * Add dns-manager-don-t-merge-split-DNS-search-domains.patch: do not add
    split DNS search domains to resolv.conf; doing so would risk leaking names
    to non-VPN DNS nameservers when attempting to resolve non- FQDN names.
    (LP: #1592721)

 -- Martin Pitt <email address hidden> Tue, 27 Sep 2016 16:29:22 +0200

Changed in network-manager (Ubuntu Xenial):
status: Fix Committed → Fix Released

The verification of the Stable Release Update for network-manager has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package systemd - 229-4ubuntu11

---------------
systemd (229-4ubuntu11) xenial; urgency=medium

  * 73-usb-net-by-mac.rules: Split kernel command line import line.
    Reportedly this makes the rule actually work on some platforms. Thanks
    Alp Toker! (LP: #1593379)
  * fsckd: Do not exit on idle timeout if there are still clients connected
    (Closes: #788050, LP: #1547844)
  * libnss-*.prerm: Remove possible [key=value] options from NSS modules as
    well. (LP: #1625584)
  * Backport networkd 231. Compared to 229 this has a lot of fixes, some of
    which we need for good netplan support. Backporting them individually
    would be a lot more work and a lot less robust, and we did not use/support
    networkd in 16.04 so far. Drop the other network related patches as they
    are included in this backport now. (LP: #1627641)
  * debian/tests/networkd: Re-enable the the DHCPv6 tests. The DHCPv6
    behaviour is fixed with the above backport now.
  * pid1: process zero-length notification messages again. Just remove the
    assertion, the "n" value was not used anyway. This fixes a local DoS due
    to unprocessed/unclosed fds which got introduced by the previous fix.
    (LP: #1628687)
  * pid1: Robustify manager_dispatch_notify_fd(). If
    manager_dispatch_notify_fd() fails and returns an error then the handling
    of service notifications will be disabled entirely leading to a
    compromised system. (side issue of LP: #1628687)

 -- Martin Pitt <email address hidden> Tue, 04 Oct 2016 21:43:04 +0200

Changed in systemd (Ubuntu Xenial):
status: Fix Committed → Fix Released
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package nplan - 0.12~16.04

---------------
nplan (0.12~16.04) xenial; urgency=medium

  [ Martin Pitt ]
  * Backport to xenial. (LP: #1627641)
  * Adjust Breaks: network-manager to version in xenial that provides the
    "read config from /run" functionality.
  * src/netplan: Add hack to support current NetworkManager snap, which
    currently cannot provide "nmcli" and "NetworkManager.service". Note that
    this is meant to be temporary until snapd gets fixed, and will NOT be
    applied to yakkety or upstream. Patch by Simon Fels.

 -- Martin Pitt <email address hidden> Mon, 26 Sep 2016 11:06:57 +0200

Changed in nplan (Ubuntu Xenial):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers