[FFe] Update to netplan.io 0.99

[Łukasz Zemczak]

[Freeze Exception Rationale]

We would like to request a Feature Freeze exception for the new netplan 0.99 as this release will be carrying important roadmap features targeted for 20.04. Everything got drastically delayed due to not having a full-time netplan maintainer for most part of the cycle, but the features we want to deliver are still considered important. We are well aware of how late it is in the cycle, and in normal circumstances we would not request such an exception, but releasing the new netplan version will also unblock some of the other teams. Notably the generation of libnetplan will unblock other feature requests in packages like NetworkManager for uc20.

=== SRU ===
[Impact]
This release contains both bug-fixes and new features and we would like to
make sure all of our supported customers have access to these improvements.

The most notable changes (besides usual bug-fixes) incoming in 0.99:
 * Adding `phase2` keyword to `auth` section
 * Allowing `critical` to be used without `dhcp4`/`dhcp6` enabled
 * Support for GSM modems in the NetworkManager backend (with the `modems` keyword)
 * Adding `emit-lldp` option for networkd backend
 * Split out the netplan parser into a separate libnetplan library
 * Adding `ipv6-address-generation` field for NM backend
 * Adding WiFi flags for `bssid`/`band`/`channel`
 * Adding support for SR-IOV network devices
 * Fix not connecting to WiFi after 'netplan apply' (LP: #1874377)

See the changelog entry below for a full list of changes and bugs.

[Test Case]
The following development and SRU process was followed:
https://wiki.ubuntu.com/NetplanUpdates

Netplan contains an extensive integration test suite that is ran using
the SRU package for each releases. This test suite's results are available here:
http://autopkgtest.ubuntu.com/packages/n/netplan.io

A successful run is required before the proposed netplan package
can be let into -updates.

The netplan team will be in charge of attaching the artifacts and console
output of the appropriate run to the bug. Netplan team members will not
mark ‘verification-done’ until this has happened.

[Regression Potential]
In order to mitigate the regression potential, the results of the
aforementioned integration tests are attached to this bug.

Focal:
https://slyon.de/files/netplan/SRU-0.99/focal_amd64_log
https://slyon.de/files/netplan/SRU-0.99/focal_arm64_log
https://slyon.de/files/netplan/SRU-0.99/focal_armhf_log
https://slyon.de/files/netplan/SRU-0.99/focal_ppc64el_log
https://slyon.de/files/netplan/SRU-0.99/focal_s390x_log

Eoan:
https://slyon.de/files/netplan/SRU-0.99/eoan_amd64_log
https://slyon.de/files/netplan/SRU-0.99/eoan_arm64_log
https://slyon.de/files/netplan/SRU-0.99/eoan_armhf_log
https://slyon.de/files/netplan/SRU-0.99/eoan_i386_log
https://slyon.de/files/netplan/SRU-0.99/eoan_ppc64el_log
https://slyon.de/files/netplan/SRU-0.99/eoan_s390x_log

Bionic:
https://slyon.de/files/netplan/SRU-0.99/bionic_amd64_log
https://slyon.de/files/netplan/SRU-0.99/bionic_arm64_log
https://slyon.de/files/netplan/SRU-0.99/bionic_armhf_log
https://slyon.de/files/netplan/SRU-0.99/bionic_i386_log
https://slyon.de/files/netplan/SRU-0.99/bionic_ppc64el_log
https://slyon.de/files/netplan/SRU-0.99/bionic_s390x_log

[Discussion]
A bug fix for #1874377 was already part of 0.99-0ubuntu2. This SRU contains the proper fix, which was applied upstream (a9f138d) and handles additional edge cases. All the changes are completely backward-compatible and users should not notice any behavior changes if not using any of the new features.

[Changelog]
Distro-Patch:
a9f138d Fix LP#1874377: Not connect to WiFi after 'netplan apply' (#133)
v0.99:
1ccf7e0 SR-IOV support for explicitly defining the VF count (#130)
b7f1d9b SR-IOV support (#129)
a5438a2 Implement WakeOnWLan for NM and networkd backends (#128)
9a1633a Implement WiFi flags for bssid/band/channel (#125)
0586a8d Add ipv6-address-generation field for NM backend (#126)
39057cc Fix missing Require= in service unit (#123)
ed63344 Rename gsms to modems and enable CDMA (#124)
e9e7681 Makefile: fix symbolic link of libnetplan.so (#122)
4ca70d1 Build libnetplan and initial parser for NetworkManager backend settings (#112)
0c06743 Merge pull request #121 from CanonicalLtd/slyon/focal-integration-tests
039e2c2 Merge branch 'master' into slyon/focal-integration-tests
069e3d8 tests:integration:base: fix stylistic issue
4a46cfd tests:integration:base: Adopt integration tests to NetworkManager v1.22
2427ab2 Fix quotation of WPA PSK 64 hex-digit keys for networkd (#120)
df5fbf5 tests:integration:base: give NetworkManager some more time to settle
261c379 Add option `emit-lldp` for networkd backend (#116)
76aa65e Initial support for GSM modems for NetworkManager backend – 2 (#119)
91c59c7 features: add feature flags for new supplicant and phase2 changes
500121f tests: fix calls for yaml.load(): add an explicit Loader=
72a09f7 Allow `critical` to be used without `dhcp4`/`dhcp6` enabled (#107)
053f2cb Handled wired wpa_supplicant driver automatically (#109)
50ac74f Add `phase2` keyword to `auth` section (#108)
77fcd4d networkd: set MTUBytes= in .network files as well (#111)
c40f70e Add example for a directly-connected gateway on IPv6. (#110)
a5397e6 Honor LDFLAGS when building netplan-dbus (#105)
43d80a6 Makefile: fix escaping in _features.h (#103)
521a695 Refresh devices after restarting backends, some new devices might appear

[Iain Lane (laney)]

"Even though regressions are impossible" - you mean possible I hope? :-)

This is a bit worryingly late, but I get the extenuating circumstances. Please go for it, but make sure to keep a close eye on incoming bugs. I would like it if someone could find the time to do manual testing *before* we get into final freeze, too. To avoid any last minute surprises in release week.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Launchpad Janitor (janitor)]

This bug was fixed in the package netplan.io - 0.99-0ubuntu1

---------------
netplan.io (0.99-0ubuntu1) focal; urgency=medium

  [ Łukasz 'sil2100' Zemczak ]
  * New upstream release: 0.99 (LP: #1871825)
    - Fixed setting MTUBytes= in .network files as well
    - Added "phase2" keyword to "auth" section
    - Allowing "critical" to be used without "dhcp4"/"dhcp6" enabled
    - Added support for GSM modems in the NetworkManager backend (with the
      "modems" keyword)
    - Added "emit-lldp" option for networkd backend (LP: #1862607)
    - Fixed netplan incorrectly generating WPA PSK hex (LP: #1867690)
    - Split out the netplan parser into a separate libnetplan library
    - Added "ipv6-address-generation" field for NM backend
    - Added WiFi flags for "bssid"/"band"/"channel"
    - Added support for SR-IOV network devices
  * debian/copyright: Change contact address as Matt is no longer available
    via the previous e-mail.
  * debian/control: Add new libnetplan packages
  * Drop d/p/0002-Adopt-integration-tests-for-NetworkManager-v1.22-foc.patch:
    included in upstream release

  [ Lukas Märdian ]
  * Drop d/p/workaround_tests_issues.patch:
    The problem was solved upstream and is integrated via
    d/p/0002-Adopt-integration-tests-for-NetworkManager-v1.22-foc.patch

 -- Łukasz 'sil2100' Zemczak <email address hidden> Thu, 16 Apr 2020 09:13:50 +0200

[Brian Murray (brian-murray)]

Hello Łukasz, or anyone else affected,

Accepted netplan.io into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/netplan.io/0.99-0ubuntu3~20.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Brian Murray (brian-murray)]

Hello Łukasz, or anyone else affected,

Accepted netplan.io into eoan-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/netplan.io/0.99-0ubuntu3~19.10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-eoan to verification-done-eoan. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-eoan. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Brian Murray (brian-murray)]

Hello Łukasz, or anyone else affected,

Accepted netplan.io into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/netplan.io/0.99-0ubuntu3~18.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted netplan.io (0.99-0ubuntu3~18.04.1) for bionic have finished running.
The following regressions have been reported in tests triggered by the package:

netplan.io/0.99-0ubuntu3~18.04.1 (amd64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/bionic/update_excuses.html#netplan.io

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted netplan.io (0.99-0ubuntu3~19.10.1) for eoan have finished running.
The following regressions have been reported in tests triggered by the package:

netplan.io/0.99-0ubuntu3~19.10.1 (ppc64el)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/eoan/update_excuses.html#netplan.io

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted netplan.io (0.99-0ubuntu3~20.04.1) for focal have finished running.
The following regressions have been reported in tests triggered by the package:

initramfs-tools/0.136ubuntu6 (amd64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/focal/update_excuses.html#netplan.io

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Lukas Märdian (slyon)]

Hi Brian,

for Eoan SRU verification, I checked the logs of the automatic integration test-suite (autopkgtests) of netplan.io (0.99-0ubuntu3~19.10.1) and made sure all tests (except the FLAKY ones) finished successfully.

I had to re-trigger one autopkgtest for eoan/ppc64el, due to a timing issue (flaky test); it ran successfully the 2nd time and all autopkgtests for Eoan are now green. All eoan test logs are attached to the description of this bug report.

Additionally, I did the manual testing for #1874377 on a Raspberry Pi 3.

All looking good to me!

[Lukas Märdian (slyon)]

Hi Brian,

for Bionic SRU verification, I checked the logs of the automatic integration test-suite (autopkgtests) of netplan.io (0.99-0ubuntu3~20.04.1) and made sure all tests (except the FLAKY ones) finished successfully.

I had to re-trigger one autopkgtest for bionic/amd64, due to a timing issue (flaky test); it ran successfully the 2nd time and all autopkgtests for Bionic are now green. All bionic test logs are attached to the description of this bug report.

All looking good to me!

[Lukas Märdian (slyon)]

Hi Brian,

in the previous comment I was testing netplan.io "0.99-0ubuntu3~18.04.1" for Bionic, of course.

for Focal SRU verification, I checked the logs of the automatic integration test-suite (autopkgtests) of netplan.io (0.99-0ubuntu3~20.04.1) and made sure all tests (except the FLAKY ones) finished successfully.

I had to re-trigger one autopkgtest for focal/amd64 (initramfs-tools regression), due to a flaky test; it ran successfully the 2nd time and all autopkgtests for Focal are now green. All focal test logs are attached to the description of this bug report.

Additionally, I did the manual testing for #1874377 on a Raspberry Pi 3.

All looking good to me!

[Chris Halse Rogers (raof)]

The verification of the Stable Release Update for netplan.io has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package netplan.io - 0.99-0ubuntu3~18.04.1

---------------
netplan.io (0.99-0ubuntu3~18.04.1) bionic; urgency=medium

  * Backport netplan.io 0.99 to 18.04. (LP: #1871825)
  * Keep patches specific to 18.04 support:
    - disable-networkd-tunnels-ipip-gre.patch: disable tests for unsupported
      tunnel types (ipip and gre) in the 18.04 version of systemd-networkd.
  * debian/control: introduce dependecies on libnetplan0

 -- Lukas Märdian <email address hidden> Mon, 04 May 2020 09:51:02 +0200

[Launchpad Janitor (janitor)]

This bug was fixed in the package netplan.io - 0.99-0ubuntu3~19.10.1

---------------
netplan.io (0.99-0ubuntu3~19.10.1) eoan; urgency=medium

  * Backport netplan.io 0.99 to 19.10. (LP: #1871825)
  * Include proper upstream fix for "Not connect to WiFi after 'netplan apply'
    (LP: #1874377)

 -- Lukas Märdian <email address hidden> Mon, 04 May 2020 11:48:04 +0200

[Launchpad Janitor (janitor)]

This bug was fixed in the package netplan.io - 0.99-0ubuntu3~20.04.1

---------------
netplan.io (0.99-0ubuntu3~20.04.1) focal; urgency=medium

  * Backport upstream bug fix for 0.99 to 20.04. (LP: #1871825)
    - Upstream provides an imporved fix, which handles additional edge cases
      for "Not connect to WiFi after 'netplan apply'" (LP: #1874377)

 -- Lukas Märdian <email address hidden> Mon, 04 May 2020 11:37:05 +0200