[SRU] Update to netplan.io 0.106.1

[Steven Friedrich]

[Impact]
This release contains both bug-fixes and new features and we would like to make sure all of our supported customers have access to those improvements.

The most notable changes (besides usual bug-fixes) incoming in 0.106.1:
- New 'netplan status' CLI by @slyon in #290
- API: implement APIs from the new specification by @daniloegea in #298
- Check and fix non-inclusive language by @daniloegea in #303
- Documentation improvements (using Diátaxis & RTD) by @slyon
- Match by PermanentMACAddress by @rlaager in #278
- Netplan api iterator by @daniloegea in #306
- API: update netplan_delete_connection() to avoid spawning another process by @slyon in #322
- NetworkManager 1.40 compat (LP#1997348) by @slyon in #300
- Migrate from (deprecated) nose to pytest by @daniloegea in #302
- parse: Add the filepath to OVS ports netdefs by @daniloegea in #295
- Check if the interface name is too long (LP#1988749) by @daniloegea in #313
- doc/examples: remove unnecessary route for IPv6 on-link gateways by @sbraz in #312
- Memory leak CI action by @daniloegea in #321
- Improve stability of autopkgtests by @slyon in #223
- Add parser support for WPA3 Personal (LP: #2023238)
- Make use of 'meson' buildsystem

[Test Plan]
The following development and SRU process was followed:
https://wiki.ubuntu.com/NetplanUpdates

Netplan contains an extensive integration test suite that is run using
the SRU package for each release. This test suite's results are available here:
http://autopkgtest.ubuntu.com/packages/n/netplan.io

A successful run is required before the -proposed netplan.io package
can be let into -updates.

The netplan team will be in charge of attaching the artifacts and console
output of the appropriate run to the bug. Netplan team members will not
mark ‘verification-done’ until this has happened.

[Where problems could occur]
Netplan being a core package, it could impact the whole networking stack of the operating system up to the point where servers would not be reachable anymore after a reboot, due to broken network config being generated by netplan at bootup.

In order to mitigate the regression potential, the results of the aforementioned integration tests are attached to this bug:

Lunar:
https://git.launchpad.net/~slyon/+git/files/tree/LP2025519/lunar_amd64.log
https://git.launchpad.net/~slyon/+git/files/tree/LP2025519/lunar_arm64.log
https://git.launchpad.net/~slyon/+git/files/tree/LP2025519/lunar_armhf.log
https://git.launchpad.net/~slyon/+git/files/tree/LP2025519/lunar_ppc64el.log
https://git.launchpad.net/~slyon/+git/files/tree/LP2025519/lunar_s390x.log

Jammy:
https://git.launchpad.net/~slyon/+git/files/tree/LP2025519/jammy_amd64.log
https://git.launchpad.net/~slyon/+git/files/tree/LP2025519/jammy_arm64.log
https://git.launchpad.net/~slyon/+git/files/tree/LP2025519/jammy_armhf.log
https://git.launchpad.net/~slyon/+git/files/tree/LP2025519/jammy_ppc64el.log
https://git.launchpad.net/~slyon/+git/files/tree/LP2025519/jammy_s390x.log
=> Failures in test:tunnels are not Netplan regressions and explained in detail in comment 19+ and bug #2037667
FAIL: test_tunnel_gre6 (__main__.TestNetworkd)
FAIL: test_tunnel_gre6_with_keys (__main__.TestNetworkd)
FAIL: test_tunnel_vti6 (__main__.TestNetworkd)

[Other Info]
The integration test logs will be attached to this bug, once the package has been accepted into -proposed and the tests have been executed on the real infrastructure.

[Changelog]
cff3d06f Update changelog
f5b4c2c2 Update symbols file for 0.106.1, avoiding any unnecessary exports
9a0e4bfd d/t/sru-compat: Add patches for Jammy LTS backwards compatibility
d6a05570 d/control: Drop python3-rich dependency to Suggests
0af85f81 Merge tag 'ubuntu/0.106.1-7ubuntu0.23.04.1' into ubuntu-jammy-sru
d44838fa Update changelog for 0.106.1-7ubuntu0.23.04.1 SRU
bfd4d502 Add patches for SRU backwards compatbility
fbf7f273 Revert "Update symbols file for 0.106"
a5400dfa d/control: Adopt Vcs-Git fields
3653a566 Update debian/gbp.conf
f9fe6045 update-maintainer
57740f43 Revert "0.106.1-4 (patches unapplied)"
776827be Revert "0.106.1-5 (patches unapplied)"
b238a96e Upload 0.106.1-7 into unstable
0d781675 Update changelog
3cf18ef2 d/control: Update my DD email address
b5b1d47f Don't drop files with just global values on 'set' (LP: #2027584)
5091f42a Update changelog for 0.106.1-6
0ec25f42 Update changelog
7fb84f3d Fix ethernets,vlans,scenarios autopkgtests on systemd 254, Closes: #1041311
ea8a4769 Update changelog for 0.106.1-5
9eaf3452 d/rules,d/netplan.io.{install,dirs}: lib/systemd/system-generators/netplan
2a820c15 Update changelog for 0.106.1-4
7f4f7314 d/netplan.io.lintian-overrides: Mute empty /usr/lib/netplan, it's used for
b8440287 Update changelog
851b478b d/rules,d/netplan.io.{install,dirs}: Install everything in usrmerged paths
d385cec0 Update changelog for 0.106.1-3
cac802bc autopkgtest:regressions: Simplify assertions to avoid buffering races
9d5a26ce d/t/prep-testbed.sh: stop NetworkManager, to start with a common state
152c0176 Add parser support for WPA3 Personal (LP: #2023238)
08526fc1 Fix tunnels tests with iproute 6.4, using JSON interface (Closes: #1040004)
110844bc Fix NetworkManager interaction with systemd v253 (Closes: #1039071)
475d9aa4 Update changelog for 0.106.1-2
392bb641 Increase test timeouts to fix FTBFS on mips64el and riscv64
f4009f33 Update changelog for 0.106.1-1
e5302294 d/control: Cleanup list of Uploaders.
e21734f0 Add d/p/0012-meson-Increase-test-timeout-for-slow-architectures.patch
749a9844 Refresh patches.
e88e3100 Merge branch 'ubuntu-mantic' into debian/unstable
4a4d6617 debian/tests: Enforce more strict integration tests
8cfdb3b6 release 0.106.1-0ubuntu2
167a8e20 Changelog
f9357b6d d/p/lp2021884: Wireguard: auto append the prefix to allowed IPs
754c4582 Add patches addressing LP: #2016473
0f6b209f Update symbols file for 0.106
3de29152 * d/patches/: Drop patches, applied upstream
96fdb2e0 Update changelog
ead4bf9c Update upstream source from tag 'upstream/0.106.1'
c07e3229 New upstream version 0.106.1
58fc884e d/t/prep-testbed.sh: Fix systemd-resolved issue in ethernets test
64b7333f d/t/{autostart,cloud-init}.sh: Systemd v253 test compat (Closes: #1033265)
316678e9 d/gbp.conf: Update for Mantic
f3278f02 d/p/lp1997467: Improve DEP-3 headers
5df4140c meson: build version to 0.106.1
4c27d3ec CI: enable Github actions on stable/**
b5d8017f CI: update canonical/setup-lxd to v0.1.1 (#359)
9cf1ac82 parse-nm: fix eap_method handling
81f0c3eb rtd: set the OS and Python versions explicitly
1af675b1 CI: make use of the canonical/setup-lxd action
274ae49f doc: Add Ubuntu Code of Conduct 2.0
cfb104b3 doc: make us of sphinx copybutton plugin
652e35cf ovs: replace openvswitch instances by Open vSwitch
3125e5b3 ovs: don't allow peers with the same name
9ec17498 ABI: convert NetplanBackendSettings to struct
89d7aace parse: validate NM backend settings usage
2f71b62d nm: check the passthrough config format
e55e54e6 wireguard: improve an error message
1603ca62 wireguard: plug a memory leak
3a0b45a1 validate: drop the YAML node parameter
0559bf31 parse: don't point to the wrong node on validation
c4552319 parse: set the backend on nm-devices to NM by default (#349)
19113efb parse: plug a memory leak (#348)
c052d345 nm: return if write_routes() fails
623a690b libnetplan: don't try to read from a NULL file
0aa4e0f3 networkd: plug a memory leak
15a422d6 vrfs: skip policies parsing if list is NULL (LP: #2016427) (#341)
ca6deb24 docs: fix bridge parameters types and add examples (#346)
c226358d parse: use "--" with systemd-escape
f88eba2b netplan: adjust the maximum buffer size to 1MB
95787dd6 cli:status: improve networkctl parameter passing
d5c1494c cli:test: use 'text' alias instead of 'universal_newlines'
3d194603 Makefile: fully cleanup coverage data
8a596750 cli:apply: use shutil.rmtree instead of 'rm -rf'
048c931b Use controlled execution environment, to avoid failure if PATH is unset (LP: #1959570) (#336)
8df0fb5f Netplan docs rework (Part 2) (#337)
7faad4cc meson: fix installation of legacy /lib/netplan/generate symlink
b295185b spread.yaml: restore /etc/netplan for each test
2b6f77af tests: add new spread based snapd integration test
32273fdc Netplan docs rework (#333)
9f1c34b6 tests: Add some integration tests for DBus
525e5ad3 dbus: Use the error set by _copy_yaml_state()
76b06087 dbus: Build the copy path correctly
761ce970 parse:ovs: Ignore deprecated OpenFlow1.6 protocol (LP: #1963735)
31dfd022 parser: Don't duplicate OVS configuration
14b983ac tests: Add tests for the new C functions
1321aa65 tests: Don't use duplicated DNS entries
f267d0e7 parser: Don't add DNS entries duplicates
a8ddad5a parser: handle duplicated routing rules
a4638755 netplan: cli: fix typo from 'unkown' to 'unknown'
e32d1854 meson: cli:utils: drop legacy generator path, use libexec instead
8761c9af meson: use meson_make_symlink.sh helper script instead of install_symlink()
5a97f445 Add all the commands to the bash completion file (LP: #1749869) (#326)
73265702 Revert "meson: use absolute link target for generator symlink"
05bc53ac parser: demote the new lacp-rate check to warning
6aa5c8d3 ATTN: parser: validate lacp-rate properly (LP: #1745648) (#324)
cbb42084 meson: use absolute link target for generator symlink
47af2446 Revert "meson: Account for UsrMerged systemd paths"
b25994f4 rpm: Refresh packaging for 0.106 and sync from Fedora
3276da79 meson: Account for UsrMerged systemd paths
6def2a5d meson: Enable finding pycoverage on RPM based distributions
2e0c0746 docs: Add a short netplan-everywhere howto (#325)
5525e8c5 CI: CodeQL: add cmocka build-deps
d7c37dd4 CI: CodeQL: add pytest build-deps
9a7d0e3d Update changelog
891f8024 Cherry-pick upstream commit fc12872 (PR#336) to fix crash in 'snapd set'
88a7c8d6 Update changelog
633bcd5f d/p/lp1997467: Improve DEP-3 headers
1cb1e455 debian/tests: Fix testbed timeout with systemd v253 (Closes: #1033265)
d448baeb Update changelog
a2c174da d/p/lp1997467/0009-dbus-Build-the-copy-path-correctly.patch: properly build the destination path before copying files in the dbus integration (LP: #1997467)
d8c5ccc0 Update changelog
29edf94f Fix DBus .Config/.Get APIs using upstream commits from PR#331 (LP: #1997467)
6d4d343e Update changelog
84e3a29b Fix DBus .Config/.Get APIs using upstream commits from PR#331 (LP: #1997467)
6368d8c5 Update changelog
2216e116 d/libnetplan0.symbols: Add netplan_parser_load_nullable_overrides() API
d7abf181 d/p/lp1997467: set only specific origin-hint if given (LP: #1997467)
f50d8273 Update changelog
0d7eedbe Update changelog for 0.106
8fe82291 d/gbp.conf: use pristine-tar
7310fc8f d/source: add lintian-overrides
3d4b07de d/source: add lintian-overrides
1257481e d/rules: cleanup whitespace
daea0778 Update d/copyright for 2023
dbd5b6b2 d/rules: cleanup whitespace
d08fcbe8 Update d/copyright for 2023
bf23b3e6 Merge tag 'ubuntu/0.106-0ubuntu1' into debian/unstable
2b075477 Update changelog
43935bcf d/libnetplan0: update .symbols file
ed578139 debian/patches: Drop patches, applied upstream
1e920906 d/control: bump Standards-Version to 4.6.2, no changes needed
1b1de704 d/t/control: prepare Debian testbed
2c8b2897 d/control: Add python3-dbus, python3-rich deps, also CMocka and Pytest B-Ds
bd29544a d/watch: fix checking for upstream tags
5efd7803 Update upstream source from tag 'upstream/0.106'
3f128bbe New upstream version 0.106
15ce044d meson: build version to 0.106
7f7ad860 abi-compat: bump for 0.106
c658bf04 abi: ignore the glob_t change related to _GNU_SOURCE
88d08d4f test:ctest: adopt for netplan_delete_connection API
5985daf3 util: update netplan_delete_connection() API
307ce818 meson: bump to _GNU_SOURCE, avoid explicit __USE_MISC
d2655551 tests:base:ethernets: Improve stability of autopkgtests (#223)
eb0ff444 parser: check for route duplicates (LP: #2003061) (#320)
2e06dde0 meson: give some more time for c coverage output to avoid timeouts
581e1473 Match by PermanentMACAddress (#278)
492d1621 vxlan: convert some settings to tristate (LP: #2000712) (#311)
01099701 CI: Memory leak action (#321)
102f189e src:parse: plug memory leaks in nullable handling
658acf6d doc:yaml: improve formatting
e99c7ef8 Check and fix non-inclusive laguange (#303)
f32b0dbd Update changelog for 0.105-0ubuntu4
348694c1 d/libnetplan0.symbols: Add netplan_parser_load_nullable_overrides() API
6982ff9b d/p/lp1997467: set only specific origin-hint if given (LP: #1997467)
d8659bbc src:parse:netplan: write global renderer depending on output file
3d178725 cli:set: fix origin-hint handling (LP: #1997467)
93985784 parse: Allow loading nullable origin-hint overrides (netdefs to be ignored)
11ad90ee test:cli:set: add regression cases for LP: #1997467
7b813373 generate:util: fix double-slash root filepath
469e695e readthedocs: add python requirements.txt
2dad7d1e Add .readthedocs.yaml configuration
5dc87982 doc:index: switch symbolic link, for easier modification through RTD
639ad01b doc:index:conf: use grid instead of table, for responsive rendering
1aa7f148 doc:index: add 'In this documentation' subtitle
db043801 doc:reference: mention YAML spec version and expected file permissions
db608aae doc/examples: remove unnecessary route for IPv6 on-link gateways (#312)
d53d96c9 Check if the interface name is too long (LP: #1988749) (#313)
22de5747 ctest:parser: adopt for netplan_error_clear rename
81fc9eab networkd: make sure VXLAN is in the right section (LP: #2000713) (#310)
45b25f6a apply: Fix crash when OVS is stopped (LP#1995598) (#307)
ac605d43 api:util: rename netplan_error_clear (from _free) & adopt to nullify
2623fb51 parser: return the correct error on failure (LP#2000324) (#308)
bfa82ad9 docs: small fix on netplan-set doc
bc751e7f ctests: fix an integer conversion issue
cab830cb tests: mock calls to systemctl
e5ff9f6d parser: plug a memory leak (#309)
bb863538 New 'netplan status' CLI (#290)
4788d4bd doc: Add Netplan logo
dc879923 meson: fix coverage-py-combine race condition (fixup PR #302)
f20389f9 cli:apply: fix logger deprecation warning
7de9726a Netplan api iterator (#306)
c193697c parse: Add the filepath to OVS ports netdefs (#295)
2ab53455 API: implement APIs from the new specification (#298)
10512d11 Update changelog for 0.105-3
bf76de9f d/control: drop python3-nose B-D in favor of python3-pytest (Closes: #1018420)
d36039fa d/patches: Migrate from deprecated nose to pytest (Closes: #1020079)
52c572de Migrate from (deprecated) nose to pytest (#302)
bf8036d4 Fix a bug in the generate --mapping command (LP: #1996941) (#301)
eda0e9c0 src:parse: Log a warning on weak .yaml file permissions (LP: #1862600)
4e15aec7 src:netplan:abi_compat: use owner (root) read-only permissions
0ed5acd8 src:parse-nm: add default wakeonlan setting only for ethernet devices
53e33fc7 doc:routes: fix docs for default route 0.0.0.0/0 vs 0/0 notation
26b5fb5f Update changelog for 0.105-0ubuntu3
5dce3b5e Update debian/gbp.conf for Lunar
a240bfc5 d/netplan.io.lintian-overrides: Fix new style override
e9e785ea d/copyright: Update metadata.
7c8779c8 d/t/control: Make it work with old and new systemd[-resolved]
781c3163 d/rules,d/control,d/libnetplan.io.install,d/libnetplan-dev.install:
54525842 d/p/0002-buildsystem-meson-fixes.patch: improve meson build
26ea3a66 src: Fix some memory leaks (#297)
5afb97a5 doc:netplan-yaml: specify default usage of WPA+WPA2 wifi authentication
cc03fba2 CI: Fix CodeQL build by installing meson on a Jammy runner
d2d92fa1 CI: upgrade to CodeQL v2
68decb8d README: link documentation objectives
6e5cc122 docs: add code-of-conduct
97b46fc7 doc:dbus-config: cross-linking to reference
69da1bfd doc: Add Netplan D-Bus reference
c225c7d5 doc: Improve CLI documentation
0419f377 doc: spelling fixes
2d8bd32a doc: integrate manpages as CLI reference
ee19e975 doc: YAML refactor generic topics into explanation, use external links in explanation
60699a32 doc:README: adopt to default website template and Diataxis navigation ordering

=== Original bug description ===

I recently bought a Dell XPS-13 Plus. I was following the netplan tutorial and it's based on netplan 0.106
My Ubuntu 22.04.2 doesn't have this latest version.
I invoke software updater daily.

[Lukas Märdian (slyon)]

We plan to backport Netplan 0.106.1 into Jammy LTS eventually, but it's currently blocked by validation through our public cloud partners: https://discourse.ubuntu.com/t/netplan-0-106-call-for-testing/33932

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Pavel Malyshev (afunix)]

So I'm affected by https://bugs.launchpad.net/netplan/+bug/2000713 and https://bugs.launchpad.net/netplan/+bug/2003061 on 22.04 LTS.
Is there any plan to backport the fixes that are available for these bugs back to 22.04?
Bugs are known for over a half a year at the moment.

[Lukas Märdian (slyon)]

@pavel: Yes, both should be fixed by the 0.106.1 backport in 22.04.

[Lukas Märdian (slyon)]

This is now pending SRU review.

Lunar (tag: ubuntu/0.106.1-7ubuntu0.23.04.1)
- https://git.launchpad.net/~ubuntu-core-dev/netplan/+git/ubuntu/log/?h=ubuntu-lunar
- https://launchpad.net/ubuntu/lunar/+queue?queue_state=1&queue_text=netplan.io

Jammy (tag: ubuntu/0.106.1-7ubuntu0.22.04.1)
- https://git.launchpad.net/~ubuntu-core-dev/netplan/+git/ubuntu/log/?h=ubuntu-jammy
- https://launchpad.net/ubuntu/jammy/+queue?queue_state=1&queue_text=netplan.io

[Chris Halse Rogers (raof)]

Hello Steven, or anyone else affected,

Accepted netplan.io into lunar-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/netplan.io/0.106.1-7ubuntu0.23.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-lunar to verification-done-lunar. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-lunar. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Steve Langasek (vorlon)]

I've reviewed the package in the jammy queue, since Chris seemed to only get to the lunar one in his SRU shift. I have some questions that I expect may apply to both uploads.

It was initially confusing to me that this was a backport of a version that's been superseded in mantic, and also that it includes the addition of a number of files under debian/patches; I wondered why these had not been committed upstream. Then I saw that these were present in the -7 package, so this is ok from that perspective.

So then I diffed the upload in the jammy queue against 0.106.1-7 and found other packaging differences that don't make sense:

--- netplan.io-0.106.1/debian/libnetplan0.symbols 2023-07-20 06:11:50.000000000 -0700
+++ netplan.io-0.106.1/debian/libnetplan0.symbols 2023-08-24 03:11:14.000000000 -0700
@@ -26,9 +26,6 @@
  get_unspecified_address@Base 0.104
  global_backend@Base 0.104
  global_state@Base 0.104
- is_route_present@Base 0.106
- is_route_rule_present@Base 0.106.1
- is_string_in_array@Base 0.106.1
  is_wireguard_key@Base 0.100
  netdefs@Base 0.99
  netdefs_ordered@Base 0.99

Why is this backport missing symbols entries that are present in the unstable version? I see you've included a patch, debian/patches/sru-compat/0024-Avoid-unnecessary-export-of-API.patch, that patches these symbols out (which, confusingly accomplishes this by removing the NETPLAN_INTERNAL tag, which on its surface would appear to do the opposite?). But the net effect here is that you are now introducing a libnetplan0 package in the archive, whose version number is >= 0.106, but whose ABI does not match what is declared in later versions - so if someone installs this package in jammy, then upgrades to mantic, apt may not have correct information about the necessary order in which packages must be upgraded, because a new package using these symbols in mantic will think that the version in jammy satisfies the dependency!

So I think the correct course of action here is to NOT patch these out of the library's ABI.

--- netplan.io-0.106.1/debian/netplan.io.install 2023-07-25 03:18:01.000000000 -0700
+++ netplan.io-0.106.1/debian/netplan.io.install 2023-08-24 01:24:24.000000000 -0700
@@ -1,3 +1,4 @@
+lib/netplan/*
 lib/systemd/system-generators/netplan
 usr/libexec/netplan/*
 usr/share/*

Why is this present only in the jammy backport and not in the unstable version? (Particularly as /lib should be avoided now in favor of /usr/lib)

[Lukas Märdian (slyon)]

Thank you for the reviews!

> It was initially confusing to me that this was a backport of a version that's been superseded in mantic, and also that it includes the addition of a number of files under debian/patches; I wondered why these had not been committed upstream. Then I saw that these were present in the -7 package, so this is ok from that perspective.

Right, I backported the 0.106.1-7 version, as I didn't want to introduce the package split into the LTS for now. It's not needed and only makes things more complicated. Also, the (non-superseeded) 0.107 version in Mantic is not yet ready for SRU, as it didn't see lots of real-world usage yet, thus the backport of 0.106.1.

> [...] But the net effect here is that you are now introducing a libnetplan0 package in the archive, whose version number is >= 0.106, but whose ABI does not match what is declared in later versions - so if someone installs this package in jammy, then upgrades to mantic, apt may not have correct information about the necessary order in which packages must be upgraded, because a new package using these symbols in mantic will think that the version in jammy satisfies the dependency!

Those symbols were exported by accident in 0.106[.1] and are (supposed to be) unused. They will be dropped with an SOVER bump in 24.04. I thought it would be better to not export them in the stable series, so people don't start to depend on it. But I see the ordering issue you're describing on the Jammy -> Mantic dist upgrade. We can easily drop that patch (and update the .symbols file accordingly).

> debian/netplan.io.install => lib/netplan/*

In Mantic we're dropping a legacy symlink for the Netplan generator in "lib/netplan/generate" (see debian/rules:execute_after_dh_auto_install), in order to make room for a fully usrmerge-ready package (still depends on some updated systemd "generator path" pkgconf data). In the stable series we want to keep that symlink on disk for backwards compatibility. In case anybody depends on it.

[Łukasz Zemczak (sil2100)]

Hello Steven, or anyone else affected,

Accepted netplan.io into lunar-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/netplan.io/0.106.1-7ubuntu0.23.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-lunar to verification-done-lunar. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-lunar. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Łukasz Zemczak (sil2100)]

Hello Steven, or anyone else affected,

Accepted netplan.io into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/netplan.io/0.106.1-7ubuntu0.22.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted netplan.io (0.106.1-7ubuntu0.23.04.2) for lunar have finished running.
The following regressions have been reported in tests triggered by the package:

initramfs-tools/unknown (s390x)
netplan.io/unknown (s390x)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/lunar/update_excuses.html#netplan.io

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted netplan.io (0.106.1-7ubuntu0.22.04.2) for jammy have finished running.
The following regressions have been reported in tests triggered by the package:

netplan.io/0.106.1-7ubuntu0.22.04.2 (amd64, arm64, ppc64el, s390x)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/jammy/update_excuses.html#netplan.io

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Danilo Egea Gondolfo (danilogondolfo)]

There are 3 tests failing in Jammy: test_tunnel_gre6, test_tunnel_gre6_with_keys and test_tunnel_vti6.

networkd is logging some "Operation not supported" messages that appear to be related:

---
Sep 08 16:45:53 jammy2 systemd-networkd[333]: tun0: netdev could not be created: Operation not supported
Sep 08 16:45:53 jammy2 systemd-networkd[333]: tun1: netdev could not be created: Operation not supported
Sep 08 16:45:53 jammy2 systemd-networkd[333]: tun2: netdev could not be created: Operation not supported
---

Trying to create the ip6gre tunnel manually also fails, which means the kernel lacks support for it:

ip link add name tun0 type ip6gre local ::1 remote 1234:4321::1
Error: Unknown device type.

NetworkManager seems to be passing the GRE6 tests:

3204s test_tunnel_gre6 (__main__.TestNetworkManager) ... tun0 ok
3206s test_tunnel_gre6_with_keys (__main__.TestNetworkManager) ... tun0 ok

But if I try it manually on Jammy the interfaces will not be created and NM will log some errors:

---
Sep 08 16:51:46 jammy2 NetworkManager[1550]: <info> [1694191906.6717] manager: (tun1): new IPTunnel device (/org/freedesktop/NetworkManager/Devices/3)
Sep 08 16:51:46 jammy2 NetworkManager[1550]: <warn> [1694191906.6727] platform-linux: do-add-link[tun1/ip6gre]: failure 95 (Operation not supported - Unknown device type)
Sep 08 16:51:46 jammy2 NetworkManager[1550]: <error> [1694191906.6728] manager: (netplan-tun1) couldn't create the device: Failed to create IPv6 tunnel interface 'tun1' for 'netplan-tun1': Operation not support>
Sep 08 16:51:46 jammy2 NetworkManager[1550]: <info> [1694191906.6730] manager: (tun0): new IPTunnel device (/org/freedesktop/NetworkManager/Devices/4)
Sep 08 16:51:46 jammy2 NetworkManager[1550]: <warn> [1694191906.6739] platform-linux: do-add-link[tun0/ip6gre]: failure 95 (Operation not supported - Unknown device type)
Sep 08 16:51:46 jammy2 NetworkManager[1550]: <error> [1694191906.6739] manager: (netplan-tun0) couldn't create the device: Failed to create IPv6 tunnel interface 'tun0' for 'netplan-tun0': Operation not support>
Sep 08 16:51:46 jammy2 NetworkManager[1550]: <info> [1694191906.6740] failed to open /run/network/ifstate
---

Now, installing the linux-virtual-hwe-22.04 kernel (6.2) seems to resolve all these issues. All the tunnels tests pass and I can create the ip6gre manually, with networkd and with Network Manager.

That said, we should investigate why the NM tests are passing. Because it clearly can't create the gre6 interfaces.

To get this moving, we probably should disable these 3 tests as it's related to the default Jammy's kernel and we can verify they work fine with the new one.

[Lukas Märdian (slyon)]

Thanks for your investigation.

The interesting bit is, that the same tests have been passing on 0.105 in Jammy on the same kernel before:
https://autopkgtest.ubuntu.com/results/autopkgtest-jammy/jammy/amd64/n/netplan.io/20230824_233135_aa3f5@/log.gz

And also passed during my pre-testing of 0.106.1 in Jammy on the same kernel:
https://autopkgtest.ubuntu.com/results/autopkgtest-jammy-slyon-testing/jammy/amd64/n/netplan.io/20230802_082036_28e79@/log.gz

Furthermore, those tests PASS in Netplan's upstream CI, which runs autopkgtests in an Jammy LXD container (on a Jammy host/kernel). So I really have the impression there's some issue with Ubuntu's test infrastructure here.

So I wonder what changed here. Was there some change in systemd-networkd? Or is it maybe missing kernel modules on the test infrastructure? Or missing some IPv6 configuration?

[Lukas Märdian (slyon)]

I tested 0.106.1-7ubuntu0.23.04.2 from lunar-proposed and attached the autopkgtest logs to the bug description.

Looking all good.

[Launchpad Janitor (janitor)]

This bug was fixed in the package netplan.io - 0.106.1-7ubuntu0.23.04.2

---------------
netplan.io (0.106.1-7ubuntu0.23.04.2) lunar; urgency=medium

  * Drop d/p/sru-compat/0024-Avoid-unnecessary-export-of-API.patch.
    Also, update the .symbols file, according to SRU review.

netplan.io (0.106.1-7ubuntu0.23.04.1) lunar; urgency=medium

  * Backport netplan.io 0.106.1-7 to 23.04 (LP: #2025519)
    - Add all the commands to the bash completion file (LP: 1749869)
    - parse-nm: fix eap_method handling (LP: 2016625)
    - ovs: don't allow peers with the same name
    - parse: validate NM backend settings usage
    - nm: check the passthrough config format
    - wireguard: plug a memory leak
    - validate: drop the YAML node parameter
    - parse: don't point to the wrong node on validation
    - parse: set the backend on nm-devices to NM by default
    - parse: plug a memory leak
    - nm: return if write_routes() fails
    - libnetplan: don't try to read from a NULL file
    - networkd: plug a memory leak
    - vrfs: skip policies parsing if list is NULL (LP: 2016427)
    - parse: use "--" with systemd-escape
    - netplan: adjust the maximum buffer size to 1MB
    - cli:status: improve networkctl parameter passing
    - cli:test: use 'text' alias instead of 'universal_newlines'
    - cli:apply: use shutil.rmtree instead of 'rm -rf'
    - Use controlled execution environment, to avoid failure if PATH is unset
      (LP: 1959570)
    - meson: fix installation of legacy /lib/netplan/generate symlink
    - dbus: Use the error set by _copy_yaml_state()
    - dbus: Build the copy path correctly (LP: 1997467)
    - parse:ovs: Ignore deprecated OpenFlow1.6 protocol (LP: 1963735)
    - parser: Don't duplicate OVS configuration (LP: 2007682)
    - parser: Don't add DNS entries duplicates (LP: 2007682)
    - parser: handle duplicated routing rules (LP: 2007682)
    - netplan: cli: fix typo from 'unkown' to 'unknown'
    - meson: cli:utils: drop legacy generator path, use libexec instead
    - meson: use meson_make_symlink.sh helper instead of install_symlink()
    - ATTN: parser: validate lacp-rate properly (LP: 1745648)
    - parser: demote the new lacp-rate check to warning
    - Don't drop files with just global values on 'set' (LP: 2027584)
      d/p/0020-netplan.c-Don-t-drop-files-with-just-global-values-o.patch
    - Add parser support for WPA3 Personal (LP: 2023238)
    - d/patches/lp2016473: Keyfile parsing for all tunnel types, LP: 2016473
    - d/p/lp2021884: WireGuard: auto append prefix to allowed IPs, LP: 2021884
    - d/p/lp1997467: Improve DEP-3 headers
    [CI]
    - CI: make use of the canonical/setup-lxd action
    - Add new spread based snapd integration test
    - Add some integration tests for DBus
    - CI: update canonical/setup-lxd to v0.1.1
    - Makefile: fully cleanup coverage data
    - spread.yaml: restore /etc/netplan for each test
    - tests: Don't use duplicated DNS entries (LP: 2007682)
    - rpm: Refresh packaging for 0.106 and sync from Fedora
    - meson: Enable finding pycoverage on RPM based distributions
    - Fix ethernets,vlans,scenarios autopkgtests on systemd 254
    - Fix NetworkManager interaction with sy...

[Chris Halse Rogers (raof)]

The verification of the Stable Release Update for netplan.io has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Danilo Egea Gondolfo (danilogondolfo)]

I'm looking at the tests failing on Jammy more carefully here. I didn't notice that the kernel used by our test environment is not the same used by the default LXD images. I'm using the cloud image now and the kernel does have the required modules for ip6gre and ip6vti interfaces.

networkd can't create the gre6 interface. It fails with "tun0: netdev could not be created: Invalid argument"

It can be reproduced with this configuration:

network:
  version: 2
  renderer: networkd
  tunnels:
    tun0:
      #renderer: NetworkManager
      mode: ip6gre
      local: fe80::1
      remote: 2001:dead:beef::2

But Network Manager works just fine, it will create the interface!

Installing the HWE kernel (6.2) fixes the issue. Running the test with "autopkgtest --setup-commands="apt install -y linux-virtual-hwe-22.04"" makes it work.

Reverting to an old systemd (249.11-0ubuntu3) doesn't fix the problem.

Reverting to an old kernel *does* fixes the problem. I installed linux-image-5.15.0-25-generic and it works!

Some kernel interface broke at some point and broke networkd...

[Danilo Egea Gondolfo (danilogondolfo)]

After testing all the 5.15.0 kernels available in the archive, I've found that it broke between versions 5.15.0-82.91 and 5.15.0-83.92.

When running the version 5.15.0-82.91, systemd will create the gre6 tunnel interface but will fail with version 5.15.0-83.92.

[Danilo Egea Gondolfo (danilogondolfo)]

I filed a bug report with a reproducer here https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2037667

[Lukas Märdian (slyon)]

Thank you for the investigation. The autopkgtest history shows that the test passed on "testbed running kernel: Linux 5.15.0-82-generic", is also passing on armhf containers "testbed running kernel: Linux 5.4.0-159-generic" and on newer kernels "testbed running kernel: Linux 6.2.0-1012-azure" but fails on current 5.15 "testbed running kernel: Linux 5.15.0-83-generic"

This also explains why it passed in my local pre-testing (still on an older kernel). I'm not sure if we should be skipping those tests on the Netplan side, as it seems to be a real regression on the 5.15 kernel that needs to be fixed via bug #2037667.

The Netplan 0.106.1 SRU should probably be hinted through as it's not causing this issue (migration-reference/0 was already triggered for affected architectures).

[Launchpad Janitor (janitor)]

This bug was fixed in the package netplan.io - 0.106.1-7ubuntu0.22.04.2

---------------
netplan.io (0.106.1-7ubuntu0.22.04.2) jammy; urgency=medium

  * Drop d/p/sru-compat/0024-Avoid-unnecessary-export-of-API.patch.
    Also, update the .symbols file, according to SRU review.

netplan.io (0.106.1-7ubuntu0.22.04.1) jammy; urgency=medium

  * Backport netplan.io 0.106.1-7 to 22.04 (LP: #2025519)
    - New 'netplan status' CLI (!290)
    - API: implement APIs from the new specification (!298)
    - Check and fix non-inclusive laguange (!303)
    - Documentation improvements (using Diátaxis & RTD)
    - Match by PermanentMACAddress (!278)
    - Netplan api iterator (!306)
    - API: update netplan_delete_connection() (!322)
    - NM 1.40 compat & file permission fixes (!300), LP: 1862600, LP: 1997348
    - Migrate from (deprecated) nose to pytest (!302)
    - parse: Add the filepath to OVS ports netdefs (!295)
    - Check if the interface name is too long (!313), LP: 1988749
    - doc/examples: remove unnecessary route for IPv6 on-link gateways (!312)
    - Memory leak CI action (!321)
    - tests:base:ethernets: Improve stability of autopkgtests (!223)
    - Add all the commands to the bash completion file (LP: 1749869)
    - parse-nm: fix eap_method handling (LP: 2016625)
    - ovs: don't allow peers with the same name
    - parse: validate NM backend settings usage
    - nm: check the passthrough config format
    - wireguard: plug a memory leak
    - validate: drop the YAML node parameter
    - parse: don't point to the wrong node on validation
    - parse: set the backend on nm-devices to NM by default
    - parse: plug a memory leak
    - nm: return if write_routes() fails
    - libnetplan: don't try to read from a NULL file
    - networkd: plug a memory leak
    - vrfs: skip policies parsing if list is NULL (LP: 2016427)
    - parse: use "--" with systemd-escape
    - netplan: adjust the maximum buffer size to 1MB
    - cli:status: improve networkctl parameter passing
    - cli:test: use 'text' alias instead of 'universal_newlines'
    - cli:apply: use shutil.rmtree instead of 'rm -rf'
    - Use controlled execution environment, to avoid failure if PATH is unset
      (LP: 1959570)
    - meson: fix installation of legacy /lib/netplan/generate symlink
    - dbus: Use the error set by _copy_yaml_state()
    - dbus: Build the copy path correctly (LP: 1997467)
    - parse:ovs: Ignore deprecated OpenFlow1.6 protocol (LP: 1963735)
    - parser: Don't duplicate OVS configuration (LP: 2007682)
    - parser: Don't add DNS entries duplicates (LP: 2007682)
    - parser: handle duplicated routing rules (LP: 2007682)
    - netplan: cli: fix typo from 'unkown' to 'unknown'
    - meson: cli:utils: drop legacy generator path, use libexec instead
    - meson: use meson_make_symlink.sh helper instead of install_symlink()
    - ATTN: parser: validate lacp-rate properly (LP: 1745648)
    - parser: demote the new lacp-rate check to warning
    - Don't drop files with just global values on 'set' (LP: 2027584)
      d/p/0020-netplan.c-Don-t-drop-files-with-just-global-values-o.patch
    - Add parser support for WPA3 Personal (LP: 2023238)
    - d/patches/lp201...

[Łukasz Zemczak (sil2100)]

I verified with the netplan developers that the package is good to go.