Update 0.105-0ubuntu2~22.04.1 seems to cause regressions with snapd integration

[Michael Vogt]

[Impact]
This release contains an important bug-fix, related to the "netplan set" subcommand,
and we would like to make sure all of our supported customers have access to this improvement.

The notable changes in the package are:

    * d/p/lp1997467: set only specific origin-hint if given (LP: #1997467)
    * d/libnetplan0.symbols: Add netplan_parser_load_nullable_overrides() API

The issues addressed by this SRU are described below.

Some regressions were introduced on netplan.io 0.105 in the subcommand "netplan set".
Users who rely on "netplan set" as part of their workflows, such as snapd, were affected
by these issues. Snapd in particular was forced to rollback to netplan.io 0.104 to unblock
their customers.

Issue 1) Deletion of the origin-hint file on update

    netplan set --origin-hint test "bridges.br54.dhcp4=false"

    The command above will create the file /etc/netplan/test.yaml
    The command below should update the file, but it will be deleted instead

    netplan set --origin-hint test "bridges.br54.dhcp4=true"

Issue 2) Updates being ignored when --origin-hint is used

    Updates on interfaces already defined in another file are ignored.
    The command below should create a new file (in this case called 90-snapd-config.yaml)
    containing the definition specified by its last parameter:

    netplan set --origin-hint 90-snapd-config bridges.br54.dhcp4=false

    Although, if the interface br54 is already defined in another file, netplan set will not
    create the file 90-snapd-config.yaml.

[Test Plan]
The following development and SRU process was followed:
https://wiki.ubuntu.com/NetplanUpdates

Netplan contains an extensive integration test suite that is ran using
the SRU package for each releases. This test suite's results are available here:
http://autopkgtest.ubuntu.com/packages/n/netplan.io

* Steps to reproduce the issue

This bug is easily reproducible on netplan.io 0.105-0ubuntu2, which is used by Kinetic and Jammy.

1) Launch a LXD Jammy instance (either VM or container):

lxc launch ubuntu:22.04 jammy-netplan

2) Access the new instance

lxc exec jammy-netplan bash

3) (Optional) Upgrade your packages

apt update && apt -y upgrade

4) Check the netplan.io version you have

dpkg --list | grep netplan.io

You should see "0.105-0ubuntu2", the version with this bug.

5) Create the file /etc/netplan/0-snapd-defaults.yaml

cat <<EOF > /etc/netplan/0-snapd-defaults.yaml
network:
  version: 2
  ethernets:
    ens3:
      addresses:
      - "10.0.2.15/24"
      dhcp4: false
      routes:
      - to: "default"
        via: "10.0.2.2"
  bridges:
    br54:
      dhcp4: true
EOF

6) Use "netplan set" to create a new file that will be used to change the bridge configuration defined in the previous file

netplan set --origin-hint 90-snapd-config network.bridges.br54.dhcp4=false

7) Checking the results

After running "netplan set" you should find a new file at /etc/netplan called 90-snapd-config.yaml
containing only the snippet "network.bridges.br54.dhcp4=false" set with netplan set. It looks like this:

# cat /etc/netplan/90-snapd-config.yaml
network:
  version: 2
  bridges:
    br54:
      dhcp4: false

If you are running the netplan.io version affected by this issue, you will not find the new file.
That is the main problem this SRU addresses.

8) Now add the PPA below which contains a netplan.io package with patches for this issue

sudo add-apt-repository ppa:danilogondolfo/netplan.io

apt update
apt upgrade

9) Try netplan set again

netplan set --origin-hint 90-snapd-config network.bridges.br54.dhcp4=false

After running "netplan set" the file /etc/netplan/90-snapd-config.yaml will be generated.

A successful run is required before the proposed netplan package
can be let into -updates.

The netplan team will be in charge of attaching the artifacts and console
output of the appropriate run to the bug. Netplan team members will not
mark ‘verification-done’ until this has happened.

[Where problems could occur]

As this patch changes a small portion of code used by the netplan generator (and the "netplan generate" command),
an issue introduced by this change would potentially affect the network configuration generated when the system boots up.
In the worst scenario it would end up removing old configuration and not generating new one or missing some parts of it
preventing the system to have a working network connection.

The new package containing the fixes is available in Lunar for many days now and no issues like the aforementioned
ones were reported.

In order to mitigate the regression potential, the results of the
aforementioned integration tests are attached to this bug.

[Other Info]

There are remaining known bugs on "netplan set" that were detected recently but we confirmed they
are not regressions caused by this patch as they were already present in netplan. They are all
documented in LP: #2003727.

The issues addressed by this SRUs are the only ones causing real impact to customers based on LP bug reports.

[Changelog]

Bug fixes:
- d/p/lp1997467: set only specific origin-hint if given (LP: #1997467)
Cherry-picked from upstream: https://github.com/canonical/netplan/pull/299
- d/libnetplan0.symbols: Add netplan_parser_load_nullable_overrides() API

==== Original report ====

Sorry that this is a bit unspecific right now, I will try to provide more details ASAP.

We see failures in our nested core20-early-config test, e.g.
https://github.com/snapcore/snapd/actions/runs/3512986269/jobs/5885568960

The test fails in the following place:
"""
...
+ remote.exec 'sudo snap set system system.network.netplan.network.bridges.br54.dhcp4=false'
+ MATCH false
+ remote.exec sudo 'netplan get bridges.br54.dhcp4'
grep error: pattern not found, got:
true
"""
at this point the files look like this (sudo is now needed which was not before btw):
"""
$ sudo cat /etc/netplan/0-snapd-defaults.yaml
network:
  version: 2
  ethernets:
    ens3:
      addresses:
      - "10.0.2.15/24"
      dhcp4: false
      routes:
      - to: "default"
        via: "10.0.2.2"
  bridges:
    br54:
      dhcp4: true

$ sudo cat /etc/netplan/50-cloud-init.yaml
# This file is generated from information provided by the datasource. Changes
# to it will not persist across an instance reboot. To disable cloud-init's
# network configuration capabilities, write a file
# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
# network: {config: disabled}
network:
    ethernets:
        enp0s2:
            dhcp4: true
            match:
                macaddress: '52:54:00:12:34:56'
            set-name: enp0s2
    version: 2
"""
When I manually run

   $ sudo snap set system system.network.netplan.network.bridges.br54.dhcp4=false

inside the VM I see no error but the file 0-snapd-defaults.yaml is unchanged and no
new file 90-snapd-config is created. The logic inside snapd is that the gadget defaults
are stored as 0-snapd-defaults and later changes are stored in 90-sanpd-config so that
the later config values win over the earlier ones.

Fwiw, this test is also run on a core20 system with netplan 104 and there the same test works succesfully afaict.

[Michael Vogt (mvo)]

As a reproducer on 22.04:
# cat > /etc/netplan/0-snapd-defaults.yaml <<EOF
network:
  version: 2
  bridges:
    br54:
      dhcp4: true
EOF
# busctl call io.netplan.Netplan /io/netplan/Netplan io.netplan.Netplan Config
# busctl call io.netplan.Netplan /io/netplan/Netplan/config/OY6LV1 io.netplan.Netplan.Config Set ss "network.bridges.br54.dhcp4=false" "90-snapd-config"
# busctl call io.netplan.Netplan /io/netplan/Netplan/config/OY6LV1 io.netplan.Netplan.Config Apply
# netplan get
network:
  version: 2
  renderer: NetworkManager
  bridges:
    br54:
      dhcp4: true

[Michael Vogt (mvo)]

Fwiw, runing the same in a 20.04 xenial lxc container:
"""
...
# root@focal:~# netplan get
network:
  version: 2
  ethernets:
    eth0:
      dhcp4: true
  bridges:
    br54:
      dhcp4: false
root@focal:~# apt list netplan.io
Listing... Done
netplan.io/focal-updates,now 0.104-0ubuntu2~20.04.2 amd64 [installed]
"""

[Lukas Märdian (slyon)]

I can confirm this is a regression and happens with an even simpler reproducer like this:

# cat > /etc/netplan/0-snapd-defaults.yaml <<EOF
network:
  version: 2
  bridges:
    br54:
      dhcp4: true
EOF
# netplan set --origin-hint 90-snapd-config "network.bridges.br54.dhcp4=false"

Seems to happen whenever the setting is about to be overridden (i.e. already exists in another file, like 0-snapd-defaults.yaml here).

Most probably related to switching the backend for `netplan set` to libnetplan: https://github.com/canonical/netplan/pull/254

Needs further debugging.

[Lukas Märdian (slyon)]

Here's a test case reproducer, which can be put into netplan's test_cli_get_set.py suite:

    def test_set_origin_hint_override(self):
        defaults = os.path.join(self.workdir.name, 'etc', 'netplan', '0-snapd-defaults.yaml')
        with open(defaults, 'w') as f:
            f.write('''network: {bridges: {br54: {dhcp4: true}}}''')
        self._set(['bridges.br54.dhcp4=false', '--origin-hint=90-snapd-config'])
        p = os.path.join(self.workdir.name, 'etc', 'netplan', '90-snapd-config.yaml')
        self.assertTrue(os.path.isfile(p))
        with open(p, 'r') as f:
            self.assertIs(False, yaml.safe_load(f)['network']['bridges']['br54']['dhcp4'])

[Lukas Märdian (slyon)]

I've come up with an initial draft for a patch.

Not sure if this is the full solution already, but at least it fixes the above mentioned test case.

@mvo This is all inside the python code. could you try to modify that file inside your test environment and see if that fixes the snapd tests?

[Lukas Märdian (slyon)]

I've prepared something here: https://github.com/canonical/netplan/pull/299

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Lukas Märdian (slyon)]

I feel like I'm pretty close to a solution now. Although, my work from PR#299 is not yet fully reviewed and merged, I prepared a fix for this in the following PPA: https://launchpad.net/~slyon/+archive/ubuntu/netplan-set-lp1997467

It could be copied to the UC22 build PPA, if needed, while we work on refining and merging the upstream PR and making it fit for SRU.

Alternatively, we could revert the switch to the libnetplan YAML parser based "netplan set" implementation for the time being (basically everything from https://github.com/canonical/netplan/pull/254 -> netplan/cli/commands/set.py)

[Michael Vogt (mvo)]

Thanks for working on the fix! Sadly I had not yet managed to find time to build a core22 that uses the PPA to test it.

However sadly this is hitting customers now and because of this I uploaded a revert of netplan to the 0.104-0ubuntu2.1 into ppa:snappy-dev/image to unblock our customers.

[Lukas Märdian (slyon)]

With the most recent changes from https://github.com/canonical/netplan/pull/299 this seems to be fixed. Both reproducers pass for me, using a PPA build including the PR#299 commits: https://launchpad.net/~slyon/+archive/ubuntu/netplan-set-lp1997467

[Launchpad Janitor (janitor)]

This bug was fixed in the package netplan.io - 0.105-0ubuntu4

---------------
netplan.io (0.105-0ubuntu4) lunar; urgency=medium

  * d/p/lp1997467: set only specific origin-hint if given (LP: #1997467)
  * d/libnetplan0.symbols: Add netplan_parser_load_nullable_overrides() API

 -- Lukas Märdian <email address hidden> Thu, 26 Jan 2023 14:34:37 +0100

[Lukas Märdian (slyon)]

A quick review on the debdiffs:
* SRU version should be "0.105-0ubuntu2.1" / "0.105-0ubuntu2~22.04.2" (see https://wiki.ubuntu.com/SecurityTeam/UpdatePreparation#Update_the_packaging)
* improve d/changelog body formatting
* Adopt .symbols version for new API function according to the SRU version
* We might want to cherry-pick/backport https://github.com/canonical/netplan/commit/102f189e8f3a1828a3beddf686dcc96c15a3b9b5 too, as this is a fixup for PR#299

Those are all small things and it lgtm otherwise.
Overall, I prefer git Merge-Proposals over debdiffs, but that's a personal preference :)

[Lukas Märdian (slyon)]

Thanks, merged and sponsored into kinetic and jammy!

[Łukasz Zemczak (sil2100)]

Hello Michael, or anyone else affected,

Accepted netplan.io into kinetic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/netplan.io/0.105-0ubuntu2.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-kinetic to verification-done-kinetic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-kinetic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Łukasz Zemczak (sil2100)]

Hello Michael, or anyone else affected,

Accepted netplan.io into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/netplan.io/0.105-0ubuntu2~22.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted netplan.io (0.105-0ubuntu2.1) for kinetic have finished running.
The following regressions have been reported in tests triggered by the package:

netplan.io/0.105-0ubuntu2.1 (amd64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/kinetic/update_excuses.html#netplan.io

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Danilo Egea Gondolfo (danilogondolfo)]

hm the test from the regression above passed for all the other architectures and it passes locally. Probably some transient issue that a retry will fix.

[Lukas Märdian (slyon)]

The kinetic/amd64 autopkgtest regression was fixed by re-try.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted netplan.io (0.105-0ubuntu2~22.04.2) for jammy have finished running.
The following regressions have been reported in tests triggered by the package:

initramfs-tools/0.140ubuntu13.1 (amd64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/jammy/update_excuses.html#netplan.io

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Danilo Egea Gondolfo (danilogondolfo)]

initramfs-tools failing only for amd64, it seems to be related with qemu timeouts. It will probably work after retrying it.

[Danilo Egea Gondolfo (danilogondolfo)]

So, it seems like we can't get the initramfs-tools tests to pass.

But it seems to be the normal: https://autopkgtest.ubuntu.com/packages/initramfs-tools/jammy/amd64
it fails multiple times with the same trigger but then eventually it works.

I tried it locally a bunch of times (using netplan.io from jammy-proposed) and it never failed.

[Lukas Märdian (slyon)]

ACK. I feel like this should be hinted in https://git.launchpad.net/~ubuntu-release/britney/+git/hints-ubuntu

[Danilo Egea Gondolfo (danilogondolfo)]

The last retry triggered with the new netplan passed all the tests: https://autopkgtest.ubuntu.com/packages/initramfs-tools/jammy/amd64

[Michael Vogt (mvo)]

I am trying to test this on core22 with:
"""
$ git diff
diff --git a/tests/nested/manual/core20-early-config/task.yaml b/tests/nested/manual/core20-early-config/task.yaml
index 797232fa26..fbb92d4240 100644
--- a/tests/nested/manual/core20-early-config/task.yaml
+++ b/tests/nested/manual/core20-early-config/task.yaml
@@ -28,6 +28,17 @@ prepare: |

     rm -f "$SNAKEOIL_KEY" "$SNAKEOIL_CERT"

+ snap download --basename=core22 core22
+ unsquashfs -d core22 core22.snap
+ wget https://launchpad.net/ubuntu/+source/netplan.io/0.105-0ubuntu2~22.04.2/+build/25592898/+files/libnetplan0_0.105-0ubuntu2~22.04.2_amd64.deb
+ dpkg-deb -x libnetplan0_0.105-0ubuntu2~22.04.2_amd64.deb ./core22
+ wget https://launchpad.net/ubuntu/+source/netplan.io/0.105-0ubuntu2~22.04.2/+build/25592898/+files/netplan.io_0.105-0ubuntu2~22.04.2_amd64.deb
+ dpkg-deb -x netplan.io_0.105-0ubuntu2~22.04.2_amd64.deb ./netplan
+ # netplan ships stuff in /lib instead of /usr/lib, fix this
+ mv netplan/lib/ netplan/usr/
+ rsync -a ./netplan/* ./core22
+ snap pack ./core22 "$(tests.nested get extra-snaps-path)"
+
     # Note that the early config is configured to set port to 8023 instead
     # of the default 8022 (which maps to 22 on the VM).
     remote.setup config --host localhost --port 8023 --user user1 --pass ubuntu

$ spread -debug -v qemu-nested:ubuntu-22.04-64:tests/nested/manual/core20-early-config
"""

With that the "core20-early-config" test that was also used to find the original issue fails in strange ways. The netplan config itself looks fine:
"""
# remote.exec "sudo netplan get "
network:
  version: 2
  ethernets:
    ens3:
      addresses:
      - "10.0.2.15/24"
      dhcp4: false
      routes:
      - to: "default"
        via: "10.0.2.2"
  bridges:
    br54:
      dhcp4: true
"""

But netplan seems to be no longer able to give dbus results:
"""
qemu-nested:ubuntu-22.04-64 .../tests/nested/manual/core20-early-config# remote.exec 'sudo dbus-send --system --print-reply --type=method_call --dest=io.netplan.Netplan /io/netplan/Netplan io.netplan.Netplan.Config'
method return time=1677251025.338293 sender=:1.27 -> destination=:1.232 serial=65 reply_serial=2
   object path "/io/netplan/Netplan/config/Q6S201"
qemu-nested:ubuntu-22.04-64 .../tests/nested/manual/core20-early-config# remote.exec 'sudo dbus-send --system --print-reply --type=method_call --dest=io.netplan.Netplan /io/netplan/Netplan/config/Q6S201 io.netplan.Netplan.Config.Get'
method return time=1677251055.826055 sender=:1.27 -> destination=:1.237 serial=68 reply_serial=2
   string ""
"""
and the same in snapd (which uses the dbus API).

I also tried to reproduce this on my desktop system and see similar results, i.e. the dbus call for a config dbus object is now giving me an empty string on my desktop. So maybe more than just a test setup issue?

[Michael Vogt (mvo)]

Fwiw, on my 22.10 machine I get something like "'network:\n version: 2\n renderer: NetworkManager\n'" from the "Get()" call.

[Michael Vogt (mvo)]

Just for reference, my test commit that exercises the new netplan is under https://github.com/snapcore/snapd/compare/master...mvo5:snappy:test-netplan-sru-1997467?expand=1

[Danilo Egea Gondolfo (danilogondolfo)]

I noticed the error below when trying to use netplan-dbus:

sudo dbus-send --system --print-reply --type=method_call --dest=io.netplan.Netplan /io/netplan/Netplan io.netplan.Netplan.Config

This is from dbus-monitor --system:

string "Failed to copy file /etc/netplan/01-network-manager-all.yaml -> /run/netplan/config-1YXM01etc/netplan/01-network-manager-all.yaml: Error opening file “/run/netplan/config-1YXM01etc/netplan/01-network-manager-all.yaml”: No such file or directory

Apparently the path is malformed: /run/netplan/config-1YXM01etc/netplan/01-network-manager-all.yaml

[Danilo Egea Gondolfo (danilogondolfo)]

This will fix the dbus issue https://github.com/canonical/netplan/pull/331

$ sudo dbus-send --system --print-reply --type=method_call --dest=io.netplan.Netplan /io/netplan/Netplan io.netplan.Netplan.Config
method return time=1677605377.300772 sender=:1.9462 -> destination=:1.9463 serial=4 reply_serial=2
   object path "/io/netplan/Netplan/config/WM6X01"

$ sudo dbus-send --system --print-reply --type=method_call --dest=io.netplan.Netplan /io/netplan/Netplan/config/WM6X01 io.netplan.Netplan.Config.Get
method return time=1677605387.117285 sender=:1.9462 -> destination=:1.9464 serial=6 reply_serial=2
   string "network:
  version: 2
  renderer: NetworkManager
  bridges:
    br123:
      dhcp4: false
"

I suppose we need to include it in the SRU as well.

[Danilo Egea Gondolfo (danilogondolfo)]

I've created two new merge requests including patches for the dbus issue

Kinetic: https://code.launchpad.net/~danilogondolfo/netplan/+git/ubuntu/+merge/438307

Jammy: https://code.launchpad.net/~danilogondolfo/netplan/+git/ubuntu/+merge/438310

A PPA with packages for both are available here https://launchpad.net/~danilogondolfo/+archive/ubuntu/netplan.io

netplan.io in lunar-proposed already has the fix.

[Łukasz Zemczak (sil2100)]

Hello Michael, or anyone else affected,

Accepted netplan.io into kinetic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/netplan.io/0.105-0ubuntu2.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-kinetic to verification-done-kinetic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-kinetic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Łukasz Zemczak (sil2100)]

Hello Michael, or anyone else affected,

Accepted netplan.io into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/netplan.io/0.105-0ubuntu2~22.04.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted netplan.io (0.105-0ubuntu2~22.04.3) for jammy have finished running.
The following regressions have been reported in tests triggered by the package:

initramfs-tools/0.140ubuntu13.1 (amd64)
netplan.io/0.105-0ubuntu2~22.04.3 (amd64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/jammy/update_excuses.html#netplan.io

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Michael Vogt (mvo)]

Thanks for the updated version.

I tested it with https://github.com/snapcore/snapd/compare/master...mvo5:snappy:test-netplan-sru-1997467?expand=1 (updated to test exactly this version) and with that our spread test works:
"""
$ spread -debug -v qemu-nested:ubuntu-22.04-64:tests/nested/manual/core20-early-config
...
2023-03-09 12:16:57 Executing qemu-nested:ubuntu-22.04-64:tests/nested/manual/core20-early-config (qemu-nested:ubuntu-22.04-64) (1/1)...
2023-03-09 12:18:11 Restoring qemu-nested:ubuntu-22.04-64:tests/nested/manual/core20-early-config (qemu-nested:ubuntu-22.04-64)...
2023-03-09 12:18:29 Restoring qemu-nested:ubuntu-22.04-64:tests/nested/manual/ (qemu-nested:ubuntu-22.04-64)...
2023-03-09 12:18:31 Restoring qemu-nested:ubuntu-22.04-64 (qemu-nested:ubuntu-22.04-64)...
2023-03-09 12:18:32 Discarding qemu-nested:ubuntu-22.04-64...
2023-03-09 12:18:32 Successful tasks: 1
2023-03-09 12:18:32 Aborted tasks: 0
"""
so +1 from me.

[Danilo Egea Gondolfo (danilogondolfo)]

Thanks, Michael!

So I believe we can proceed with the SRU.

I did another batch of tests today. The results are described below:

The steps below were used to validate the fix on Jammy and Kinetic:

1) Launch a LXD Jammy instance and access it

lxc launch ubuntu:22.04 jammy-netplan
lxc shell jammy-netplan

2) Create the base config

cat <<EOF > /etc/netplan/0-snapd-defaults.yaml
network:
  version: 2
  ethernets:
    ens3:
      addresses:
      - "10.0.2.15/24"
      dhcp4: false
      routes:
      - to: "default"
        via: "10.0.2.2"
  bridges:
    br54:
      dhcp4: true
EOF

3) Try to change it with netplan set

netplan set --origin-hint 90-snapd-config network.bridges.br54.dhcp4=false

4) Check the the new file WASN'T created

cat /etc/netplan/90-snapd-config.yaml

output: cat: /etc/netplan/90-snapd-config.yaml: No such file or directory

5) Check that the DBus interface IS working properly:

busctl call io.netplan.Netplan /io/netplan/Netplan io.netplan.Netplan Config

output: o "/io/netplan/Netplan/config/88ZI11" <<---- USE THIS PATH IN THE COMMANDS BELOW

busctl call io.netplan.Netplan /io/netplan/Netplan/config/88ZI11 io.netplan.Netplan.Config Set ss "network.bridges.br54.dhcp4=false" "90-snapd-config"

output: b true

busctl call io.netplan.Netplan /io/netplan/Netplan/config/88ZI11 io.netplan.Netplan.Config Get

output: s "network:\n version: 2\n ethernets:\n ens3:\n addresses:\n - \"10.0.2.15/24\"\n dhcp4: false\n routes:\n - to: \"default\"\n via: \"10.0.2.2\"\n eth0:\n dhcp4: true\n bridges:\n br54:\n dhcp4: true\n"

6) Enable -proposed and update your netplan.io packages

echo "deb http://archive.ubuntu.com/ubuntu jammy-proposed main" >> /etc/apt/sources.list.d/proposed.list
apt update && apt -y upgrade

# kill netplan-dbus to make sure the new one will run when you try busctl again

killall netplan-dbus

7) Use netplan set again to change the configuration created in the second step

netplan set --origin-hint 90-snapd-config network.bridges.br54.dhcp4=false

8) Check the the new file WAS created this time

cat /etc/netplan/90-snapd-config.yaml

network:
  version: 2
  bridges:
    br54:
      dhcp4: false

9) Check that the DBus interface IS still working properly:

busctl call io.netplan.Netplan /io/netplan/Netplan io.netplan.Netplan Config

output: o "/io/netplan/Netplan/config/PMTC11" <<---- USE THIS PATH IN THE COMMANDS BELOW

busctl call io.netplan.Netplan /io/netplan/Netplan/config/PMTC11 io.netplan.Netplan.Config Set ss "network.bridges.br54.dhcp4=false" "90-snapd-config"

output: b true

busctl call io.netplan.Netplan /io/netplan/Netplan/config/PMTC11 io.netplan.Netplan.Config Get

output: s "network:\n version: 2\n ethernets:\n ens3:\n addresses:\n - \"10.0.2.15/24\"\n dhcp4: false\n routes:\n - to: \"default\"\n via: \"10.0.2.2\"\n eth0:\n dhcp4: true\n bridges:\n br54:\n dhcp4: false\n"

---------

Validation on Kinetic

The same steps can be performed to valid netplan.io on kinetic

1) Launch a LXD Kinetic instance and access it

lxc launch ubuntu:22.10 kinetic-netplan
lxc shell kinetic-net...

[Launchpad Janitor (janitor)]

This bug was fixed in the package netplan.io - 0.105-0ubuntu2.2

---------------
netplan.io (0.105-0ubuntu2.2) kinetic; urgency=medium

  * Fix and improvements for the DBus integration (LP: #1997467)
    Cherry-picked from upstream: https://github.com/canonical/netplan/pull/331
    - d/p/lp1997467/0009-dbus-Build-the-copy-path-correctly.patch
      Properly build the destination path before copying files in the dbus
      integration and improve error handling
    - d/p/lp1997467/0010-tests-Add-an-integration-test-for-netplan-dbus.patch
      Add an integration test to exercise the code path where the issue was
      addressed.

netplan.io (0.105-0ubuntu2.1) kinetic; urgency=medium

  * d/p/lp1997467: set only specific origin-hint if given (LP: #1997467)
    Cherry-picked from upstream: https://github.com/canonical/netplan/pull/299
    - d/libnetplan0.symbols: Add netplan_parser_load_nullable_overrides() API
    - d/p/0008-src-parse-plug-memory-leaks-in-nullable-handling.patch backport
      upstream commit 40c53bb (memory leak fixup of PR#299)

 -- Danilo Egea Gondolfo <email address hidden> Fri, 03 Mar 2023 12:35:15 +0000

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for netplan.io has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package netplan.io - 0.105-0ubuntu2~22.04.3

---------------
netplan.io (0.105-0ubuntu2~22.04.3) jammy; urgency=medium

  * Fix and improvements for the DBus integration (LP: #1997467)
    Cherry-picked from upstream: https://github.com/canonical/netplan/pull/331
    - d/p/lp1997467/0009-dbus-Build-the-copy-path-correctly.patch
      Properly build the destination path before copying files in the dbus
      integration and improve error handling
    - d/p/lp1997467/0010-tests-Add-an-integration-test-for-netplan-dbus.patch
      Add an integration test to exercise the code path where the issue was
      addressed.

netplan.io (0.105-0ubuntu2~22.04.2) jammy; urgency=medium

  * d/p/lp1997467: set only specific origin-hint if given (LP: #1997467)
    Cherry-picked from upstream: https://github.com/canonical/netplan/pull/299
    - d/libnetplan0.symbols: Add netplan_parser_load_nullable_overrides() API
    - d/p/0008-src-parse-plug-memory-leaks-in-nullable-handling.patch backport
      upstream commit 40c53bb (memory leak fixup of PR#299)

 -- Danilo Egea Gondolfo <email address hidden> Fri, 03 Mar 2023 13:14:22 +0000