Netplan crashes when attempting to rename interface in 'up' state

[Frank Villaro]

[Impact]
Netplan crashes when trying to apply rules to certain interfaces

[Description]
Starting with version 0.99, netplan introduced a different way of processing link changes that introduced a regression handling interfaces that are still in the 'up' state.
It's possible for interfaces to be marked for link operations (e.g. "set-name") by their MAC address, even if the interface should have been skipped due to being up.

This regression has been fixed upstream by the commit below:
- Fix process_link_changes handling 'up' interfaces (8f77deec17ce)

The fix is not currently present in any Ubuntu release, and the following Netplan 0.99 versions are affected:

$ rmadison netplan.io -s bionic-updates,eoan-updates,focal-updates,groovy
 netplan.io | 0.99-0ubuntu3~18.04.2 | bionic-updates | source, ...
 netplan.io | 0.99-0ubuntu3~19.10.1 | eoan-updates | source, ...
 netplan.io | 0.99-0ubuntu3~20.04.1 | focal-updates | source, ...
 netplan.io | 0.99-0ubuntu3 | groovy | source, ...

[Test Case]
1) Create new dummy interface for testing
# modprobe dummy
# ip link add eth0-test type dummy
# ip link set eth0-test up

2) Add netplan file for new interface with "set-name" directive
# cat /etc/netplan/99-test.yaml
network:
  version: 2
  ethernets:
    eth0-test:
      match:
        macaddress: 56:f6:56:31:2f:ae
      set-name: uplink

3) Attempt to apply changes
# netplan apply
Traceback (most recent call last):
  File "/usr/sbin/netplan", line 23, in <module>
    netplan.main()
  File "/usr/share/netplan/netplan/cli/core.py", line 50, in main
    self.run_command()
  File "/usr/share/netplan/netplan/cli/utils.py", line 186, in run_command
    self.func()
  File "/usr/share/netplan/netplan/cli/commands/apply.py", line 46, in run
    self.run_command()
  File "/usr/share/netplan/netplan/cli/utils.py", line 186, in run_command
    self.func()
  File "/usr/share/netplan/netplan/cli/commands/apply.py", line 172, in command_apply
    stderr=subprocess.DEVNULL)
  File "/usr/lib/python3.6/subprocess.py", line 311, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['ip', 'link', 'set', 'dev', 'eth0-test', 'name', 'uplink']' returned non-zero exit status 2.

[Regression Potential]
The upstream patch restores the behavior present in 0.98 versions, so the regression potential should be low.
Possible issues with this fix could cause netplan to skip interfaces it's not meant to, but the netplan test suite and general smoke testing revealed no major problems.

[Arif Ali (arif-ali)]

This has also appeared in the maas images from the daily sync, and we are no longer able to commission nodes correctly

This is effecting a customer, after they were deploying new nodes into the PCB build

I have reproduced this problem internally, and can say that it's due to an updated netplan.io package from 0.98 to 0.99

From the daily sync of image on 7th May everything works, latest image from 22nd May doesn't. The following pastebin a transcript of a node booted from the 7th May image, and showing netplan working, and then upgrading netplan.io then causes the machine not to apply the netplan configuration

https://pastebin.ubuntu.com/p/6bFSns6WGV/

[Heitor Alves de Siqueira (halves)]

This is indeed a regression, introduced by some changes in the way netplan verifies changes in the interfaces. I've opened a pull request [0] that should fix it, and we can pull that into Ubuntu once it's been discussed upstream.

[0] https://github.com/CanonicalLtd/netplan/pull/143

[Łukasz Zemczak (sil2100)]

Approved and merged in master! Yes, this is indeed a regression introduced by mistake. Thank you for reporting it and providing a fix. I think this feels like good SRU-material, if anything.

[Heitor Alves de Siqueira (halves)]

I'm attaching a debdiff for focal, but it might need some versioning adjustment to be in sync with the other releases. Please let me know if any changes are needed, and I'll refactor it!

[Ubuntu Foundations Team Bug Bot (crichton)]

The attachment "lp1875411-focal.debdiff" seems to be a debdiff. The ubuntu-sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. If the attachment isn't a patch, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are member of the ~ubuntu-sponsors, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issue please contact him.]

[Lukas Märdian (slyon)]

Hello Heitor, the SRU version should read: 0.99-0ubuntu3~20.04.2 – Otherwise, the patch looks good to me!

Maybe a sponsor can quickly adjust that.

[Łukasz Zemczak (sil2100)]

As discussed with Heitor yesterday, I have sponsored the fix for all the supported series (groovy, focal, eoan, bionic), pushing the change to the git repository as well. I renamed the patch a bit since I used gbp for patch addition - but otherwise it's all the same. Version numbers should be good too.

Let's wait for someone from the SRU team to review the package.

[Launchpad Janitor (janitor)]

This bug was fixed in the package netplan.io - 0.99-0ubuntu4

---------------
netplan.io (0.99-0ubuntu4) groovy; urgency=medium

  * d/p/0002-Fix-process_link_changes-handling-up-interfaces.patch:
    - Fix process_link_changes handling 'up' interfaces (LP: #1875411)

 -- Heitor Alves de Siqueira <email address hidden> Thu, 28 May 2020 21:09:51 +0000

[Brian Murray (brian-murray)]

Hello Frank, or anyone else affected,

Accepted netplan.io into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/netplan.io/0.99-0ubuntu3~18.04.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Brian Murray (brian-murray)]

Hello Frank, or anyone else affected,

Accepted netplan.io into eoan-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/netplan.io/0.99-0ubuntu3~19.10.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-eoan to verification-done-eoan. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-eoan. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Brian Murray (brian-murray)]

Hello Frank, or anyone else affected,

Accepted netplan.io into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/netplan.io/0.99-0ubuntu3~20.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted netplan.io (0.99-0ubuntu3~19.10.2) for eoan have finished running.
The following regressions have been reported in tests triggered by the package:

netplan.io/0.99-0ubuntu3~19.10.2 (ppc64el)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/eoan/update_excuses.html#netplan.io

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted netplan.io (0.99-0ubuntu3~18.04.3) for bionic have finished running.
The following regressions have been reported in tests triggered by the package:

netplan.io/0.99-0ubuntu3~18.04.3 (amd64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/bionic/update_excuses.html#netplan.io

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Frank Villaro (f-ran-k)]

Hi Colleagues,

This fixes it for me on focal.

Cheers

[Heitor Alves de Siqueira (halves)]

Thanks for the confirmation, Frank! I was also able to validate the fix on focal, according to the test case from the description:
root@netplan-focal:~# dpkg -l | grep netplan.io
ii netplan.io 0.99-0ubuntu3~20.04.2 amd64 YAML network configuration abstraction for various backends
root@netplan-focal:~# netplan apply

root@netplan-focal:~# ip l show uplink
4: uplink: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether 96:69:08:01:75:be brd ff:ff:ff:ff:ff:ff

[Heitor Alves de Siqueira (halves)]

Validated on Eoan according to the test case from the description:

root@netplan-eoan:~# dpkg -l | grep netplan.io
ii netplan.io 0.99-0ubuntu3~19.10.2 amd64 YAML network configuration abstraction for various backends
root@netplan-eoan:~# netplan apply
root@netplan-eoan:~# ip l show eth0-test
4: eth0-test: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether 7e:a3:1e:53:50:71 brd ff:ff:ff:ff:ff:ff

[Heitor Alves de Siqueira (halves)]

Validated on Bionic according to the test case from the description:

root@netplan-bionic:~# dpkg -l | grep netplan.io
ii netplan.io 0.99-0ubuntu3~18.04.3 amd64 YAML network configuration abstraction for various backends
root@netplan-bionic:~# netplan apply
root@netplan-bionic:~# ip l show eth0-test
3: eth0-test: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether 52:c0:d4:89:44:57 brd ff:ff:ff:ff:ff:ff

[Launchpad Janitor (janitor)]

This bug was fixed in the package netplan.io - 0.99-0ubuntu3~20.04.2

---------------
netplan.io (0.99-0ubuntu3~20.04.2) focal; urgency=medium

  * d/p/0002-Fix-process_link_changes-handling-up-interfaces.patch:
    - Fix process_link_changes handling 'up' interfaces (LP: #1875411)

 -- Heitor Alves de Siqueira <email address hidden> Thu, 28 May 2020 21:09:51 +0000

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for netplan.io has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package netplan.io - 0.99-0ubuntu3~19.10.2

---------------
netplan.io (0.99-0ubuntu3~19.10.2) eoan; urgency=medium

  * d/p/0002-Fix-process_link_changes-handling-up-interfaces.patch:
    - Fix process_link_changes handling 'up' interfaces (LP: #1875411)

 -- Heitor Alves de Siqueira <email address hidden> Thu, 28 May 2020 21:09:51 +0000

[Launchpad Janitor (janitor)]

This bug was fixed in the package netplan.io - 0.99-0ubuntu3~18.04.3

---------------
netplan.io (0.99-0ubuntu3~18.04.3) bionic; urgency=medium

  * d/p/0004-Fix-process_link_changes-handling-up-interfaces.patch:
    - Fix process_link_changes handling 'up' interfaces (LP: #1875411)

 -- Heitor Alves de Siqueira <email address hidden> Thu, 28 May 2020 21:09:51 +0000