Activity log for bug #1979951

Date Who What changed Old value New value Message
2022-06-27 06:49:04 Trent Lloyd bug added bug
2022-06-28 15:39:51 Lukas Märdian bug task added systemd (Ubuntu)
2022-06-28 15:46:47 Lukas Märdian tags sts rls-ff-incoming rls-jj-incoming sts
2022-06-30 15:25:27 Matthieu Clemenceau tags rls-ff-incoming rls-jj-incoming sts fr-2514 rls-ff-incoming rls-jj-incoming sts
2022-06-30 15:26:15 Lukas Märdian nominated for series Ubuntu Jammy
2022-06-30 15:26:15 Lukas Märdian bug task added systemd (Ubuntu Jammy)
2022-06-30 15:26:15 Lukas Märdian bug task added netplan.io (Ubuntu Jammy)
2022-06-30 15:26:15 Lukas Märdian nominated for series Ubuntu Focal
2022-06-30 15:26:15 Lukas Märdian bug task added systemd (Ubuntu Focal)
2022-06-30 15:26:15 Lukas Märdian bug task added netplan.io (Ubuntu Focal)
2022-06-30 15:26:44 Lukas Märdian tags fr-2514 rls-ff-incoming rls-jj-incoming sts fr-2514 sts
2022-06-30 15:27:34 Lukas Märdian systemd (Ubuntu): status New Fix Released
2022-07-21 19:23:57 Nick Rosbrook description If you configure the loopback interface using netplan without listing the standard loopback addresses (127.0.0.1 / ::1) that are auto-configured on boot then they are removed by netplan in r104+ but were not in r103. Confusingly this only happens the second time "netplan apply" is invoked and not the first time. There is some argument this is not a valid configuration, however - It worked on r103 and broke in the r104 SRU for 20.04 and breaks on upgrade - Since it only occurs on the second "netplan apply" it is likely people will accidentally create a configuration that on first verification works but breaks later - Adding addresses to a loopback or dummy interface is a common need - There is no dummy interface support in Netplan - Removing the loopback IP doesn't immediately cause obvious breakage in an SSH session but causes all sorts of hard to diagnose issues with applications Thus I consider this a high priority regression because of the multiple ways this can break a system at an unexpected/unrelated time to when the original change was made. As a workaround you can add 127.0.0.1/::1 to the file and this seems to generally work and product almost the same configuration with the exception that "brd 127.255.255.255" is added to the "ip addr" output compared to the auto-created configuration. All of the different routing tables seem otherwise the tame. = Test Case = (1) Add IP configuration for lo to /etc/netplan/lo.yaml lo: match: name: lo addresses: - 10.10.10.17/24 - 10.10.10.19/24 (2) Run "netplan apply" and observe the "ip addr show dev lo" output. Expected: 127.0.0.1/::1 are still there. (3) Run "netplan apply" a second time and observe the "ip addr show dev lo" output. Expected: 127.0.0.1/::1 are now removed. (4) Repeat the same test under netplan r103. Expected: 127.0.0.1/::1 are still there in both cases. = Observations = I found this change of behaviour happens only in r104. It is suspected but not clear that this is due to the configuraiton diffing behaviour introduced in r104. It's not clear to me why it doesn't happen on the first "netplan apply". I also found a recent upstream systemd commit to prevent networkd removing 127.0.0.1/::1 which solves this issue on r104 as well. It's possible this is a better fix to backport to solve this: https://github.com/systemd/systemd/commit/b69bfa43052586cd6894c32d7d3d447195539ec5 Reverting to r103 *or* running a newer systemd with that patch resolves the issue on 20.04 and 22.04 in my testing. [Impact] Users who wish to configure additional addresses on lo may inadvertently remove the 127.0.0.1/8 address from the interface. And, because this only happens on the second invocation of `netplan apply`, it can create confusing breakages for users. Although this is not a recommended configuration, it was previously supported and the current behavior is a regression. [Test Plan] * Configure additional addresses on the lo interface. I am testing in a LXD container, so I added the following to /etc/netplan/10-lxc.yaml under `ethernets:` lo: addresses: ["10.10.10.17/24","10.10.10.19/24"] * Apply the config, and then inspect the lo interface's addresses: $ netplan apply $ ip addr show dev lo * Observe that the 127.0.0.1/8 address is still present. Now, run the same commands again: $ netplan apply $ ip addr show dev lo * On an affected system, observe that the 127.0.0.1/8 address is now gone. On a patched system, the address should still be present. [Where problems could occur] The patch removes cases where systemd-networkd will drop foreign addresses from a managed link. If problems were to occur, it would be related to the addresses configured on a managed link. [Original Description] If you configure the loopback interface using netplan without listing the standard loopback addresses (127.0.0.1 / ::1) that are auto-configured on boot then they are removed by netplan in r104+ but were not in r103. Confusingly this only happens the second time "netplan apply" is invoked and not the first time. There is some argument this is not a valid configuration, however - It worked on r103 and broke in the r104 SRU for 20.04 and breaks on upgrade - Since it only occurs on the second "netplan apply" it is likely people will accidentally create a configuration that on first verification works but breaks later - Adding addresses to a loopback or dummy interface is a common need - There is no dummy interface support in Netplan - Removing the loopback IP doesn't immediately cause obvious breakage in an SSH session but causes all sorts of hard to diagnose issues with applications Thus I consider this a high priority regression because of the multiple ways this can break a system at an unexpected/unrelated time to when the original change was made. As a workaround you can add 127.0.0.1/::1 to the file and this seems to generally work and product almost the same configuration with the exception that "brd 127.255.255.255" is added to the "ip addr" output compared to the auto-created configuration. All of the different routing tables seem otherwise the tame. = Test Case = (1) Add IP configuration for lo to /etc/netplan/lo.yaml lo:   match:     name: lo   addresses:   - 10.10.10.17/24   - 10.10.10.19/24 (2) Run "netplan apply" and observe the "ip addr show dev lo" output. Expected: 127.0.0.1/::1 are still there. (3) Run "netplan apply" a second time and observe the "ip addr show dev lo" output. Expected: 127.0.0.1/::1 are now removed. (4) Repeat the same test under netplan r103. Expected: 127.0.0.1/::1 are still there in both cases. = Observations = I found this change of behaviour happens only in r104. It is suspected but not clear that this is due to the configuraiton diffing behaviour introduced in r104. It's not clear to me why it doesn't happen on the first "netplan apply". I also found a recent upstream systemd commit to prevent networkd removing 127.0.0.1/::1 which solves this issue on r104 as well. It's possible this is a better fix to backport to solve this: https://github.com/systemd/systemd/commit/b69bfa43052586cd6894c32d7d3d447195539ec5 Reverting to r103 *or* running a newer systemd with that patch resolves the issue on 20.04 and 22.04 in my testing.
2022-07-22 19:29:09 Launchpad Janitor merge proposal linked https://code.launchpad.net/~enr0n/ubuntu/+source/systemd/+git/systemd/+merge/427303
2022-07-22 19:31:51 Launchpad Janitor merge proposal linked https://code.launchpad.net/~enr0n/ubuntu/+source/systemd/+git/systemd/+merge/427304
2022-09-05 08:57:03 Łukasz Zemczak systemd (Ubuntu Jammy): status New Fix Committed
2022-09-05 08:57:04 Łukasz Zemczak bug added subscriber Ubuntu Stable Release Updates Team
2022-09-05 08:57:06 Łukasz Zemczak bug added subscriber SRU Verification
2022-09-05 08:57:08 Łukasz Zemczak tags fr-2514 sts fr-2514 sts verification-needed verification-needed-jammy
2022-09-05 13:11:02 Łukasz Zemczak systemd (Ubuntu Focal): status New Fix Committed
2022-09-05 13:11:08 Łukasz Zemczak tags fr-2514 sts verification-needed verification-needed-jammy fr-2514 sts verification-needed verification-needed-focal verification-needed-jammy
2022-09-06 10:20:53 Launchpad Janitor merge proposal linked https://code.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/+git/systemd/+merge/429490
2022-09-07 20:55:02 Nick Rosbrook tags fr-2514 sts verification-needed verification-needed-focal verification-needed-jammy fr-2514 sts verification-done-focal verification-needed verification-needed-jammy
2022-09-12 00:08:15 Matthieu Clemenceau tags fr-2514 sts verification-done-focal verification-needed verification-needed-jammy foundations-todo fr-2514 sts verification-done-focal verification-needed verification-needed-jammy
2022-09-12 00:08:24 Matthieu Clemenceau bug added subscriber Ubuntu Foundations Team
2022-09-12 19:26:36 Nick Rosbrook tags foundations-todo fr-2514 sts verification-done-focal verification-needed verification-needed-jammy foundations-todo fr-2514 sts verification-done verification-done-focal verification-done-jammy
2022-09-15 15:31:42 Matthieu Clemenceau removed subscriber Ubuntu Foundations Team
2022-09-15 15:31:48 Matthieu Clemenceau bug added subscriber Ubuntu Foundations Bugs
2022-09-22 09:08:57 Launchpad Janitor systemd (Ubuntu Jammy): status Fix Committed Fix Released
2022-09-22 09:09:22 Łukasz Zemczak removed subscriber Ubuntu Stable Release Updates Team
2022-09-22 09:15:24 Launchpad Janitor systemd (Ubuntu Focal): status Fix Committed Fix Released
2022-09-29 15:51:37 Nick Rosbrook tags foundations-todo fr-2514 sts verification-done verification-done-focal verification-done-jammy fr-2514 sts verification-done verification-done-focal verification-done-jammy
2022-10-11 09:05:34 Lukas Märdian netplan.io (Ubuntu): status Confirmed Invalid
2022-10-11 09:05:36 Lukas Märdian netplan.io (Ubuntu Focal): status New Invalid
2022-10-11 09:05:37 Lukas Märdian netplan.io (Ubuntu Jammy): status New Invalid