replacement of ifupdown with netplan needs integration for /etc/network/if{up,down}.d scripts

[Scott Moser]

when network is configured with ifupdown, scripts in /etc/network/ifup.d/ were called on network being brought up and /etc/network/ifdown.d were called on network being brought down.

Any packages that shipped these hooks need to be verified to have the same functionality under a netplan configured system.

# binpkgs=$(apt-file search /etc/network/if-up | sed 's,: .*,,' | sort -u)
# for i in $binpkgs; do
  src=$(apt-cache show $i | awk '$1 == "Source:" { print $2; exit(0); }');
  [ -z "$src" ] && src="$i"; echo $src; done | sort -u

aiccu
aoetools
avahi
bind9
chrony
clamav
controlaula
epoptes
ethtool
guidedog
htpdate
ifenslave
ifmetric
ifupdown-extra
ifupdown-multi
ifupdown-scripts-zg2
isatapd
lprng
miredo
mythtv-backend
nss-pam-ldapd
ntp
openntpd
openresolv
openssh
openvpn
postfix
quicktun
resolvconf
sendmail
shorewall-init
sidedoor
slrn
tinc
ubuntu-fan
ucarp
uml-utilities
uruk
vlan
vzctl
wide-dhcpv6
wpa

Related bugs:
 * bug 1718227: replacement of ifupdown with netplan needs integration for /etc/network/if{up,down}.d scripts
 * bug 1713803: replacement of resolvconf with systemd needs integration
 * bug 1717983: replacement of isc-dhcp-client with with systemd-networkd for dhclient needs integration

ProblemType: Bug
DistroRelease: Ubuntu 17.10
Package: netplan (not installed)
ProcVersionSignature: Ubuntu 4.12.0-11.12-generic 4.12.5
Uname: Linux 4.12.0-11-generic x86_64
NonfreeKernelModules: zfs zunicode zavl zcommon znvpair
ApportVersion: 2.20.7-0ubuntu1
Architecture: amd64
CurrentDesktop: GNOME
Date: Tue Sep 19 10:53:08 2017
EcryptfsInUse: Yes
InstallationDate: Installed on 2015-07-23 (789 days ago)
InstallationMedia: Ubuntu 15.10 "Wily Werewolf" - Alpha amd64 (20150722.1)
ProcEnviron:
 TERM=xterm-256color
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: plan
UpgradeStatus: No upgrade log present (probably fresh install)

[Lars Düsing (lars.duesing)]

aiccu package should be taken out of distribution due to closing of sixxs.net - there is no running tunnel broker any more.

[Vagrant Cascadian (vagrantc)]

$ apt show plan netplan | grep -A 3 Description

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

Description: X/Motif day planner
 Plan is a schedule planner based on X/Motif. It displays a month calendar
 similar to xcal, but every day box is large enough to show appointments in
 small print. By pressing on a day box, the appointments for that day can be
--
Description: network server for `plan'
 Plan is a schedule planner based on X/Motif.
 .
 Netplan adds to plan multiuser capability using an IP server.

This doesn't sound like the sort of thing to replace ifupdown... some more information about what this actually refers to would be helpful.

[Steve Langasek (vorlon)]

On Wed, Jan 31, 2018 at 06:27:35PM -0000, Vagrant Cascadian wrote:
> This doesn't sound like the sort of thing to replace ifupdown... some
> more information about what this actually refers to would be helpful.

The package name is 'nplan' due to conflict with pre-existing (and scarcely
relevant) software.

[Christian Ehrhardt  (paelzer)]

FYI: The discussion in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=889010 nicely outlined that we rely on those hooks in chrony.
That makes having them working critical for 18.04 to have it working correctly for late or changing network topology.

We (cyphermox and me) discussed how I could convert the chrony hooks but found not clear solution. It ended in the app needing to listen for netlink events, but we can't/won't do that for all the affected packages.
That said implementing sort of a generic solution a package can plug into gets more and more important.

[Crashbit (crashbit-gmail)]

I have a problem with tinc and netplan.

I've using Ubuntu 17.10 server. When I try to start tincd daemon with systemd, it doesn't start.
When start tinc daemon without systemd, it runs correct.

Fail:
crashbit@sun:~$ sudo systemctl start tinc
crashbit@sun:~$ sudo systemctl status tinc
● tinc.service - Tinc VPN
   Loaded: loaded (/lib/systemd/system/tinc.service; enabled; vendor preset: enabled)
   Active: active (exited) since Thu 2018-02-15 00:58:30 CET; 20s ago
  Process: 26868 ExecStart=/bin/true (code=exited, status=0/SUCCESS)
 Main PID: 26868 (code=exited, status=0/SUCCESS)
      CPU: 1ms

feb 15 00:58:30 sun systemd[1]: Starting Tinc VPN...
feb 15 00:58:30 sun systemd[1]: Started Tinc VPN.

crashbit@sun:~$ ifconfig | grep cephnet
crashbit@sun:~$

--------------------

Succes:

crashbit@sun:~$ sudo tincd -c /etc/tinc/cephnet/ -n cephnet
Both netname and configuration directory given, using the latter...
crashbit@sun:~$ ifconfig | grep cephnet
cephnet: flags=4305<UP,POINTOPOINT,RUNNING,NOARP,MULTICAST> mtu 1500
crashbit@sun:~$

-----------------------

crashbit@sun:/etc/netplan$ cat 01-netcfg.yaml
# This file describes the network interfaces available on your system
# For more information, see netplan(5).
network:
  version: 2
  renderer: networkd
  ethernets:
    enp129s0f0:
            dhcp4: false

  bridges:
    lxdbr0:
            interfaces: [enp129s0f0]
            addresses: [172.26.0.2/24]
            gateway4: 172.26.0.1
            parameters:
                    stp: false
                    forward-delay: 0
            nameservers:
                    addresses: [172.26.0.5]
crashbit@sun:/etc/netplan$

[Steve Langasek (vorlon)]

On Thu, Feb 15, 2018 at 12:06:04AM -0000, Crashbit wrote:
> I have a problem with tinc and netplan.

> I've using Ubuntu 17.10 server. When I try to start tincd daemon with systemd, it doesn't start.
> When start tinc daemon without systemd, it runs correct.

Then that doesn't sound related to netplan. You should file a separate bug
against tinc.

[Christian Ehrhardt  (paelzer)]

Any update on the integration of networkd-dispatcher or a similar technology to allow the dependent packages to use that?

[Launchpad Janitor (janitor)]

This bug was fixed in the package ubuntu-fan - 0.12.10

---------------
ubuntu-fan (0.12.10) bionic; urgency=medium

  * fan-net: support systemd-networkd controlled interfaces (LP: #1718227)
  * fanctl: Fix net stop internal command (LP: #1744296)
  * fan-net: Harden networkctl usage (LP: #1718227)
  * ifupdown/ubuntu-fan: Test for missing fanctl (LP: #1742712)

 -- Stefan Bader <email address hidden> Fri, 16 Mar 2018 11:47:46 +0100

[Andreas Hasenack (ahasenack)]

bind9 in bionic at least detects nic changes and does not need to be reloaded.

Interface being removed:
Mar 28 13:59:04 bionic-bind9 named[7098]: no longer listening on 192.168.122.138#53

Interface being added:
Mar 28 13:59:04 bionic-bind9 named[7098]: listening on IPv4 interface ens7, 192.168.122.186#53

Removed again:
Mar 28 14:00:40 bionic-bind9 named[7098]: no longer listening on 192.168.122.186#53
Mar 28 14:00:40 bionic-bind9 systemd-networkd[8515]: ens7: Lost carrier
Mar 28 14:00:40 bionic-bind9 systemd-networkd[8515]: ens7: DHCP lease lost

The file the package has in /etc/network/if-up.d/bind9 is a noop. We can drop it, but that would add a delta with debian. I think it's not necessary to take any action.

How would you prefer the bind9 bug task be handled in this bug, given the above?

[Steve Langasek (vorlon)]

Thanks, closing the bind9 task as invalid.

[Steve Langasek (vorlon)]

vlan support is native in netplan, no need for the vlan package to integrate with it.

[Steve Langasek (vorlon)]

ifenslave is used to configure bonds, which is supported natively in netplan. No integration required.

[Steve Langasek (vorlon)]

ifmetric is a package that sets route metrics - which seems awfully pointless, 'metric' is already supported natively by ifupdown and by netplan. This does not need integration.

(The only possible utility is that ifmetric lets you set metrics on the route for the local network, which netplan does not; but in the unlikely event that this is useful, it should be implemented by extending netplan.)

[Steve Langasek (vorlon)]

ifupdown-scripts-zg2 no longer exists in Ubuntu as of bionic.

[Steve Langasek (vorlon)]

vzctl is a set of tools for OpenVZ, a very early container technology which I think it's fair to say is obsoleted by lxd. For anyone running openvz, it would still matter to have vzctl integration with networkd, but I think this would be a low priority to fix.

[Steve Langasek (vorlon)]

openresolv is an alternative implementation of resolvconf. If you are using netplan, you should be using systemd-resolved; not resolvconf.

[Andreas Hasenack (ahasenack)]

clamav explicitly mentions ifupdown in a debconf question regarding how the user would like freshclam (updating the virus signatures db) to work. One of the options is a ifupdown hook attached to what the user calls "the internet interface". That option should be removed, or better yet, migrated to a systemd "something" attached to that specific interface.

[Alkis Georgopoulos (alkisg)]

For the maintainers of the affected packages to be able to help in this issue,
addressing the main question already stated above would be most helpful:

To support netplan, do we have to implement netlink events listeners ourselves?
I'm not sure all maintainers are willing to do that.
Or is there another package that provides this functionality that we could rely on?
E.g. NetworkManager does have a dispatcher.d directory that we can use, but does Ubuntu ship something similar (networkd-dispatcher?) in systems that don't use NetworkManager?

[Steve Langasek (vorlon)]

On Thu, Mar 29, 2018 at 06:27:18PM -0000, Alkis Georgopoulos wrote:

> For the maintainers of the affected packages to be able to help in this
> issue, addressing the main question already stated above would be most
> helpful:

> To support netplan, do we have to implement netlink events listeners ourselves?
> I'm not sure all maintainers are willing to do that.

> Or is there another package that provides this functionality that we could
> rely on? E.g. NetworkManager does have a dispatcher.d directory that we
> can use, but does Ubuntu ship something similar (networkd-dispatcher?) in
> systems that don't use NetworkManager?

We are looking at integrating networkd-dispatcher for 18.04.

Many of these package tasks may become SRU material after 18.04 releases,
due to the timing.

[Christian Ehrhardt  (paelzer)]

I heard people talk about it, but realized the tracker is missing a Task for openvswitch:
/etc/network/if-post-down.d/openvswitch
/etc/network/if-pre-up.d/openvswitch

IIRC all the discussions correctly that was one of the harder cases due to "Pre" not really being a defined thing anymore.

The question is how much OVS relies on that to work as PRE, or if it can be later (or totally ignored).

I'll ping Jamespage about this for his OVS experience.

P.S. sorry if I duplicate some work here, but I can't find it in the bug at all, so better twice than missed.

[Christian Ehrhardt  (paelzer)]

Lets break this into use cases in Bionic:

I was not sure who should win in each case.
We might either want the clear "order" chrony > ntp > openntp > systemd-timesyncd
Or we might want a "last installed" approach, but that is hard as upgrades to not count here only real "install". What would "--reinstall" be in these cases?
Maybe we should stick with the clear order, that at least seems more deterministic.
Cases 4-6 try to cover testing that order invariancy.

This is an "ideal world" approach, not sure if we can achieve that in the short term.
After the "=>" assignment is the service that should run (and only this one).

0. default install - systemd-timesyncd

1. default install - install chrony => Chrony
1b. - remove chrony => systemd-timesyncd

2. default install - install ntp => NTP
2b. - remove ntp => systemd-timesyncd

3. default install - install openntp => openntp
3b. - remove openntp => systemd-timesyncd

4. default install - install ntp, install chrony => Chrony
4b. remove chrony => NTP
4c. remove NTP => systemd-timesyncd

5. default install - install chrony, install NTP => Chrony
5b. remove Chrony => NTP
5c. remove NTP => systemd-timesyncd

6. default install - install openntp => openntp
6b. install NTP => NTP
6c. install chrony => chrony
6d. remote NTP & Chrony => openntp
6e. remove openntp => systemd-timesyncd

7. xenial with ntp - upgrade to B => NTP

8. xenial with ntp - upgrade to B, install chrony => Chrony

9. xenial with ntp - upgrade to B, remove NTP => systemd-timesyncd

10. xenial without ntp - upgrade to B => systemd-timesyncd

[Christian Ehrhardt  (paelzer)]

Nice summary, but wrong bug - sorry for the noise here :-/

[Christian Ehrhardt  (paelzer)]

Tested on chrony which has a NetworkManager dispatch script that also works as a hook for networkd-dispatcher.

Works fine by just dropping the links for now.
Changes visible when these hooks are in place

1. when sources get unreachable it detects offlining immediately (instead of trying all the time)
2. when a network drops but sources stay reachable nothing happens (no accidental offline)
3. when sources are offline and network is attached without connecting to anything they stay offline
4. when sources are offline and a connecting network is attached they all become online immediately
5. when a network is lost that was connecting to just to some sources only those get set offline.

P.S. most of these cases can be well tested with virsh attach-device / detach-device with multiple network cards (one that connects to network and one that does not for example)

The biggest issue is that reusing that is very nice, but OTOH dangerous if it gets NM only code.
I'll start discussing that upstream before using it in any way.
That happened post 3.2 in
  b563048 "examples: ignore non-up/down events in nm-dispatcher script"

I'm now doing:
1. discussing upstream how we want to do it
2. bring that upstream for networkd-dispatcher
3. backport the change to Bionic chrony package
4. place the files to trigger the callbacks

[Christian Ehrhardt  (paelzer)]

Fix for chrony (following networkd-dispatcher change in bug 1765152) uploaded to bionic-unapproved as 3.2-4ubuntu4

[Launchpad Janitor (janitor)]

This bug was fixed in the package chrony - 3.2-4ubuntu4

---------------
chrony (3.2-4ubuntu4) bionic; urgency=medium

  * d/postrm: re-establish systemd-timesyncd on removal (LP: #1764357)
  * Notify chrony to update sources in response to systemd-networkd
    events (LP: #1718227)
    - d/links: link dispatcher script to networkd-dispatcher events routable
      and off
    - d/control: set Recommends to networkd-dispatcher
    - d/p/lp-1718227-ignore-non-up-down-events-in-nm-dispatcher.patch
    - d/p/lp-1718227-nm-dispatcher-for-networkd.patch

 -- Christian Ehrhardt <email address hidden> Mon, 16 Apr 2018 17:04:06 +0200

[Steve Langasek (vorlon)]

controlaula was not shipped in 18.04. Invalid for trunk, wontfix for artful and earlier.

[Christian Ehrhardt  (paelzer)]

We handled chrony and ntp was demoted, so set ntp to Won't Fix in regard to this bug.

[Steve Langasek (vorlon)]

even if not the default ntp implementation, I think this is still a valid bug task and should be tracked as such.

[Launchpad Janitor (janitor)]

This bug was fixed in the package openssh - 1:7.9p1-1

---------------
openssh (1:7.9p1-1) unstable; urgency=medium

  * New upstream release (https://www.openssh.com/txt/release-7.9):
    - ssh(1), sshd(8): allow most port numbers to be specified using service
      names from getservbyname(3) (typically /etc/services; closes:
      #177406).
    - ssh(1): allow the IdentityAgent configuration directive to accept
      environment variable names. This supports the use of multiple agent
      sockets without needing to use fixed paths.
    - sshd(8): support signalling sessions via the SSH protocol. A limited
      subset of signals is supported and only for login or command sessions
      (i.e. not subsystems) that were not subject to a forced command via
      authorized_keys or sshd_config.
    - ssh(1): support "ssh -Q sig" to list supported signature options.
      Also "ssh -Q help" to show the full set of supported queries.
    - ssh(1), sshd(8): add a CASignatureAlgorithms option for the client and
      server configs to allow control over which signature formats are
      allowed for CAs to sign certificates. For example, this allows
      banning CAs that sign certificates using the RSA-SHA1 signature
      algorithm.
    - sshd(8), ssh-keygen(1): allow key revocation lists (KRLs) to revoke
      keys specified by SHA256 hash.
    - ssh-keygen(1): allow creation of key revocation lists directly from
      base64-encoded SHA256 fingerprints. This supports revoking keys using
      only the information contained in sshd(8) authentication log messages.
    - ssh(1), ssh-keygen(1): avoid spurious "invalid format" errors when
      attempting to load PEM private keys while using an incorrect
      passphrase.
    - sshd(8): when a channel closed message is received from a client,
      close the stderr file descriptor at the same time stdout is closed.
      This avoids stuck processes if they were waiting for stderr to close
      and were insensitive to stdin/out closing (closes: #844494).
    - ssh(1): allow ForwardX11Timeout=0 to disable the untrusted X11
      forwarding timeout and support X11 forwarding indefinitely.
      Previously the behaviour of ForwardX11Timeout=0 was undefined.
    - sshd(8): when compiled with GSSAPI support, cache supported method
      OIDs regardless of whether GSSAPI authentication is enabled in the
      main section of sshd_config. This avoids sandbox violations if GSSAPI
      authentication was later enabled in a Match block.
    - sshd(8): do not fail closed when configured with a text key revocation
      list that contains a too-short key.
    - ssh(1): treat connections with ProxyJump specified the same as ones
      with a ProxyCommand set with regards to hostname canonicalisation
      (i.e. don't try to canonicalise the hostname unless
      CanonicalizeHostname is set to 'always').
    - ssh(1): fix regression in OpenSSH 7.8 that could prevent public-key
      authentication using certificates hosted in a ssh-agent(1) or against
      sshd(8) from OpenSSH <7.8 (LP: #1790963).
    - All: support building against the openssl-1.1 API (releases 1.1.0g and
      later). The openssl-1.0 AP...

[Christian Ehrhardt  (paelzer)]

Done in Chrony and setting to Won't Fix (to manage expectations) for NTP as it is universe only and if people care they can drop in networkd-dispatcher files to get it working (open for community contribution, but main will rely on chrony).

[Rafael David Tinoco (rafaeldtinoco)]

An example of dns update after putting:

up /etc/openvpn/update-resolv-conf
down /etc/openvpn/update-resolv-conf

in the openvpn .config file:

Wed Dec 11 15:04:25 2019 /etc/openvpn/update-resolv-conf tun0 1500 1558 10.172.67.194 255.255.192.0 init

update-resolv-conf uses:

[ -x /sbin/resolvconf ] || exit 0

so I guess it needs a systemd-resolved.service integration instead.

[Rafael David Tinoco (rafaeldtinoco)]

For openvpn + systemd-resolve:

With "up / down" openvpn config file commands you can wrap "systemd-resolve --set-dns=XXX" and update the given DNS servers.

[Simon Déziel (sdeziel)]

On 2019-12-11 12:33 p.m., Rafael David Tinoco wrote:
> For openvpn + systemd-resolve:
>
> With "up / down" openvpn config file commands you can wrap "systemd-
> resolve --set-dns=XXX" and update the given DNS servers.

There's a package for that: openvpn-systemd-resolved

[Simon Déziel (sdeziel)]

> For openvpn + systemd-resolve:
>
> With "up / down" openvpn config file commands you can wrap "systemd-
> resolve --set-dns=XXX" and update the given DNS servers.

There's a package for that: openvpn-systemd-resolved

[Tessa (unit3)]

just tried to use netplan on a public facing server, discovered there's no `ucarp` integration, and discovered this bug.

is there any ETA on these outstanding items? network HA is a nightmare without CARP, this feels like essential server functionality.

[Sergio Durigan Junior (sergiodj)]

FWIW, I submitted a Merge Request against the Debian nss-pam-ldapd here:

https://salsa.debian.org/debian/nss-pam-ldapd/-/merge_requests/2

Hopefully it will be accepted & merged into Ubuntu soon. If not, I will submit an MP to add this change as a delta to our package.

[Sergio Durigan Junior (sergiodj)]

I've also submitted a Merge Request against the Debian clamav package:

https://salsa.debian.org/clamav-team/clamav/-/merge_requests/4

Hopefully it will be accepted & merged into Ubuntu soon. If not, I will submit an MP to add this change as a delta to our package.

[Sergio Durigan Junior (sergiodj)]

MR for the Debian postfix package also submitted:

https://salsa.debian.org/postfix-team/postfix-dev/-/merge_requests/13

[Launchpad Janitor (janitor)]

This bug was fixed in the package postfix - 3.5.13-1ubuntu2

---------------
postfix (3.5.13-1ubuntu2) jammy; urgency=medium

  * Support networkd-dispatcher. (LP: #1718227) (Debian BTS #999867)
    - d/postfix.dirs: Add usr/lib/networkd-dispatcher/{routable,off}.d.
    - d/rules: Install debian/ip-{up,down}.d scripts into
      usr/lib/networkd-dispatcher/{routable,off}.d, respectively.

 -- Sergio Durigan Junior <email address hidden> Wed, 17 Nov 2021 18:03:41 -0500

[Launchpad Janitor (janitor)]

This bug was fixed in the package nss-pam-ldapd - 0.9.12-1

---------------
nss-pam-ldapd (0.9.12-1) unstable; urgency=medium

  [ Arthur de Jong ]
  * New upstream release:
    - allow explicitly configuring an empty search base (for LDAP servers
      that support that)
    - support LDAP attributes with minus characters in attribute mapping
      expressions
    - add tls_reqsan, tls_crlfile and tls_crlcheck options (thanks
      Sebastien Blavier)
    - support generating ldaps:// URIs from DNS SRV records for port 389 by
      using DNSLDAPS in the uri option
    - prefer the first URI listed in nslcd.conf after reconnecting after
      idle_timelimit
    - fix handling of pam_authc_ppolicy no
    - fix debug logging of ldap timeout values
    - documentation improvements (thanks Filip Dvorak and Benedict Reuschling)
    - add pam_authc_ppolicy support to pynslcd
    - fix Python 3 compatibility in chsh.ldap
    - fix for running pynslcd without the uid option
    - partial support for running tests with slapd 2.5 (thanks Ryan Tandy)
  * Switch to using /run/nslcd (fixes lintian warnings)
  * Update debian/upstream/metadata
  * Drop lintian override
  * Use debhelper-compat instead of debian/compat
  * Update debhelper compatibility level to 13
  * Specify Rules-Requires-Root: no
  * Update debian/watch version
  * Upgrade to standards-version 4.6.0 (no changes needed)
  * Include more output on test suite failure
  * Add lintian override for pam_ldap(8) manual page warning
  * Run tests with netbase installed for /etc/services and /etc/protocols

  [ Sergio Durigan Junior ]
  * Support networkd-dispatcher notifications
    - d/nslcd.if-up: Adjust script to make it compatible with
      networkd-dispatcher
    - d/nslcd.links: Install the proper nslcd script inside the
      /usr/lib/networkd-dispatcher/routable.d/ directory (LP: #1718227)

 -- Arthur de Jong <email address hidden> Sat, 20 Nov 2021 16:10:03 +0100

[Lukas Märdian (slyon)]

Transitioning this bug report to the netplan.io package. "nplan" is no more. There's also a duplicate at LP: #1664818

[Launchpad Janitor (janitor)]

This bug was fixed in the package clamav - 0.103.8+dfsg-0ubuntu2

---------------
clamav (0.103.8+dfsg-0ubuntu2) mantic; urgency=medium

  * Extend ifupdown script to support networkd-dispatcher.
    - d/clamav-freshclam-ifupdown: Modernize some parts of
      the script. Implement support for networkd-dispatcher.
    - d/clamav-freshclam.links: Install the
      clamav-freshclam-ifupdown script inside the proper
      /usr/lib/networkd-dispatcher/{off,routable}.d/
      directories.
    (LP: #1718227)

 -- Sergio Durigan Junior <email address hidden> Fri, 12 May 2023 15:58:29 -0400