Hi Ed,

I looked into this, and the issue only happens if such interfaces (with state DOWN and NO-CARRIER) are managed by systemd-networkd (check with 'networkctl list').

Per systemd-networkd-wait-online.service man page [1]:

'By default, it will wait for all links it is aware of and which are managed by systemd-networkd.service(8) to be fully configured or failed,'

[1] https://www.freedesktop.org/software/systemd/man/systemd-networkd-wait-online.service.html

The issue can be reproduced in a VM with a NIC configured in libvirt XML as "<interface type='ethernet'><link state=down'/></interface>" that is managed by netplan. Steps below.

Hope this helps,
Mauricio

---

Create a VM (bionic):
---

$ uvt-simplestreams-libvirt sync release=bionic arch=amd64

$ uvt-kvm create --cpu 2 --memory 2048 --disk 4 --password password bionic release=bionic arch=amd64
$ uvt-kvm wait bionic


Give it an ethernet interface with link down:
---

$ virsh edit bionic
...
    <interface type='ethernet'>
       <model type='virtio'/>
       <link state='down'/>
    </interface>
...


Re-start the VM:
---

$ virsh shutdown bionic
$ virsh start bionic

$ uvt-kvm wait bionic
$ uvt-kvm ssh bionic

Check that systemd-networkd-wait-online.service is happy with 'ens3' only:
---

By default, only the 'ens3' interface is configured in netplan
(thus managed by systemd-networkd, default renderer in netplan).

$ ip -o l
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000\    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000\    link/ether 52:54:00:b8:ff:f1 brd ff:ff:ff:ff:ff:ff
3: ens7: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000\    link/ether 52:54:00:fb:c6:b6 brd ff:ff:ff:ff:ff:ff

$ grep -r ens[0-9]: /etc/netplan
/etc/netplan/50-cloud-init.yaml:        ens3:

$ ls -1 /run/systemd/network/*.network
/run/systemd/network/10-netplan-ens3.network

Notice that the 'ens7' interface (other/new) SETUP status is 'unmanaged':

$ sudo networkctl list
IDX LINK             TYPE               OPERATIONAL SETUP     
  1 lo               loopback           carrier     unmanaged 
  2 ens3             ether              routable    configured
  3 ens7             ether              off         unmanaged 

3 links listed.

Thus, despite being 'state DOWN', systemd-networkd-wait-online.service doesn't care, and succeeds:

$ systemctl status systemd-networkd-wait-online.service | grep Process
  Process: 611 ExecStart=/lib/systemd/systemd-networkd-wait-online (code=exited, status=0/SUCCESS)


Check that systemd-networkd-wait-online.service gets unhappy with 'ens7' too:
---

If you just configure 'ens7' in netplan, even without setting/getting IP:

$ cat <<EOF | sudo tee /etc/netplan/ens7.yaml
network:
  ethernets:
    ens7:
      match:
        macaddress: 52:54:00:fb:c6:b6
      set-name: ens7
  version: 2
EOF

$ sudo netplan apply

$ grep -r ens[0-9]: /etc/netplan
/etc/netplan/ens7.yaml:    ens7:
/etc/netplan/50-cloud-init.yaml:        ens3:

$ ls -1 /run/systemd/network/*.network
/run/systemd/network/10-netplan-ens3.network
/run/systemd/network/10-netplan-ens7.network

Now systemd-networkd-wait-online.service cares about it (SETUP status is 'configuring'):

$ sudo networkctl list
IDX LINK             TYPE               OPERATIONAL SETUP     
  1 lo               loopback           carrier     unmanaged 
  2 ens3             ether              routable    configured
  3 ens7             ether              no-carrier  configuring

3 links listed.

And sure enough, systemd-networkd-wait-online.service now times out:

$ time sudo systemctl restart systemd-networkd-wait-online.service 
<... wait ...>
Job for systemd-networkd-wait-online.service failed because the control process exited with error code.
See "systemctl status systemd-networkd-wait-online.service" and "journalctl -xe" for details.

real	2m0.260s
user	0m0.012s
sys	0m0.011s

$ systemctl status systemd-networkd-wait-online.service
● systemd-networkd-wait-online.service - Wait for Network to be Configured
...
  Process: 1215 ExecStart=/lib/systemd/systemd-networkd-wait-online (code=exited, status=1/FAILURE)
 Main PID: 1215 (code=exited, status=1/FAILURE)
...
Feb 12 00:01:08 bionic systemd-networkd-wait-online[1215]: Event loop failed: Connection timed out
Feb 12 00:01:08 bionic systemd[1]: systemd-networkd-wait-online.service: Main process exited, code=exited, status=1/FAILURE
Feb 12 00:01:08 bionic systemd[1]: systemd-networkd-wait-online.service: Failed with result 'exit-code'.
Feb 12 00:01:08 bionic systemd[1]: Failed to start Wait for Network to be Configured.

Also, notice that the NO-CARRIER bit only showed up now, presumably after systemd-networkd set the bit UP:

$ ip -o l
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000\    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000\    link/ether 52:54:00:b8:ff:f1 brd ff:ff:ff:ff:ff:ff
3: ens7: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN mode DEFAULT group default qlen 1000\    link/ether 52:54:00:fb:c6:b6 brd ff:ff:ff:ff:ff:ff
ubuntu@bionic:~$