Hi Ed,
I looked into this, and the issue only happens if such interfaces (with state DOWN and NO-CARRIER) are managed by systemd-networkd (check with 'networkctl list').
Per systemd-networkd-wait-online.service man page [1]:
'By default, it will wait for all links it is aware of and which are managed by systemd-networkd.service(8) to be fully configured or failed,'
[1] https://www.freedesktop.org/software/systemd/man/systemd-networkd-wait-online.service.html
The issue can be reproduced in a VM with a NIC configured in libvirt XML as "" that is managed by netplan. Steps below.
Hope this helps,
Mauricio
---
Create a VM (bionic):
---
$ uvt-simplestreams-libvirt sync release=bionic arch=amd64
$ uvt-kvm create --cpu 2 --memory 2048 --disk 4 --password password bionic release=bionic arch=amd64
$ uvt-kvm wait bionic
Give it an ethernet interface with link down:
---
$ virsh edit bionic
...
...
Re-start the VM:
---
$ virsh shutdown bionic
$ virsh start bionic
$ uvt-kvm wait bionic
$ uvt-kvm ssh bionic
Check that systemd-networkd-wait-online.service is happy with 'ens3' only:
---
By default, only the 'ens3' interface is configured in netplan
(thus managed by systemd-networkd, default renderer in netplan).
$ ip -o l
1: lo: mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000\ link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: ens3: mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000\ link/ether 52:54:00:b8:ff:f1 brd ff:ff:ff:ff:ff:ff
3: ens7: mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000\ link/ether 52:54:00:fb:c6:b6 brd ff:ff:ff:ff:ff:ff
$ grep -r ens[0-9]: /etc/netplan
/etc/netplan/50-cloud-init.yaml: ens3:
$ ls -1 /run/systemd/network/*.network
/run/systemd/network/10-netplan-ens3.network
Notice that the 'ens7' interface (other/new) SETUP status is 'unmanaged':
$ sudo networkctl list
IDX LINK TYPE OPERATIONAL SETUP
1 lo loopback carrier unmanaged
2 ens3 ether routable configured
3 ens7 ether off unmanaged
3 links listed.
Thus, despite being 'state DOWN', systemd-networkd-wait-online.service doesn't care, and succeeds:
$ systemctl status systemd-networkd-wait-online.service | grep Process
Process: 611 ExecStart=/lib/systemd/systemd-networkd-wait-online (code=exited, status=0/SUCCESS)
Check that systemd-networkd-wait-online.service gets unhappy with 'ens7' too:
---
If you just configure 'ens7' in netplan, even without setting/getting IP:
$ cat <
Job for systemd-networkd-wait-online.service failed because the control process exited with error code.
See "systemctl status systemd-networkd-wait-online.service" and "journalctl -xe" for details.
real 2m0.260s
user 0m0.012s
sys 0m0.011s
$ systemctl status systemd-networkd-wait-online.service
● systemd-networkd-wait-online.service - Wait for Network to be Configured
...
Process: 1215 ExecStart=/lib/systemd/systemd-networkd-wait-online (code=exited, status=1/FAILURE)
Main PID: 1215 (code=exited, status=1/FAILURE)
...
Feb 12 00:01:08 bionic systemd-networkd-wait-online[1215]: Event loop failed: Connection timed out
Feb 12 00:01:08 bionic systemd[1]: systemd-networkd-wait-online.service: Main process exited, code=exited, status=1/FAILURE
Feb 12 00:01:08 bionic systemd[1]: systemd-networkd-wait-online.service: Failed with result 'exit-code'.
Feb 12 00:01:08 bionic systemd[1]: Failed to start Wait for Network to be Configured.
Also, notice that the NO-CARRIER bit only showed up now, presumably after systemd-networkd set the bit UP:
$ ip -o l
1: lo: mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000\ link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: ens3: mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000\ link/ether 52:54:00:b8:ff:f1 brd ff:ff:ff:ff:ff:ff
3: ens7: mtu 1500 qdisc fq_codel state DOWN mode DEFAULT group default qlen 1000\ link/ether 52:54:00:fb:c6:b6 brd ff:ff:ff:ff:ff:ff
ubuntu@bionic:~$