initramfs clevis no carrier after 1s, no retry

[Thorsten Bonhagen]

[Impact]
Several users have complained of issues with bringing up networking in the initramfs, which causes automatic LUKS volume unlocking via a network server to fail. The original report for this bug is one example, others can be found in the upstream bug:
  https://github.com/latchset/clevis/issues/145

The symptoms vary - see the commit message in [Fix] below for an enumerated list - but the general root cause is that we currently fail to wait for device enumeration to complete before trying to configure an interface.

[Test Case]
Boot a system with a slow-to-enumerate network device and confirm that the system is able to use it to automatically decrypt a LUKS root device.

[Fix]
https://github.com/latchset/clevis/commit/f670383c276d6a61e165ff9e498e19271f8e168c

[Regression Potential]
The biggest risk I see is if a user was somehow benefiting from some unknown side-effect of the existing behavior. The existing "eth_check()" code brings up each interface and checks for a carrier. The new code leaves it to configure_networking() to bring up the device when its actually time to configure it.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[dann frazier (dannf)]

Hi @Thorsten! This sounds like an issue we've been discussing upstream[1], and I think I have an approach that might work for you. It requires changes to both clevis and initramfs-tools. I've prepared test packages in a PPA[2]. Would it be possible for you to test these?

Note that you will need to specify the network interface to use on the kernel command line using the ip= syntax - for example: ip=:::::enp5s0:dhcp

[1] https://github.com/latchset/clevis/issues/145
[2] https://launchpad.net/~dannf/+archive/ubuntu/clevis

[Launchpad Janitor (janitor)]

This bug was fixed in the package clevis - 13-2ubuntu1

---------------
clevis (13-2ubuntu1) groovy; urgency=medium

  * initramfs: Fix parsing of interface names when bringing the network
    back down in local-bottom, which also avoids a mess of "ip: can't find
    device '/sys/class/net/$iface'" errors on the console. LP: #1896294.
  * initramfs: Warn users with multiple interfaces that they should consider
    specifying an 'ip=' parameter for reliable operation. LP: #1896289.
    As a side-effect, also fix interface parsing while bringing links
    up. LP: #1873593.
  * initramfs: Wait for interface to appear before attempting configuration.
    LP: #1873914.

 -- dann frazier <email address hidden> Mon, 21 Sep 2020 11:04:00 -0600

[Brian Murray (brian-murray)]

Hello Thorsten, or anyone else affected,

Accepted clevis into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/clevis/12-1ubuntu2.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Brian Murray (brian-murray)]

Hello Thorsten, or anyone else affected,

Accepted clevis into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/clevis/8-1ubuntu0.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[dann frazier (dannf)]

= focal verification =
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... Volume group "ubuntu-vg" not found
  Cannot process volume group ubuntu-vg
Please unlock disk dm_crypt-0: Begin: clevis: Waiting for interface enx00249b194b8e to become available ... done.
IP-Config: enx00249b194b8e hardware address 00:24:9b:19:4b:8e mtu 1500 DHCP RARP
IP-Config: no response after 2 secs - giving up
IP-Config: enx00249b194b8e hardware address 00:24:9b:19:4b:8e mtu 1500 DHCP RARP
IP-Config: enx00249b194b8e complete (dhcp from 192.168.1.1):
 address: 192.168.1.218 broadcast: 192.168.1.255 netmask: 255.255.255.0
 gateway: 192.168.1.1 dns0 : 192.168.1.1 dns1 : 0.0.0.0
 host : xps13
 domain : dannf
 rootserver: 192.168.1.1 rootpath:
 filename :

cryptsetup: dm_crypt-0: set up successfully
done.

[dann frazier (dannf)]

= bionic verification =
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... WARNING: Failed to connect to lvmetad. Falling back to device scanning.
  Volume group "clevis-bionic-vg" not found
  Cannot process volume group clevis-bionic-vg
  WARNING: Failed to connect to lvmetad. Falling back to device scanning.
  Volume group "clevis-bionic-vg" not found
  Cannot process volume group clevis-bionic-vg
Please unlock disk vda3_crypt: Begin: clevis: Waiting for interface enx00249b194b8e to become available ... done.
IP-Config: enx00249b194b8e hardware address 00:24:9b:19:4b:8e mtu 1500 DHCP RARP
IP-Config: no response after 2 secs - giving up
IP-Config: enx00249b194b8e hardware address 00:24:9b:19:4b:8e mtu 1500 DHCP RARP
IP-Config: enx00249b194b8e complete (dhcp from 192.168.1.1):
 address: 192.168.1.218 broadcast: 192.168.1.255 netmask: 255.255.255.0
 gateway: 192.168.1.1 dns0 : 192.168.1.1 dns1 : 0.0.0.0
 host : xps13
 domain : dannf
 rootserver: 192.168.1.1 rootpath:
 filename :

[ 8.363381] NET: Registered protocol family 38
  WARNING: Failed to connect to lvmetad. Falling back to device scanning.
  Reading all physical volumes. This may take a while...
  Found volume group "clevis-bionic-vg" using metadata type lvm2
  WARNING: Failed to connect to lvmetad. Falling back to device scanning.
  2 logical volume(s) in volume group "clevis-bionic-vg" now active
cryptsetup (vda3_crypt): set up successfully
done.

[Launchpad Janitor (janitor)]

This bug was fixed in the package clevis - 8-1ubuntu0.2

---------------
clevis (8-1ubuntu0.2) bionic; urgency=medium

  * initramfs: Fix parsing of interface names when bringing the network
    back down in local-bottom, which also avoids a mess of "ip: can't find
    device '/sys/class/net/$iface'" errors on the console. LP: #1896294.
  * initramfs: Warn users with multiple interfaces that they should consider
    specifying an 'ip=' parameter for reliable operation. LP: #1896289.
    As a side-effect, also fix interface parsing while bringing links
    up. LP: #1873593.
  * initramfs: Wait for interface to appear before attempting configuration.
    LP: #1873914.
  * initramfs: Make network configuration as-needed. This functionality
    depends on the new clevis-luks-list command which is also backported.
    LP: #1896509. Requires new build-dep on asciidoctor in order to build
    the clevis-luks-list manpage.

 -- dann frazier <email address hidden> Wed, 14 Oct 2020 11:23:01 -0600

[Chris Halse Rogers (raof)]

The verification of the Stable Release Update for clevis has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package clevis - 12-1ubuntu2.2

---------------
clevis (12-1ubuntu2.2) focal; urgency=medium

  * initramfs: Fix parsing of interface names when bringing the network
    back down in local-bottom, which also avoids a mess of "ip: can't find
    device '/sys/class/net/$iface'" errors on the console. LP: #1896294.
  * initramfs: Warn users with multiple interfaces that they should consider
    specifying an 'ip=' parameter for reliable operation. LP: #1896289.
    As a side-effect, also fix interface parsing while bringing links
    up. LP: #1873593.
  * initramfs: Wait for interface to appear before attempting configuration.
    LP: #1873914.
  * initramfs: Make network configuration as-needed. This functionality
    depends on the new clevis-luks-list command which is also backported.
    LP: #1896509.

 -- dann frazier <email address hidden> Tue, 13 Oct 2020 17:00:47 -0600